Diagnostic accuracy of rapid antigen test for COVID-19 in an emergency department.

Use of antigen tests for the diagnosis of COVID-19 has become widespread. The aim of this study was to evaluate the diagnostic accuracy of the nasopharyngeal rapid antigen diagnostic (RAD) immunoassay LumiraDx UK in an Emergency Department (ED). All patients admitted to our ED between November 11 and December 8, 2020, and had both a RAD test and a real-time-reverse-transcription-polymerase-chain-reaction (RT-PCR) test were enrolled. RAD was considered as the index test and RT-PCR test was used as the reference standard. Sensitivity, specificity, negative and positive predictive values, and likelihood ratios were calculated with the 95% confidence interval. The sensitivity and specificity of RAD were 34.2% and 92.3%. Positive and negative likelihood ratios were 4.4 and 0.71. Our results demonstrate that the diagnostic accuracy of the LumiraDx RAD test is too low for routine use as a diagnostic method in the ED.

Introduction

The most used test to identify coronavirus disease 2019 (COVID-19) patients is the real-time reverse transcription-polymerase chain reaction (RT-PCR) for the detection of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) sequence in a nasal swab [1]. However, a time of 1 to 3 hours is needed to perform the RT-PCR test. Moreover, the sample has to be processed by personnel in a specific laboratory setting. To overcome these problems, some point-of-care diagnostic tests that detect viral antigens have been devised. Antigens tests are now frequently used, thanks to the rapidity of the results and the possibility of administering them at the point of care instead of in a laboratory.

Rapid antigen diagnostic (RAD) tests have quickly become widespread; and in some settings, the diagnostic accuracy of antigen tests has been considered sufficient to replace the RT-PCR [2]. However, no reliable data exist about the diagnostic accuracy of antigen tests in everyday clinical practice, particularly in the Emergency Department (ED).

Therefore, the aim of the present study was to assess the diagnostic accuracy of the nasopharyngeal RAD immunoassay LumiraDx compared to the RT-PCR test for the detection of SARS-CoV-2 infection in patients presenting to the ED.

Materials and methods

Study design

This study was an observational, cross-sectional investigation in patients presenting to the ED to evaluate the diagnostic accuracy of the LumiraDx RAD test compared to the RT-PCR test, which was used as the reference standard, for the detection of SARS-CoV-2 infection.

Patients

In the clinical practice of our ED, all patients that came to the ED and could be hospitalized were tested for SARS-CoV-2 infection by the LumiraDx RAD test. If a patient presented with symptoms of COVID-19 (cough, fever, dyspnea, sore throat, and/or loss of taste and smell) and had a positive chest X-ray, the patient was considered at high risk of COVID-19 and sorted into the COVID-suspected area. A positive chest X-ray was considered if the patient showed inflammatory findings such as interstitial thickening. The thickening of bronchovascular bundles alone was not sufficient to define a positive X-ray. In the case of asymptomatic patients (without symptoms suggesting COVID-19) with a negative chest X-ray and no history of contacts with COVID-19-positive individuals, the probability of SARS-CoV-2 infection was considered low and patients were sorted into the non-COVID area. In the case of a patient presenting with COVID-19 symptoms and a negative chest X-ray or no symptoms and a positive X-ray, the patient was considered as being at intermediate risk (and sorted into the COVID-suspected area). According to the Italian Minister of Health's indication, an RT-PCR test was performed for all patients, except for those at high risk of COVID-19 infection with a positive RAD test result.

All patients admitted to our ED between November 11 and December 8, 2020, and who were administered both the LumiraDx RAD test and the RT-PCR-based test were enrolled in this study. Patients who underwent only RAD or RT-PCR testing were excluded from this study.

Index test and reference standard

The RAD assay LumiraDx SARS-CoV-2 Ag Test (LumiraDx UK Ltd; Alloa, UK) was considered as the index test. The COVID-19 Plus RealAmp Kit (GeneFinderTM, Elitech, Torino, Italy) or Xpert XpressSARS-CoV-2 test (Cepheid s.r.l., Pero, Italy) were considered as the reference standard. PCR-test were used alternatively depending on their availability. The RNA-dependent RNA polymerase, nucleocapsid protein, and envelope protein genes are targeted by the first system; while only the nucleocapsid protein and envelope protein genes are targeted by the second.

Sampling for both tests was performed by trained nurses or physicians. The specimen was collected from both nostrils using a flocked swab. The RAD assay was performed immediately after sampling, according to the manufacturer's instructions. Briefly, the swab was placed and rotated in the extraction vial, and then three drops from the vial were placed into the test cassette. The test result was read and interpreted 12 minutes after the sample application by a trained nurse. For the PCR test, the specimen was inserted into a test tube avoiding contamination, packed, labeled, and sent to the laboratory. Both tests were performed within 1 hour from the previous one. Following the indications of the Italian Ministry of Health, in the case of a high pretest probability of COVID-19 and a positive RAD result, the RT-PCR test was not performed.

Statistical analysis

Data are expressed as the median and interquartile range. To calculate the diagnostic accuracy of the RAD test, the sensitivity, specificity, negative and positive predictive values, and likelihood ratios were calculated with their 95% confidence interval (CI). A sensitivity analysis was also performed considering RAD-positive samples from symptomatic patients as true positive.

To give a practical example of the change in probability of the disease, the posttest probability considering the median prevalence of infection in COVID-19-suspected patients as well as in patients not suspected of infection was calculated. Moreover, considering that an a priori posttest probability greater than 98% was sufficient to admit a patient to a COVID unit and a posttest probability lower than 2% was sufficient to admit a patient to a non-COVID unit, we calculated the pretest probability so that the RAD could be useful.

Results

A total of 234 patients, including 118 females and 116 males, were administered both the RAD and the RT-PCR tests, while 109 patients did not receive the RT-PCR because they were at high risk of infection and had a positive RAD test and were therefore excluded from this study. The median age of the patients was 72 years old. Sixty patients were triaged and hospitalized in the COVID-suspected area, and 174 patients were hospitalized in the non-COVID area. Both areas are in the ED and correspond to symptomatic and asymptomatic patients, as described in the materials and methods section.

Of the 234 patients, 13 (5%) tested positive by the RT-PCR and RAD tests, and 181 (77%) tested negative by both methods. Fifteen patients had a positive RAD test result but were negative by the RT-PCR test, while 25 patients first tested negative with the RAD test but then had a positive result with the molecular RT-PCR test. Data are summarized in Table 1 and Fig. 1 .

Table 1

RAD test results compared to PCR in Covid-suspected and non-covid area.

Number of samples with RAD test result shown
	True positives (n)	True negatives(n)	False positives(n)	False negatives(n)
Covid-suspected Area n = 60	11	29	0	20
Non-Covid Area n = 174	2	152	15	5

Fig. 1

For patients with high pre-test probability and positive RAD test, the PCR was not performed. All other patients were sorted into COVID-suspected and non-COVID areas, depending on probability of infection. PCR = polymerase chain reaction; RAD = rapid antigen diagnostic.

The overall sensitivity and specificity of the RAD test were 34 % (13/38) (95% CI: 19.6–51.3%) and 92 % (181/196) (95% CI: 87.6–95.6%), respectively. The positive likelihood ratio was 4 (95% CI: 2.3–8.6), while the negative likelihood ratio was 0.71 (95% CI: 0.5–0.9). According to the sensitivity analysis, considering RAD-positive samples from symptomatic patients as true positive, sensitivity would be 83% (122/147) (95% CI: 75.9−88.7%).

The mean prevalence of SARS-CoV-2 infection between October and December 2020 (during the second wave of the COVID-19 pandemic in Italy) in the COVID-suspected area was 70%; while in the non-COVID area, it was 6%. Since predictive values are modified by the prevalence of disease, the positive predictive value in a high-prevalence area was 91 % (95% CI: 84.4–95.2%), and the negative predictive value was 37 % (95% CI: 32.2–43.1%), while the positive predictive value in the low-prevalence area was 22 % (95% CI: 12.89–35.49%), and the negative predictive value was 96 % (95% CI: 94.57−96.52%).

To reach a posttest probability greater than 98%, the pretest probability should be 85%; while to reach a posttest probability less than 2%, the pretest probability should be 3%.

Discussion

Our results demonstrate that the diagnostic accuracy of the Lumira Dx RAD test in detecting SARS-CoV-2 infection is too low to allow it to be widely used as a diagnostic method in the ED even if, thanks to its specificity, the Italian Minister of Health recommended its use to confirm symptomatic COVID-19 cases. According to the sensitivity analysis, considering all RAD-positive samples as true positive (109 symptomatic patients that resulted positive at RAD test and did not perform RT-PCR), the sensitivity would be 83%. Such sensitivity in this setting could be problematic, since 2 out of 10 positive patients could be placed in the non-COVID area, thus exposing other patients to the infection and creating a risk of infection spread among all hospitalized patients. Although the RAD has several advantages compared to the RT-PCR test, i.e., the rapidity of the execution of the test and the point-of-care strategy above all, given its low diagnostic accuracy, it should only be used during an epidemic peak, when the pretest probability of infection in symptomatic patients is very high (in our case, greater than 85%). These results are in line with the indication of the Italian Ministry of Health that all symptomatic patients with a positive antigen test should be considered as positive. In contrast, it should not be used to rule out the infection in asymptomatic patients because, even if the specificity is 92%, in a low prevalence population (6%) the PPV could be as low as 22%. This means that for every positive patient that is really COVID-infected, there are 5 false-positive patients that could be placed in a COVID-suspected area.

Our study has several limitations that must be addressed. First, both the PCR test and the RAD test were performed only when there was a discrepancy between the clinically suspected diagnosis and the RAD test result. For this reason, the specificity and sensitivity could be over-and underestimated, respectively. Second, the RT-PCR test was used as the reference standard, but some authors have underlined its poor diagnostic performance [3], [4], [5], [6]. Third, the RAD test was used at the point of care and was performed by ED nurses and physicians instead of personnel from the laboratory. However, this characteristic is one of the most powerful advantages of the test.

In conclusion, this study demonstrates that the diagnostic accuracy of the RAD test is poor. We suggest only using the Lumira Dx RAD test in the ED for very high pretest probability patients during an epidemic peak.

Declaration of competing interest

All authors declare no conflict of interest.