Evaluation of Four Rapid Antigen Tests for Detection of SARS-CoV-2 virus.

OBJECTIVES: Considering the increasing, significant burden that coronavirus disease 2019 (COVID-19) imposes on the healthcare system, the need for simple, rapid, and affordable diagnostic tests to support the existing costly and demanding polymerase chain reaction (PCR) assay becomes required. This prospective diagnostic test accuracy study aims to evaluate the performance of four different COVID-19 rapid antigen tests compared to real-time reverse transcription PCR (rRT-PCR) between June and July 2020 to determine the feasibility of integrating these tests into the diagnostic algorithm in clinical settings.
METHODS: Swabs were collected from 306 patients and analyzed using rRT-PCR and antigen tests from four different providers.
RESULTS: The antigen tests' sensitivities were 65.8%, 69.8%, 64.0%, and 64.3% for the STANDARD™ Q COVID-19 Ag test, PCL COVID-19 Ag Rapid fluorescent immunoassay (FIA) test, BIOCREDIT COVID-19 Ag test, and Sofia SARS-CoV-2 antigen FIA test, respectively. Specificity was 94.1% for PCL COVID-19 Ag Rapid test and 100% for the other three assays. All assays showed a significant negative correlation between the reference rRT-PCR Ct values and Ag test results. Besides, sensitivities of the STANDARD™ Q COVID-19 Ag test, PCL COVID-19 Ag Rapid FIA test, and BIOCREDIT COVID-19 Ag test improved to ≥ 85% after exclusion of samples with PCR Ct values > 30.
CONCLUSIONS: The high specificity of the rapid antigen tests and other parameters like simplicity, rapidity, and affordability suggest that antigen tests are likely to be helpful if integrated and interpreted appropriately in stepwise diagnostic algorithms. Given the low sensitivity of 64.0-69.8% of the antigen tests, we recommend that clinically relevant negative results undergo further testing Ag to confirm or exclude a COVID-19 diagnosis. The OMJ is Published Bimonthly and Copyrighted 2021 by the OMSB.

Introduction

The emerging coronavirus disease 2019 (COVID-19), first identified in December 2019, spread worldwide and was declared a public health emergency of international concern and a pandemic in February and March 2020, respectively, by the World Health Organization (WHO).[1,2] COVID-19 is caused by a novel, enveloped RNA betacoronavirus that has a phylogenetic similarity to severe acute respiratory syndrome coronavirus (SARS-Co-V) and has been formally named severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).[3] Given the high transmissibility and the significant burden this pandemic caused on healthcare systems, timely and accurate diagnosis is considered key in its management. Early detection can assist in both appropriate management of cases and prompt application of infection control measures to reduce its transmission in the community.[4] Nucleic acid amplification tests (NAATs), such as real-time reverse transcription polymerase chain reaction (rRT-PCR) assays based on the detection of SARS-CoV-2 genetic material, have been widely used and recommended for diagnosing COVID-19.[5,6] However, several challenges were faced with NAATs due to the increased burden of testing, including the high cost, need for human resources, and most importantly the delay in results.[7] Rapid antigen tests are designed to detect viral particles from samples such as the throat or nasopharyngeal swabs in a dramatically shorter time than rRT-PCR. Tests are now available from several manufacturers, many of which are Food and Drug Administration-approved.[8] However, the main concern of these tests is the variably lower sensitivity compared to rRT-PCR, with the potential risk of missing active cases.[9] This study evaluates the performance of four different rapid antigen tests compared to rRT-PCR to determine the feasibility of integrating these tests into the diagnostic algorithm in clinical settings.

Methods

We conducted a prospective diagnostic test accuracy study of four rapid SARS-CoV-2 antigen detection tests compared to rRT-PCR betweem June and July 2020. It was conducted in Central Public Health Laboratories (CPHL), Muscat, which run rRT-PCR testing of COVID-19, in addition to two centers of disease control and prevention.

All clinically suspected COVID-19 patients who attended Medical Fitness Center (Darsait and A’Seeb) with acute respiratory symptoms or pneumonia during the evaluation period were included in the study regardless of severity or onset of symptoms. Asymptomatic patients were excluded. Nasopharyngeal and/or throat swabs were collected in viral transport media (VTM) from patients and tested via rRT-PCR and antigen assay according to manufacturers’ instructions. When indicated by the antigen kit provider, additional nasopharyngeal and/or throat samples (using swabs provided by the manufacturer) were collected for antigen tests in addition to routine VTM swabs for rRT-PCR.

We evaluated four COVID-19 rapid antigen tests to detect SARS-CoV-2 compared to rRT-PCR. The antigen fluorescent immunoassay (FIA) tests included the following:

STANDARD™ Q COVID-19 Ag test (SD BIOSENSOR, Korea); a chromatographic immunoassay for the qualitative detection of specific antigens to SARS-CoV-2. Positive results are indicated by the visual appearance of a line in the designated window of the kit.

PCL COVID-19 Ag Rapid FIA test (PCL, Korea); a FIA to detect SARS-CoV-2 nucleoprotein antigens in human nasopharyngeal swab specimen. Results are read automatically by an analyzer.

BIOCREDIT COVID-19 Ag test (RapiGEN Inc., Korea); a lateral flow immunochromatographic assay. Positive results are shown by the appearance of a black line in the result window of the kit.

Sofia SARS-CoV-2 antigen FIA test (Quidel, USA); The test uses immunofluorescence-based lateral flow technology in a sandwich design for the qualitative detection of nucleocapsid protein from SARS-CoV-2. Results are read by an analyzer and shown on a screen.

All samples were collected and tested according to manufacturer instructions.

RNA extraction was performed using a Liferiver extractor® (Shanghai ZJ Bio-Tech Co., Ltd.) or QIAmp viral RNA mini extraction kit (Qiagen).

Detection of SARS-CoV-2 was done by molecular assays using Liferiver Novel Coronavirus (2019-nCoV) Real-Time Multiplex RT-PCR Kit, Sansure Biotech COVID-19 Nucleic Acid Test Kit, or the Roche cobas® 6800 SARS-CoV-2 test. The SARS-CoV-2 genes targeted by the PCR assays are ORF, N, and E genes by Liferiver assay, ORF, and N genes by Sansure assay, ORF, and E genes by COBAS6800 assay.

Data were described using frequency and percentage. Diagnostic accuracy measures were calculated using MedCalc software (version 19.1.6). The correlation between rRT-PCR Ct value (ORF gene) and antigen test results was established using a point-biserial correlation coefficient. A p-value < 0.050 was considered statistically significant. According to epidemiological data, the positive predictive value (PPV) and negative predictive value (NPV) were calculated based on an assumed prevalence of COVID-19 in Oman of 9.2%.

Results

A total of 306 (nasopharyngeal, throat, or both) swabs were included in this study. Sixty-six samples were tested by the STANDARD™ Q COVID-19 Ag test, 87 samples by the PCL COVID-19 Ag Rapid FIA test, 75 samples by the BIOCREDIT COVID-19 Ag test, and 78 samples by the Sofia SARS-CoV-2 antigen FIA test. Results of antigen tests were compared with rRT-PCR. The sensitivity, specificity, accuracy, PPV, and NPV were calculated [Table 1]. All antigen tests demonstrated a specificity of 100% except for PCL, which has a lower specificity of 94.1%. However, sensitivities of the four antigen tests ranged between 64.0%and 69.8%.

Table 1

Diagnostic test characteristics of the four antigen detection assays and comparison of sensitivity, PPV, and NPV of rapid antigen tests in relation to Ct values.

Antigen detection kit	PCR Ct value	Sensitivity	Specificity	PPV	NPV	Correlation withCt values
STANDARDTM Q COVID-19 Ag test	Overall	65.8(48.65–80.37)	100(87.66–100)	100	96.7(94.89–97.82)	rpb = -0.549, p = 0.001
	Ct ≤ 35	71.0(51.96–85.78)		100	97.1(95.15–98.33)
	Ct ≤ 30	85.0(62.11–96.79)		100	98.5(95.86–99.47)
	Ct ≤ 25	92.3(63.97–99.81)		100	99.2(95.13–99.88)
PCL COVID-19 Ag Rapid FIA test	Overall	69.8(55.66–81.66)	94.1(80.32–99.28)	54.6(23.65–82.35)	96.9(95.30–97.91)	rpb = -0.744, p < 0.001
	Ct ≤ 35	75.5(61.13–99.28)		56.5(25.14–83.44)	97.4(95.84–98.42)
	Ct ≤ 30	91.9(78.09–98.30)		61.3(29.14–85.90)	99.1(97.47–99.71)
	Ct ≤ 25	100 (85.75–100)		63.3(30.99–86.86)	100
BIOCREDIT COVID-19 Ag test	Overall	64.0(49.19–77.08)	100(86.28–100)	100	96.5(94.99–97.54)	rpb = -0.645, p = 0.004
	Ct ≤ 35	71.1(55.69–83.63)		100	97.2(95.58–98.18)
	Ct ≤ 30	85.7(69.74–95.19)		100	98.6(96.84–99.36)
	Ct ≤ 25	92.3(74.87–99.05)		100	99.2(98.13–99.79)
Sofia SARS-CoV-2 antigen FIA test	Overall	64.3(50.36–76.64)	100(84.56–100)	100	96.5(95.11–97.52)	rpb = -0.820, p < 0.001
	Ct ≤ 35	66.0(51.73–78.48)		100	96.7(95.23–97.69)
	Ct ≤ 20	100 (89.11–100)		100	100

PPV: positive predictive value; NPV: negative predictive value; PCR: polymerase chain reaction; rpb: point-biserial correlation coefficient; FIA: fluorescent immunoassay.

Data given as percentage (95% confidence interval).

All assays showed a significant negative correlation between the reference rRT-PCR and Ag tests with statistically significant p-values [Figure 1 and Table 1]. Table 1 also shows each rapid antigen test’s sensitivity, PPV, and NPV after excluding samples with high Ct values (low viral loads). Sensitivity improved to ≥ 85% for the STANDARD™ Q COVID-19 Ag test, PCL COVID-19 Ag Rapid FIA test, and BIOCREDIT COVID-19 Ag test after excluding samples with PCR Ct values > 30. For the Sofia SARS-CoV-2 antigen FIA test, this data was not available. However, its sensitivity was 100% for samples with PCR Ct values < 20.

Figure 1

Correlation of rapid antigen tests (STANDARD™ Q COVID-19, PCL COVID-19 Ag Rapid FIA, BIOCREDIT COVID-19 Ag, and Sofia FIA) results with rRT-PCR Ct values: 0, Negative; 1, Positive.

Discussion

As the COVID-19 pandemic continued to spread, the crucial role of diagnostic tests was proven. rRT-PCR assays have been used widely and played a vital role in many countries’ response to the disease by allowing epidemiologists to more effectively track the spread and determine infection rates in given geographical areas. However, rRT-PCR is costly, time-consuming, and labor-intensive. Thus, the need for rapid, affordable tests is necessary. Recently, rapid SARS-CoV-2 antigen tests have been developed. Therefore, this study was undertaken to assess four commercial antigen detection assays’ diagnostic accuracy via comparison to molecular-based tests to determine if a person has a current SARS-CoV-2 infection.

Our results showed test sensitivities in the range of 64.0–69.8%, with the PCL assay demonstrating a slightly higher sensitivity of 69.8% compared to the other three tests. However, the PCL assay specificity was 94.1%, compared to 100% for the other three assays. As shown in Table 1, sensitivities increase when the CT values are low, which occurs early in infection and probably in the first few days of symptoms.

Generally, although rRT-PCR assays detect SARS-CoV-2 at an average sensitivity of 95.2% and specificity of 98.9%,[10] the detection limits and the ability to differentiate between true negatives and positives at low RNA concentrations vary between assays. In individual laboratories, careful evaluation is required to determine Ct value cut-offs for differentiating between positives and negatives. It is possible that at high Ct values, genetic fragments of the virus are detected, which are not indicative of live virus and therefore not clinically meaningful.[11]

All assays showed a significant negative correlation between the reference rRT-PCR Ct values and rapid antigen test results. This indicates that rapid antigen tests are likely to perform better with high viral loads (lower Ct values). High viral loads usually occur in the pre-symptomatic phase (1–3 days before symptoms onset) and early symptomatic phase (within the first 5–7 days) of the illness, which are also the periods with the highest rate of infectivity.[9]

The WHO has set minimum performance requirements at ≥ 80% sensitivity and ≥ 97% specificity for COVID-19 assays, which was also agreed on by the European Center for Disease Prevention and Control.[9,12] In our study, excluding samples with PCR Ct values > 30 and > 25 resulted in sensitivity improvements up to ≥ 85% and > 90%, respectively, for the STANDARD™ Q COVID-19 Ag test, the PCL COVID-19 Ag Rapid FIA test, and the BIOCREDIT COVID-19 Ag test. As the Sofia SARS-CoV-2 antigen FIA test assessment did not include samples in the PCR Ct value between 20–30 category, the evaluation of sensitivity at Ct < 30 or < 25 was not possible. However, it had 100% sensitivity for samples with PCR Ct value < 20.

Although the principal concern of antigen detection assays are the false-negative rates due to low viral load or high Ct values, the clinical significance of this limitation might be mitigated to some extent, knowing that infectivity of patients with Ct > 24 and duration of symptoms over eight days may be low.[13] In addition, setting a diagnostic algorithm to confirm negative antigen test results when clinically relevant may further mitigate the risk of missing active cases.

All four assays were easy to perform in the clinical laboratory within < 30 min, with the BIOCREDIT assay demonstrating the shortest timeframe (5–8 min). However, one limitation of the BIOCREDIT and STANDARD™ assays is that the assay’s result is determined by the visual presence or absence of a line, which is recorded by an operator, making it prone to operator subjectivity. In contrast, the fluorescence readout of the PCL and Sofia assays are generated on automated analyzers, preventing operator bias.

This study has some limitations. Clinical information, including symptoms and duration of symptoms at sampling time, were not available to correlate with PCR and antigen results. Another limitation is that different PCR assays were used to compare antigen results. Finally, for most patients, PCR and antigen tests were performed from two swabs taken at the same visit. While this might be considered a strength, it is well-known that sampling quality might affect the results, and maintaining the same level of quality for each pair of swabs cannot be guaranteed.

Conclusion

The accurate and rapid diagnosis of people infected with the SARS-CoV-2 virus is essential to address the global spread of COVID-19. Given the simplicity, rapidity, low cost, and high specificity of antigen tests, we speculate that integrating antigen tests into the clinical diagnostic algorithms would help contain the outbreak if correctly performed and interpreted. The limitation of the low sensitivity of rapid antigen tests is probably overestimated since the missed cases are likely to have low viral loads and are less infectious. However, confirming negative cases (where clinically relevant) by repeating the test or using a more sensitive assay like PCR is recommended to decrease the chance of missing active cases.