Comparative analyses of eighteen rapid antigen tests and RT-PCR for COVID-19 quarantine and surveillance-based isolation.

Background: Rapid antigen (RA) tests are being increasingly employed to detect SARS-CoV-2 infections in quarantine and surveillance. Prior research has focused on RT-PCR testing, a single RA test, or generic diagnostic characteristics of RA tests in assessing testing strategies.
Methods: We have conducted a comparative analysis of the post-quarantine transmission, the effective reproduction number during serial testing, and the false-positive rates for 18 RA tests with emergency use authorization from The United States Food and Drug Administration and an RT-PCR test. To quantify the extent of transmission, we developed an analytical mathematical framework informed by COVID-19 infectiousness, test specificity, and temporal diagnostic sensitivity data.
Results: We demonstrate that the relative effectiveness of RA tests and RT-PCR testing in reducing post-quarantine transmission depends on the quarantine duration and the turnaround time of testing results. For quarantines of two days or shorter, conducting a RA test on exit from quarantine reduces onward transmission more than a single RT-PCR test (with a 24-h delay) conducted upon exit. Applied to a complementary approach of performing serial testing at a specified frequency paired with isolation of positives, we have shown that RA tests outperform RT-PCR with a 24-h delay. The results from our modeling framework are consistent with quarantine and serial testing data collected from a remote industry setting. Conclusions: These RA test-specific results are an important component of the tool set for policy decision-making, and demonstrate that judicious selection of an appropriate RA test can supply a viable alternative to RT-PCR in efforts to control the spread of disease.

Introduction

Testing for SARS-CoV-2 infections has played a central role in combating the COVID-19 pandemic. Despite vaccination, testing will continue to be essential for screening and surveillance[1-3], enabling timely detection of new variants and isolation of infected individuals to reduce the risk of further disease spread. Additionally, testing can inform quarantine strategies and sufficient durations to alleviate onward transmission. For instance, previous studies have shown that a 14-day quarantine with no testing for a close contact of a case can safely be shortened to seven days if an RT-PCR test is conducted on exit from the quarantine[4-6]. Implementation of this shortened quarantine for close contacts of an identified case is now specified by the Centers for Disease Control and Prevention (CDC)[7]. Complementary analyses have also evaluated the optimal frequency of RT-PCR serial testing in at-risk populations to minimize the probability of an outbreak[8-18].

Throughout the pandemic, the diversity of SARS-CoV-2 tests with regulatory approval has increased immensely. However, there has been some dispute surrounding the utility of rapid antigen (RA) tests in infection screening and control efforts[19-24]. Although the RT-PCR tests remain the gold standard for diagnosis, RA tests have aided in scaling up testing capacities worldwide. The fast turnaround time, wider availability, and lower costs make RA tests an attractive choice for workplace screening, especially in remote environments (e.g. offshore shift teams). The RA test kits require minimal training and can be self-administered, without requiring substantial ongoing equipment maintenance and calibration.

Many businesses and organizations are shifting to using RA tests for screening employees instead of solely relying on the more costly and time-consuming RT-PCR[25-28]. Evaluation of the performance of serial RA testing in identifying cases has occurred in both the health care[29,30] and university setting[31,32]. These studies conducted screening during an active COVID-19 outbreak[31] or in a tertiary hospital setting[30]. Outside these settings, screening asymptomatic individuals without known or suspected exposure to SARS-CoV-2 has been proposed and discussed in the literature[33,34]. In remote industrial settings, exposure occurs predominantly within the isolated population, and there are distinct challenges that differ from the healthcare and university environment. Specifically, there are logistical constraints to imposing isolation or offering treatment. Therefore, we present serial testing data for an offshore oil site as there is currently minimal published evidence of the effectiveness of large-scale serial RA testing in mitigating outbreaks within an industrial setting.

One concern with RA tests is their higher rate of false positives and negatives compared to RT-PCR[21,22,35]. As a tool for workplace screening or community surveillance, testing frequency is critical to avoiding an outbreak (i.e., attaining an effective reproduction number R that is below one)[9,35]. However, increasing the number of tests used in screening increases costs and elevates the number of false positives obtained. False-positive results do not entail direct epidemiological risks, but do lead to undesirable logistical and cost challenges. For example, in an offshore and or remote workplace setting, a false positive could necessitate medical evacuation via helicopter or other aviation platforms. From a workplace risk-analysis perspective, a false positive is less disruptive to operations than false negatives resulting in transmission and a full-scale outbreak.

There has been extensive analysis and evaluation of the optimal strategies for both RT-PCR and RA testing to mitigate SARS CoV-2 transmission[4-6,8-18,36,37]. However, most of these analyses do not quantify the degree to which their use suppresses individual-level transmission in applications of quarantine[36] or serial surveillance testing with isolation of positives[10,11,13,15,16], and most address a generalized or single RA test[5,6,9,11,12,16,18,36,37], or focus only on RT-PCR[4,8,10,13,15,17]. In contrast, multiple RA tests have received regulatory Emergency Use Authorization (EUA)[38], each with distinct temporal diagnostic sensitivity. Although past research has compared the properties of different RA tests, these temporal differences in diagnostic sensitivity have yet to be evaluated on a daily basis since infection[39,40]. These distinct diagnostic properties could produce pronounced changes in reducing onward transmission for testing strategies previously determined when using RT-PCR testing[33].

Here we construct the temporal diagnostic sensitivity curves for 18 RA tests using data on percent positive agreement (PPA) with an RT-PCR test and temporal diagnostic sensitivity of an RT-PCR test. To determine when these RA tests can serve as a suitable alternative to the more costly and laborious RT-PCR tests, we calculated (i) their associated probabilities of post-quarantine transmission (PQT) for quarantine durations from one to 14 days with testing on exit or both entry and exit upon random entry into quarantine; (ii) their extents of onward transmission for serial testing conducted every day to every 14 days; and (iii) their associated probabilities of false-positives during serial testing. We further evaluated the utility of RA tests using data collected from two offshore oil companies in the context of quarantine and serial testing within an industrial environment.

Methods

Infectivity profile

The infectivity profile was generated by fitting a Gamma distribution to observed generation times for the Delta variant[41]. Specifying a fixed duration of the incubation period of 4.4 days[41], infected individuals were considered infectious no longer than 20 days after symptom onset[42-44]. We calculated results based on infected individuals producing an average of 3.2 secondary infections in the absence of self-isolation[41].

Diagnostic sensitivity of the RT-PCR test

To construct a temporal diagnostic sensitivity curve, we fitted a log-Normal distribution to nasopharyngeal RT-PCR testing percent-positivity data from Hellewell et al.[11] using a maximum-likelihood approach (Supplementary Methods; Supplementary Table 1; Supplementary Fig. 1).

Diagnostic sensitivity of antigen tests

We fitted a linear logit model to the discrete PPA data to estimate a continuous PPA curve from the time of symptom onset for each RA test. (Supplementary Data 1–2; Supplementary Methods). PPA data were only available for post-symptom onset. Therefore, we inferred the pre-symptomatic diagnostic sensitivity of the RA tests by constructing a mapping between the inferred diagnostic sensitivity post symptom onset and the level of infectivity, then applying that mapping to pre-symptomatic levels of infectivity (Supplementary Methods)[4]. The sensitivity of a RA test was calculated as the product of the PPA curve and the diagnostic sensitivity of a RT-PCR test at the specified times (Supplementary Figs. 2–22).

For our baseline results, we examined the five most commonly used RA tests: LumiraDx, Sofia, BinaxNOW, BD Veritor, and CareStart[45]. For the analysis of the LumiraDx and CareStart antigen test, we utilized the PPA data for the anterior nasal swab, as this method of sampling was used in gathering data for the BD Veritor, BinaxNOW and Sofia antigen tests. Furthermore, the anterior nasal sample can be obtained by a broad range of individuals with less specialized training compared to a nasopharyngeal sample[46-48]. We also examined both the anterior nasal and nasopharyngeal swab for the LumiraDx and CareStart antigen test (Supplementary Data 1–2).

We compared the PPA datasets submitted to the U.S.A. FDA with those obtained from independent studies that were conducted in a real-world setting. Specifically, we considered the independent studies for BinaxNOW[49], CareStart[50], and Sofia[32]. Both the BinaxNOW and CareStart studies were conducted at a community testing site, where the trained site collector obtained the samples[49,50]. The study for Sofia was conducted in a university setting, where the samples informing the PPA of the RA test with RT-PCR were from the university in which medical professionals conducted the swabbing[32].

Probability of post-quarantine transmission

Specifying 35.1% of infections are asymptomatic[51] and isolation upon symptom onset, we quantified the effectiveness of quarantine and testing strategies in reducing PQT by calculating the probability of PQT for individuals entering quarantine randomly—and not identified through contact tracing—in the absence of symptoms[4]. Accounting for substantial variance in transmission among COVID-19 cases[4,52-56], we specified that secondary cases were negative-binomially distributed:with dispersion parameter k = 0.25[4,56] and p = k / (k + R)—such that the average number of secondary cases is equal to the expected PQT, denoted R. This value for the dispersion parameter is consistent with estimates from other studies[52-55]. Accordingly, the probability of PQT was calculated as 1 − f(0|k, p).

For quarantine durations varying from one to 14 days, we compared the probability of PQT when performing a single RT-PCR test on exit to a RA test on exit or RA tests on both entry and exit. The objective of the additional RA test on entry to the one on exit is to compensate for the reduced diagnostic sensitivity of a single RA test compared to RT-PCR.

Probability of a false-positive

With a specificity ζi for test i (Supplementary Data 3) and testing every f days, we computed the average probability that at least one false-positive occurred over a two-week period where τ is the number of tests to occur in the jth two-week period since the start of serial testing. For each testing frequency f (i.e., the time between two consecutive tests), we investigated the sequence of test times {1,1+f,1+2f,...,1+13f} that comprises all the unique testing patterns possible over a two-week period to calculate the average probability. Defining RT-PCR to be the gold standard for testing accuracy, the specificity of a RA test was estimated as the specificity of the RT-PCR test multiplied by the percent negative agreement of the RA test with RT-PCR. This calculation provides a lower bound for the RA test specificity, given the possibility of a false-negative RT-PCR test.

Scenario analyses

We conducted scenario analyses to determine the impact of (i) the incubation period; (ii) the reduced diagnostic sensitivity of RA tests for low-levels of infectivity; (iii) the proportion of asymptomatic infections and the basic reproduction number; and (iv) the RT-PCR diagnostic sensitivity curve on the onward transmission after quarantine and during serial testing.

To evaluate the robustness of our results to a potentially longer duration of incubation than 4.4 days[41], we evaluated the impact of an alternate incubation period of 5.72 days and the infectivity profile associated with that longer duration[37]. A positive RA test indicates active infection, suggesting that there is a substantial concentration of virus within the sample site. Studies have indicated that SARS-CoV-2 cannot be successfully cultured 10 days after symptom onset[44]. Thus, it is possible at some stages of the disease that a RA test will return a negative while RT-PCR still yields a positive. There is also uncertainty in the inferred diagnostic sensitivity in the later stages of disease, as the PPA was extrapolated past the last recorded data point for all RA tests. To address whether differences in the results of these tests among low levels of infectivity make a difference to our findings, we imposed a threshold on the level of infectivity below which the RA test will only return a negative: if the infectivity at the time of the RA tests was below the infectivity at 10 days post-symptom onset, then the RA test returned a negative result (Supplementary Fig. 23). We also examined a stricter threshold based on the infectivity at 5.6 days post-symptom onset (where there is 1% infectivity remaining). To assess the sensitivity of our results to variation in the proportion of asymptomatic infection and the basic reproduction number, we conducted a grid analysis with values ranging from 0.1–0.95 to 1.05–5, respectively. Furthermore, we evaluated an alternative model for the temporal RT-PCR diagnostic sensitivity curve, substituting a log-Student’s-t distribution for the log-Normal distribution used in the baseline analysis.

Paired testing of oil workers in quarantine

The BD Veritor kit was selected for use by BP/BHP because at the time it was possible to secure a sufficient volume of the test kits and readers for a potential six-month evaluation. With informed consent from the offshore oil workers, onshore paired testing was conducted by laboratory-based RT-PCR and RA testing on entry into quarantine (day 0), day three, and day four. For both RA and RT-PCR testing, swabbing within quarantine was conducted by medical personnel. The study period spanned 22 November 2020 to 17 January 2021, a period that included an observed rise in community transmission across Texas and Louisiana, the states of residence for the majority of the offshore platform workers. Before entry into quarantine, workers had to pass a pre-screening questionnaire that filtered symptomatic individuals and those with recent exposure. During quarantine, a positive RT-PCR led to removal of the individual from the quarantine environment, placing them in isolation for 10 days with medical follow-up. Workers were then approved to return to work after two negative RT-PCR tests.

Offshore serial rapid antigen testing

Implementation of a test-and-fly protocol began at the start of the staged vaccination rollout in the USA—2 March 2021 to 24 May 2021. Thus, none of the offshore workers were vaccinated nor likely to be vaccinated for several months. The test-and fly-strategy consisted of initial screening for symptoms, an RT-PCR test on entry to a quarantine of approximately 24 h. The exact quarantine duration was dependent on the commercial laboratory-based RT-PCR turnaround time: after receiving a negative RT-PCR test, the worker was permitted to enter the offshore work environment. With an RT-PCR positive test, the individual was removed from the quarantine environment, and advised to conduct a 10-day home-based self-isolation with medical follow-up.

Upon entering the offshore work environment, the worker underwent serial antigen testing within their first nine of 14 or more days being offshore. Testing of a worker occurred with their informed consent either on days three, six, and nine; or on days two, five, and eight. The two patterns were selected based on the results from our data-driven model for the degree of suppression achieved at each frequency of testing, by the BD Veritor test kit (Supplementary Fig. 4). All testing was conducted by the platform medic using the BD Veritor kit and reader. Any positive individual was isolated pending helicopter transfer (typically within 12–24 h) to the established onshore medical facility, whereupon an RT-PCR nasal swab was obtained and sent to a commercial laboratory. A positive antigen test was considered to be a false positive if the follow-up RT-PCR was negative.

Conducting this study of the onshore and offshore testing of the oil platform employees, and the use of the resulting data, was approved by the Human Participants Review Sub-Committee, York University’s Ethics Review Board (Certificate Number: 2021–003). All employees that participated in the study provided informed consent. Ethical approval was not required for the datasets of the RT-PCR diagnostic sensitivity, test specificity, and the PPA for each RA test because they are available in the public domain.

Estimating the positive agreement between BD Veritor and RT-PCR during quarantine

We calculated the probability that an infected individual would test positive with both the RA test and RT-PCR on entry, day three of quarantine, and day four. This calculation accounts for isolation both upon symptom onset and upon a positive RT-PCR test.

Effectiveness of the offshore serial rapid antigen testing

To determine the number of infectious individuals that were in the offshore environment, we used our model framework for serial testing, the estimates for the specificity and sensitivity of BD Veritor from the FDA reports, the number of false positives and false negatives from the RA test, RT-PCR confirmed positives, and the probability of onward transmission from these cases using a maximum-likelihood approach.

Table 1

Paired BD Veritor and RT-PCR tests conducted, results, their agreement, and false positives.

Quarantine day after entrya	Day 0b	Day 3	Day 4	Total
Paired tests conducted	818	726	675	2219
RT-PCR tests positive	12	5	3	20
RA tests positive	4	3	0	7
RA test false negative c (RA−/RT-PCR +)	9	4	3	16
RA test false positives c (RA +/RT-PCR −)	1	2	0	3
RA test true positive c (RA +/RT-PCR +)	3	1	0	4

BHP onshore quarantine testing, 22 November 2020 to 17 January 2021.

Test on entry into quarantine.

Assessed by comparison to RT-PCR.

Table 2

Serial offshore BD Veritor rapid antigen tests, positives, and false positives on days 3, 6, and 9.

Day after off-shore entrya	Day 3	Day 6	Day 9	Total
RA tests conducted	458	458	457	1373
RA tests positive	2	3	0	5
RA false positivesb	2	0	0	2

aBP platform test data, 2 March 2021 to 22 May 2021.

bSubsequent onshore laboratory RT-PCR was negative.

Table 3

Serial offshore BD Veritor rapid antigen tests, positives, and false positives on day 2, 5, and 8.

Day after offshore entry a	Day 2	Day 5	Day 8	Total
RA tests conducted	124	121	96	341
RA tests positive	0	0	0	0
RA false positives	0	0	0	0

BP platform test data, 5 March 2021 to 24 May 2021.