COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay: A national systematic evaluation of sensitivity and specificity for mass-testing.

BACKGROUND: Lateral flow device (LFD) viral antigen immunoassays have been developed around the world as diagnostic tests for SARS-CoV-2 infection. They have been proposed to deliver an infrastructure-light, cost-economical solution giving results within half an hour.
METHODS: LFDs were initially reviewed by a Department of Health and Social Care team, part of the UK government, from which 64 were selected for further evaluation from 1st August to 15th December 2020. Standardised laboratory evaluations, and for those that met the published criteria, field testing in the Falcon-C19 research study and UK pilots were performed (UK COVID-19 testing centres, hospital, schools, armed forces).
FINDINGS: 4/64 LFDs so far have desirable performance characteristics (orient Gene, Deepblue, Abbott and Innova SARS-CoV-2 Antigen Rapid Qualitative Test). All these LFDs have a viral antigen detection of >90% at 100,000 RNA copies/ml. 8951 Innova LFD tests were performed with a kit failure rate of 5.6% (502/8951, 95% CI: 5.1-6.1), false positive rate of 0.32% (22/6954, 95% CI: 0.20-0.48). Viral antigen detection/sensitivity across the sampling cohort when performed by laboratory scientists was 78.8% (156/198, 95% CI 72.4-84.3).
INTERPRETATION: Our results suggest LFDs have promising performance characteristics for mass population testing and can be used to identify infectious positive individuals. The Innova LFD shows good viral antigen detection/sensitivity with excellent specificity, although kit failure rates and the impact of training are potential issues. These results support the expanded evaluation of LFDs, and assessment of greater access to testing on COVID-19 transmission. FUNDING: Department of Health and Social Care. University of Oxford. Public Health England Porton Down, Manchester University NHS Foundation Trust, National Institute of Health Research.

Evidence before this study

Lateral flow devices are a new form of testing for SARS-CoV-2. They differ from RT-PCR tests in that they rely on the detection of viral antigens by immunoassays and their utility has not yet been fully defined. A literature review was performed in PubMed and bioRxiv/medRxiv for all studies using lateral flow devices for the detection of SARS-CoV-2 viral antigen. This used the search terms “COVID-19”, “SARS-CoV-2”, “viral antigen” and “lateral flow devices” and was not limited to English language publications. To date, the majority of studies have been largely single centre studies analysing a single test and there are contrasting results with some LFDs showing good sensitivity and specificity, and others demonstrating poorer performance.

Added value of this study

This UK COVID-19 Lateral Flow Oversight group study is the largest national evaluation undertaken of viral antigen LFDs for COVID-19. We have flagged four LFDs with the best performance characteristics from our assessments. The Innova LFD has been tested the most extensively and has high specificity with acceptable sensitivity. Our data has also highlighted the critical importance of training. We also note the need for further clinical studies to demonstrate that the identification of individuals with higher viral loads will be of benefit in interrupting transmission.

Implications of all the available evidence

Our data indicates that LFDs for COVID-19 have performance characteristics attractive for the UK mass testing program. Ongoing iterative evaluation of the population-level roll-out of LFDs in reducing transmission of COVID-19, and the contribution of such tests to reducing the risk of morbidity and mortality for clinically vulnerable individuals, is desirable. Further work is required to determine the amount and content of “training” to derive optimal test performance.

Alt-text: Unlabelled box

Introduction

National governments and international organisations including the World Health Organisation (WHO) and European Commission have highlighted the importance of individual testing, mass population testing and subsequent contact tracing to halt the chain of transmission of SARS-CoV-2, the virus responsible for COVID-19 [1,2,3]. The current diagnostic test involves reverse-transcription polymerase chain reaction (RT-PCR) testing of nose/throat swabs in specialised laboratories. Such capacity in the UK is currently estimated at ~500,000 tests/day [4], [5], [6], [7]. and this is used with contact tracing procedures and mobile applications to identify close symptomatic contacts of infected symptomatic individuals [8], [9], [10]. However, there are significant challenges in creating testing capacity to identify those with asymptomatic infections or to test contacts of individuals with COVID-19. To date, turnaround time for issuing results from RT-PCR has been typically slow (>24 h), though, the time of test of RT-PCR is usually a few hours.

To better understand and control SARS-CoV-2 transmission, there is an urgent need for large-scale, accurate, affordable and rapid diagnostic testing assays, with the ability to detect infectious individuals. Lateral flow device (LFD) immunoassays can be designed to test for different protein targets and are routinely used in healthcare settings principally as a result of their affordability, ease of use, short turnaround time, and high-test accuracy. In brief, a sample is placed on a conjugation pad where the analyte (or antigen) of interest is bound by conjugated antibodies. The analyte-antibody mix subsequently migrates along a membrane by capillary flow across both ‘test’ and ‘control’ strips. These strips are coated with antibodies detecting the analyte of interest and a positive test is confirmed by the appearance of coloured control and test lines [11].

Newly developed SARS-CoV-2 antigen LFDs identify the presence of specific viral proteins, using conjugated antibodies to bind spike, envelope, membrane or nucleocapsid proteins. In contrast to the IgM/IgG “antibody tests”, these antigen tests directly identify viral proteins, and are not reliant on the host's immune response. In contrast to RT-PCR, results for LFDs are observed in 10–30 min depending on the device, providing a window for early interventions to halt the chain of transmission earlier in the disease course when individuals are most infectious [12].

To date, many manufacturers have developed first-generation rapid SARS-CoV-2 antigen-detecting LFDs. However, many of these tests have not been independently validated. There is evidence of variable performance when assessing test sensitivity and specificity, although several candidates looked promising on the basis of early data [13], [14], [15]. An independent national evaluation of these devices is important to facilitate population-level or mass testing initiatives globally.

Here, we report the diagnostic performance of first-generation SARS-CoV-2 antigen-detecting LFD for rapid point-of-care (POC) testing in work that was commissioned by the UK's Department of Health and Social Care (DHSC) from PHE Porton Down and the University of Oxford.

Methods

A phased evaluation of available SARS-CoV-2 antigen LFDs was undertaken from 15th August to December 2020. The number of tests required at each phase of evaluations were defined a priori and published online [16].

Department of health and social care evaluation (Phase 1 evaluation)

The DHSC identified manufacturers supplying SARS-CoV-2 antigen LFDs that could enable mass testing at a population level. A desktop review was performed to ensure there were appropriate instructions for use and to assess manufacturers’ claimed performance and manufacturing capabilities. Manufacturers were not assessed if they had limited evidence of efficacy, a low manufacturing capacity, in very early stages of development or unwilling to supply kits for testing in the UK development pipeline.

Pre-clinical evaluation (Phase 2 evaluation)

Pre-clinical evaluation of candidate LFDs was performed by trained laboratory scientists at Public Health England (PHE) Porton Down. LFDs were evaluated against SARS-CoV-2 spiked positive controls and known negative controls, consisting of saliva collected from healthy adult staff volunteers. Approval for this project was obtained by PHE Porton down, and as no patient samples were utilised, consent was not required. Viral inactivation was performed using AVL buffer (Qiagen) and 5% Triton-X100.

Results were recorded as positive, negative or void with no possibility of recording indeterminate results. Pre-defined and publicly available “prioritisation” criteria to pass on to the next evaluation phase had to be met for LFDs, consisting of (i) a kit failure rate of <10%; (ii) an analytical specificity of ≥97%, and (iii) an analytical LOD of ≥9 of 15 (60%) at 102 pfu/mL (plaque forming units per millilitre), corresponding to a RT-PCR cycle threshold (Ct) of approximately 25 (~100,000 RNA copies/ml); and (iv) lack of cross-reactivity with seasonal coronaviruses to further test analytical specificity [16].

Retrospective secondary care evaluation (Phase 3a evaluation)

Evaluation using patient samples retrospectively was started in August 2020 at PHE Porton Down. Institutional ethical assessment and approval for this project was obtained from the University of Oxford. Surplus samples were used anonymously as an audit of COVID-19 tests and enabled evaluation without express consent from individuals. Samples were obtained from a secondary healthcare setting (Oxford University Hospitals NHS Foundation Trust). Samples were analysed against RT-PCR to determine false positives and negatives (see supplementary methods).

1000 SARS-CoV-2 negative samples: fresh swabs samples in VTM held at 4oC were supplied the day after they were tested negative by RT-PCR by the laboratory service at the John Radcliffe Hospital, Oxford, UK.

200 SARS-CoV-2 positive samples: swabs collected in VTM from patients admitted to hospital during the first wave of the UK pandemic (March-June 2020) [17]. These swabs generally consisted of MWE microbiology transport swabs. These were diluted 1:4 SARS-CoV-2 RT-PCR negative saliva, aliquoted and frozen at −20 °C for later use. For each positive sample, in addition to the original diagnostic RT-PCR Ct value, a confirmatory RT-PCR was performed at PHE Porton Down on the diluted sample to determine the new Ct value.

Community research evaluation (Phase 3b evaluation)

We undertook a field evaluation using samples from volunteers in the community in collaboration with the National Institute for Health Research (NIHR) funded CONDOR Platform “COVID-19 National Diagnostic Research and Evaluation Platform”. This was performed under ethical approval obtained from the FALCON—C19 study (Facilitating Accelerated Clinical Validation Of Novel diagnostics for COVID-19, 20/WA/0169, IRAS 284,229. All individuals gave expressed consent for use of their data.). The study was conducted between 17th September and 23rd October 2020. This involved the recruitment and re-testing of a consecutive series of consenting adults with a RT-PCR-confirmed diagnosis of SARS-CoV-2 infection within 5 days of the original PCR result. LFD testing was performed at PHE Porton Down with the operators being unaware of clinical information from the study participants.

For the Innova SARS-CoV-2 Antigen Rapid Qualitative Test, testing was additionally performed for a subset of samples on-site at four COVID-19 testing centres by trained research staff using the “dry swabs” to evaluate “real-life”/diagnostic performance. Dry swabs are swabs included in the kit provided by the manufacturer that are not placed into viral transport medium prior to performing the LFD test.

Community field service evaluation (Phase 4 evaluation)

Wider field service evaluations were performed within a number of UK institutions and settings. These evaluations utilised the Innova SARS-CoV-2 Antigen Rapid Qualitative Test. These institutions included a secondary healthcare setting (John Radcliffe Hospital, Oxford), PHE Porton Down, armed forces members (following an outbreak) and in secondary schools (pupils aged 11–18). Evaluations were also undertaken at regional COVID-19 testing centres as part of an NHS Test and Trace service evaluation involving the general public. The John Radcliffe Hospital, Oxford performed an evaluation as part of their asymptomatic staff screening service using the Respiratory Diagnostic Kit Evaluation (‘Red Kite’) study (Research Ethics Committee reference: 19/NW/0730; North West-Greater Manchester South Research Ethics Committee).

Statistical analyses

Fisher's exact were used to determine non-random associations between categorical variables. Statistical analyses and data visualisation were performed using R version 4.0.3. Sensitivity and specificity and 95% confidence intervals were calculated using the exact Clopper-Pearson method. Missing data was excluded. P-value less than or equal to 0.05 is considered statistically significant.

Role of the funding source

This study was funded by the UK Government's Department of Health and Social Care. This work was supported by the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Healthcare Associated Infections and Antimicrobial Resistance at Oxford University in partnership with Public Health England (PHE) (NIHR200915), Manchester University NHS Foundation Trust, National Institute of Health Research and the NIHR Biomedical Research Centre, Oxford. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research, the Department of Health or Public Health England. LYWL is supported by the NIHR Oxford BRC. DWE is a Robertson Foundation Fellow and an NIHR Oxford BRC Senior Fellow. These funders had no role in the design of this study and did not have any role during its execution, analyses, interpretation of the data, or decision to submit results

Results

Phase 1

A total of 132 suppliers of SARS-CoV-2 antigen detection LFDs were identified and referred to the DHSC for initial Phase 1 review. among these, at the time of publication, 64 were selected by the DHSC for further evaluation by the UK lateral flow oversight group.

Phase 2

As part of Phase 2 evaluations, 9692 LFD tests were performed at PHE Porton Down across the 64 candidate devices as of the 3rd December 2020. 5 LFDs had a kit failure rate above the pre-specified threshold for exclusion (>10%), 17 kits had a false-positive rate below the pre-defined specificity threshold (<97%) and 28 kits a false-negative rate below the LOD threshold (<60% at 102 pfu/m). In total, across all three criteria, nineteen kits performed at a level in accordance with the UK Lateral Flow Oversight Group's a priori “prioritisation criteria”. All nineteen kits also passed cross-reactivity analyses against seasonal human coronaviruses.

The limit of detection of plaque forming units was studied more extensively with Innova (see Table1). This analysis consisted of saliva spiked with SARS-CoV-2 with stock of SARS-CoV-2 with a standardised PFU.  Under these ideal concentrations, at an estimated PFU of 390/mL, which corresponds to a Ct of ~25, the LFD identified all samples.

Table 1

Limit of detection for SARS-CoV-2 detection by the Innova LFD for antigen detection using saliva sample spiked with SARS-CoV-2. Ct - cycle threshold. PFU - plaque forming units.

PFU/ml	Ct equivalent	Positive LFD tests/total LFD tests	% positive
100,000	16	20/20	100
10,000	19	25/25	100
1000	23.7	65/65	100
390	25.2	5/5	100
100	25.5	63/65	96
40	28.5	3/5	60
20	29.3	0/5	0
10	30.2	0/5	0
5	31	0/5	0
2.5	31.7	0/5	0
1.2	32.5	0/5	0

False positive rates of LFDs

The False positive rate varied from <0.1% to 0.3% on all the LDS using PCR negative saliva samples (see Supplementary Table 1). More extensive testing was performed on the Innova LFD, for which we had a sufficient supply of kits available for wider testing at the time. Device specificity was determined through an analysis of 6954. The percentage of false-positives ranged from 0.00 to 0.49%, with an overall specificity of 99.68%. The false-positive rate was centre-dependent (p = 0.014, Fisher's exact test). These evaluations noted that where there were challenges in interpreting the results when the test result was “weak” (i.e. the test line was very faint) (Table 2).

Table 2

Number of false positives in negative samples in each evaluation stage for the Innova LFD. 95% confidence intervals presented in each case.

Evaluation Phase	False positives/total number	False positive rate (95% confidence interval)
Phase 2 evaluation	0/72	0.0% (0.0–5.0)
Phase 3a evaluation- negative samples	0/940	0.0% (0.0–0.4)
Phase 4 evaluation- hospital staff	1/329*	0.3% (0.01–1.7)
Phase 4 evaluation- armed forces	0/105	0.0% (0.0–3.5)
Phase 4 evaluation- PHE staff	0/209	0.0% (0.0–1.8)
Phase 4 evaluation- school 1	9/1855**	0.5% (0.2–0.9)
Phase 4 evaluation- school 2 + 3 + 4	7/2130**	0.3% (0.1–0.7)
Phase 4 evaluation- COVID-19 testing centre	5/1314***	0.4% (0.1–0.9)
TOTAL	22/6954	0.3% (0.2–0.5)

*This was 1 weak positive result that was also a weak positive on repeating; ** Weak positive results were negative on retesting with Innova; *** Not photographed or repeated. Taken in a setting of prevalence of 14% LFD positive results.

Kit failure rate of innova

Across Phase 2–4 evaluation stages, 8951 Innova LFD tests were performed, including a diverse cohort of populations as part of Phase 3b and Phase 4 testing, namely out-patient SARS-CoV-2 cases, healthcare staff, armed forces personnel and secondary school children. The overall kit failure rate for the Innova LFD was 5.6% (502/8951, 95% CI: 5.1–6.1) (Table 3). The most common reason for kit failure was poor transfer of the liquid within the device from the reservoir onto the test strip.

Table 3

Evaluations of the Innova LFD across Phases 2–4. The table demonstrates the kit failure rate.

Innova LFD evaluation phase	LFD failures (%)
Phase 2 negatives	0/72 (0.0%)
Phase 2 positive dilution series	0/60 (0.0%)
Phase 2 positive extended dilution series	0/155 (0.0%)
Phase 2 Swab comparison	0/187 (0.0%)
Phase 3a positives	13/191 (6.8%)
Phase 3a negatives	50/990 (5.1%)
Phase 3b FALCON (Dry swabs- field)	27/267 (10.1%)
Phase 3b FALCON (Dry swabs- lab)	9/212 (4.2%)
Phase 3b FALCON (VTM swabs)	9/157 (5.7%)
Phase 4 hospital staff	17/358 (4.7%)
Phase 4 armed forces	6/157 (3.8%)
Phase 4 PHE staff	19/212 (8.9%)
Phase 4 school 1	311/1855 (16.8%)
Phase 4 school 2 + 3 + 4	14/2132 (0.7%)
Phase 4 COVID-19 testing centre	27/1946 (1.4%)
	502/8951 (5.6%)

Sensitivity of LFDs against viral load determined by qPCR (Phase 3)

To date, seven swab-based LFDs have passed Phase 3a evaluation, namely: Innova SARS-CoV-2 Antigen Rapid Qualitative Test (Innova), Zhejiang orient Gene Biotech Co. Coronavirus Ag Rapid Test Cassette (Swab) (Orient Gene), Anhui Deepblue Medical Technology COVID-19 (Sars-CoV-2) Antigen Test kit (Colloidal Gold) (Deepblue), Fortress Diagnostics Coronavirus Ag Rapid Test (Fortress), Roche SD Biosensor Standard Q COVID-19 Ag Test (SD Bio swab), Surescreen Diagnostics SARS-CoV-2 Antigen Rapid Test Cassette (Nasopharyngeal swab (Surescreen) and Abbott Panbio COVID-19 Ag Rapid Test Device (Abbott) (Supplementary Table 1). Three LFDs did not pass 3a evaluation and the remaining LFDs are currently undergoing evaluation. Four LFDs (Deepblue, Innova, Orientgene, Abbott) have passed Phase 3b evaluation in which the LFDs were used according to the manufacturer's instructions by either a laboratory worker and (for Innova) a health care worker and the results were compared to the viral load of a second swab a second swab taken at the same time and placed in viral transport medium for subsequent qPCR. (Table 4). There was a strong association between viral load detection (RNA copies/mL) determined through RT-PCR and viral antigen detection by LFD (Fig. 1). Confirming earlier analyses, sensitivity of LFDs is highest in samples with higher viral loads [18 19].

Table 4

Results of the Phase 3b evaluations showing viral antigen detection/sensitivity of four LFD tests using dry-swab samples from community sampling tested by Health Care Workers and Laboratory Scientists. Tests were performed by laboratory scientists. Ct – cycle threshold on RT-PCR.

Viral Load	Average Ct	Health Care Workers		Laboratory Scientists
(RNA copies/ml)		Innova Pos/total Prop (95% CI)		Innova Pos/total Prop (95% CI)		abbott pos/total prop (95% ci)		orientate gene pos/total prop (95% ci)		Deepblue  Pos/total Prop (95% CI)
>10million	<18	36/39	0.92 (0.79–0.98)	31/31	1 0.00 (0.89–1)	13/14	0.93 (0.66–1.00)	17/17	1.00 (0.80–1.00)	24/24	1.00 (0.86–1.00)
1–10 million	18–21.5	43/45	0.96 (0.85–0.99)	49/51	0.960 (0.87–1.00)	18/20	0.90 (0.68–0.99)	16/16	1.00 (0.79–1.00)	35/36	0.97 (0.85–1.00)
0.1–1 million	21.5–25	45/48	0.94 (0.83–0.99)	36/40	0.90 (0.76–0.97)	13/13	1.00 (0.75–1.00)	19/20	0.95 (0.75–1.00)	37/37	1.00 (0.91–1.00)
10,000–100,000	25–28	19/38	0.50 (0.33–0.67)	25/32	0.78 (0.60–0.91)	17/19	0.89 (0.67–0.99)	14/17	0.82 (0.57–0.96)	24/33	0.73 (0.54–0.87)
1000–10,000	28–31	11/34	0.32 (0.17–0.50)	11/32	0.34 (0.19–0.53)	9/22	0.41 (0.21–0.64)	9/17	0.53 (0.28–0.77)	9/35	0.26 (0.12–0.43)
100–1000	31–34.5	2/19	0.11 (0.01–0.33)	4/12	0.33 (0.01–0.65)	3/11	0.27(0.06–0.61)	3/8	0.38 (0.09–0.76)	0/12	0.00 (0.00–0.26)
Overall	Na	156/223	0.70(0.63–0.76)	156/198	0.79 (0.72–0.84)	73/99	0.74(0.64–0.82)	78/95	0.82 (0.730–0.89)	129/177	0.73 (0.66–0.79)

Fig. 1

A. Logistic curves showing association between viral detection/sensitivity and viral load (RNA copies/ml) for Abbott, Deepblue, Orientgene LFDs when performed by Lab worker. B. Logistic curve showing association between viral antigen detection/sensitivity and viral load (RNA copies/mL and Ct) for Innova LFD when performed by trained laboratory scientists and trained healthcare workers.

Optimal viral antigen detection/sensitivity when performed by laboratory scientists, was 78.8% (95% CI 72.4–84.3%; 156/198 cases where a paired PCR was performed; see below for differing performance by test operator category). Subgroup analyses showed there were no discernible differences in viral antigen detection/sensitivity in those without symptoms vs. symptomatic individuals (27/41 [65.9%] vs. 249/344 [72.4%], p = 0.37). We did not find any evidence of associations between LFD positivity and symptoms or past medical history, with the exception of presence of headache (Supplementary Table 2).

LFD test performance by operator

As part of Phase 3b-4 evaluations, work was performed to report on the effect of the operator on viral antigen detection/sensitivity in RT-PCR-positive cases using the Innova LFD. Tests were classified according to whether they were performed by a laboratory scientist, a fully trained research health care worker or by a self-trained lay individual working at a regional NHS Test and Trace centre. Performance was optimal when the LFD was used by laboratory scientists (156/198 LFDs positive [78.8%, 95% CI: 72.4–84.3%]) relative to trained healthcare-workers (156/223 LFDs positive [70.0%, 95% CI: 63.5–75.9%]) and self-trained members of the public given a protocol (214/372 LFDs positive [57.5%, 95% CI: 52.3–62.6%]; p<0.0001) (Fig. 2).

Fig. 2

Effect of training and operator on the viral detection/sensitivity of the Innova LFD in COVID-19 PCR-positive patients. Whiskers represent 95% confidence intervals.

Discussion

We report on our national evaluation of SARS-CoV-2 viral antigen-detecting LFDs, focussing on the Innova SARS-CoV-2 Antigen Rapid Qualitative Test, which has a viral antigen detection (sensitivity) of 78.8% when performed by laboratory scientists and a specificity of 99.7%, using RT-PCR as ‘gold standard’ for positive and negative status. Test performance to detect SARS-CoV-2-positive samples was improved at lower Ct values/higher viral loads, and were >90% at Ct values <25 (equating to ~390 pfu/mL, 100,000 RNA copies/ml). We are not able to comment on whether there is a concentration of viral load where LFDs become more sensitive than PCR and further work is required on the precise lower bound of detection using a greater number of samples. However, there is an expanding body of evidence that suggests viral load/antigen is important as individuals with the highest viral loads are the most infectious, [20] and the presence/absence of viral antigens determined by LFDs is more strongly associated with a viral culture than RT-PCR positivity [21]. In our evaluation, test performance was largely maintained across different settings and cohorts; however, performance was partly operator-dependent and kit failures are not infrequent.

Our experience is that many LFDs entering our national evaluation program do not perform at a level required for mass population deployment and this reflects the literature. To date, an increasing number of evaluations of SARS-CoV-2 antigen-detecting LFD have been published with variable results. A number of LFDs show good, [22,23,13,19,24,25] or acceptable sensitivity and specificity, [26,27] however, many studies have identified tests with poor sensitivities or specificities [28,15].

A challenge for most countries during the SARS-CoV-2 pandemic has been the expansion of capacity for diagnostic testing to support the identification of symptomatic and asymptomatic cases. This would aid in offering testing to “contacts” of COVID-19 and enable targeted testing to better safeguard vulnerable populations e.g. care home residents. Reliance on RT-PCR involves significant infrastructural and specialist human resources to implement at increasing scale. Both the World Health Organisation and European commission have issued guidance supporting wider implementation of antigen-targeting LFDs, and in November, Slovakia became the first country in the world to implement entire population testing using LFDs [1,3,29]. The UK has similar aspirations to pursue a strategy of mass testing and has implemented a city wide mass testing in Liverpool using the Innova LFD in this study [30].

It is important to note that there are some potential issues with considering RT-PCR as the gold standard test for COVID-19. Many individuals have persisting viral RNA fragments that can linger for weeks-months without any evidence of active viral replication; in this instance a PCR-positive is likely to overcall the “infectious” status of an individual [31]. Indeed, when compared to the ability to perform viral culture, data suggest that RT-PCR tends to overestimate the presence of replicating or infectious virions. [32].

In field testing, performance of the Innova LFD was dependent on the test operator. Individuals who had read a protocol immediately prior to self-sampling did not perform as well as individuals with hands-on training, or clinical laboratory personnel who had performed several hundred LFD tests. Optimal test performance, is therefore likely to reflect the ability of operators to perform the test according to the manufacturer's instructions in the field. In a hospital staff testing setting, recently published work confirmed good performance of these tests for home-testing [33]. It is assumed that the use of LFDs to successfully identify individuals with higher viral loads and enabling an earlier diagnosis will be of benefit in interrupting transmission. However, like any new test, the magnitude of benefit versus potential for theoretical harm in different settings will need to be ascertained and has caused ongoing debate in the academic community [34 35].

SARS-CoV-2 control will benefit from a variety of testing strategies. This might include those optimised for determining past infection/exposure (e.g. serology), those that are of benefit in determining current/recent infection (e.g. RT-PCR), or those identifying potential infectivity. A combination of approaches incorporating the strengths of each of these tests can be effectively used for individuals and for population-level management of the pandemic. Based on our analyses we do believe there is a role for LFDs in clinical use using the 4 LFDs identified. We do not believe comparisons between these LFDs is appropriate in this dataset as there are slight difference in both the viral load bands and number of tests performed. Approaches to testing will remain relevant even when effective vaccines become available as it may take several months for an appreciable effect on transmission to be fully realised [36].

In conclusion, we completed late stage evaluations of seven LFDs. We report sensitivities of 70–80% and specificities ≥99.7% for each LFD evaluated in phase 3b, which involved testing by laboratory personnel or trained healthcare professionals. To identify patients with higher viral loads (Ct<25), each LFD had >90% sensitivity. Sensitivity was lower in phase 4 evaluations, while specificity was maintained. The simplicity of LFDs, without a requirement for specialist training or equipment, mean that they are an attractive option for mass testing. Future research should focus on post-implementation evaluation of diagnostic accuracy and strategies to improve the sensitivity and accuracy of these tests. This will include the potential benefits of regular serial sampling, digital results interpretation and incorporation of enhanced training strategies utilising videos to improve accuracy and reduce transmission.

Funding

The report presents independent research funded by the National Institute for Health Research, Wellcome Trust and the Department of Health. The views expressed in this publication are those of the authors and not necessarily those of the NHS, Wellcome Trust, the National Institute for Health Research, the Department of Health or Public Health England.

Author contributions

Specific roles within the consortia authorship are listed in the supplementary materials.

Data sharing statement

De-identified testing data is available with an associated data dictionary. Access to data will be determined by the chief investigator, Professor Tim Peto after review of the proposal with a signed data access agreement.

Declaration of Competing Interest

The authors do not have any conflicts of interest.