Real-world evaluation of COVID-19 lateral flow device (LFD) mass-testing in healthcare workers at a London hospital; a prospective cohort analysis.

OBJECTIVES: Real-world evaluation of the performance of the Innova lateral flow immunoassay antigen device (LFD) for regular COVID-19 testing of hospital workers.
METHODS: This prospective cohort analysis took place at a London NHS Trust. 5076 secondary care healthcare staff participated in LFD testing from 18 November 2020 to21 January 2021. Staff members submitted results and symptoms via an online portal twice weekly. Individuals with positive LFD results were invited for confirmatory SARS CoV-2 PCR testing. The positive predictive value (PPV) of the LFD was measured. Secondary outcome measures included time from LFD result to PCR test and staff symptom profiles.
RESULTS: 284/5076 individuals reported a valid positive LFD result, and a paired PCR result was obtained in 259/284 (91.2%). 244 were PCR positive yielding a PPV of 94.21% (244/259, 95% CI 90.73% to 96.43%). 204/259 (78.8%) staff members had the PCR within 36 hours of the LFD test. Symptom profiles were confirmed for 132/244 staff members (54.1%) with positive PCR results (true positives) and 13/15 (86.6%) with negative PCR results (false positives). 91/132 true positives (68.9%) were symptomatic at the time of LFD testing: 65/91 (71.4%) had symptoms meeting the PHE case definition of COVID-19, whilst 26/91 (28.6%) had atypical symptoms. 18/41 (43.9%) staff members who were asymptomatic at the time of positive LFD developed symptoms in the subsequent four days. 9/13 (76.9%) false positives were asymptomatic, 1/13 (7.7%) had atypical symptoms and 3/13 (23.1%) had symptoms matching the PHE case definition.
CONCLUSIONS: The PPV of the Innova LFD is high when used amongst hospital staff during periods of high prevalence of COVID-19, yet we find frequent use by symptomatic staff rather than as a purely asymptomatic screening tool. LFD testing does allow earlier isolation of infected workers and facilitates detection of individuals whose symptoms do not qualify for PCR testing.

Introduction

The World Health organisation has highlighted the importance of widespread testing in reducing the transmission of SARS-CoV-2, the pathogen responsible for the COVID-19 pandemic. Lateral flow immunoassay devices (LFD) have recently been rolled out for the regular testing of healthcare workers (HCW) across the United Kingdom (UK). LFDs are an attractive point-of-care test, detecting the presence of the SARS-CoV-2 viral antigen from self-performed nasal swabs, and typically produce a result within 30 minutes, providing a more rapid, accessible, and affordable alternative to the current gold standard polymerase chain reaction (PCR) test. , It has been suggested that rapid, systematic screening of NHS staff members could reduce nosocomial transmission by detecting asymptomatic or paucisymptomatic infection, particularly as viral load and transmissibility remains high in the absence of symptoms.

Emerging data on the performance of the Innova LFD antigen test have been variable. Research commissioned by Public Health England and the University of Oxford demonstrated an in vivo sensitivity of 76.8% compared with PCR, whereas real-world community mass testing at the University of Liverpool showed a sensitivity of only 40.0%. , We evaluate the performance of the Innova LFD for self-testing by staff members within a real-world clinical setting, during a peak prevalence of COVID-19 infection in the UK.

Methods

Testing program

Commencing 18 November 2020, all 6702 staff members of Chelsea and Westminster NHS Foundation Trust were invited to participate in home testing with Innova LFDs (Innova Medical Group, USA). Those reporting confirmed SARS CoV-2 infection within the preceding 90 days were excluded from the testing program. Staff were provided with Innova LFDs and instructions for test application (including instructions via a video link) in line with the manufacturer information leaflet. Staff were required to submit results via an online portal during the testing period. Staff members with positive results were required to self-isolate according to Government guidance and to undergo confirmatory testing by PCR of nasopharyngeal swabs. Staff developing symptoms of SARS CoV-2 infection were advised to source PCR testing immediately and advised not to undertake LFD testing to investigate their symptoms.

Staff were asked to report any symptoms when submitting positive LFD results via the online portal. All staff reporting a positive LFD result were contacted via telephone within 24 h by dedicated clinical staff to confirm their result, evaluate any symptoms, and to facilitate PCR testing and follow up, including isolation and contact tracing.

Study design

To evaluate the clinical utility of the testing program, we evaluated all LFD results submitted via the online portal between 18 November 2020 and 21 January 2021. Demographics and occupations (both staff group and job role) of all enrolled staff were collected using electronic staff records. Staff in medical, nursing, allied health, domestic, security and portering roles were considered patient-facing, whilst staff in administration, management and operator line roles non-patient-facing. We reported the proportion with positive LFDs. For the purpose of this analysis, true positives were deemed as those seen in individuals with paired positive LFD and PCR results. The positive predictive value (PPV) of the Innova LFD was measured. Secondary outcome measures included time from LFD result to PCR test and symptom profiles of staff members around the time of LFD testing.

Duplicate positive LFD results submitted by individual staff members were excluded. Some staff members were unable to be contacted to confirm results due to incorrect submission of identifiable details, and some staff members reported falsely submitting a positive LFD result via the portal in error. These individuals were excluded from further analysis. The PPV is therefore presented as an estimated value.

Those with positive LFD results but negative PCR tests in the absence of previous SARS CoV-2 infection (false positives) were contacted retrospectively via telephone to confirm their symptomology at the time of the LFD. Symptom profiles were compared against the Public Health England (PHE) Covid-19 case definition, described as any of the following: high temperature, a new cough, or a loss of or change in sense of taste and/or smell.

Regardless of enrolment in the LFD home-testing program, all staff developing symptoms associated with the PHE COVID-19 case definition were required to report their symptoms and undergo a PCR test. In an attempt to determine the negative predictive value of LFDs, we also evaluated any LFD results (if available) of all staff reporting a positive SARS CoV-2 PCR result undertaken for other indications during the study period.

Data analysis

Descriptive statistics were used in the analysis of staff demographics, job role, and concordance between positive LFD and PCR results. For statistical analysis, MedCalc Statistical Software Version 19.8 was used. A positive predictive value for the LFD was calculated. Patient demographics between groups were analysed using Chi-Squared test. Significance threshold was set to p < 0.05.

Results

Overall positive LFDs

The majority of eligible staff self-enrolled for LFD testing during the study period: 5076/6072 (75.4%). Overall, 45,022 LFD test results from 5076 individuals were submitted via the online portal during the study period. Of these, 346/45,022 (0.76%) LFDs were positive. Those who reported mistakenly submitting a positive result via the online portal were removed from the evaluation (n = 27). Results pertaining to staff who were not able to be contacted due to incorrect submission of identifiable details (n = 30) and duplicate LFD results submitted by individual staff members were also excluded from analysis (n = 5) (Fig. 1 ).

Fig. 1

CONSORT diagram summarizing healthcare worker cohort participation and SARS-CoV-2 lateral flow antigen test analysis.

After these exclusions, 284 individuals submitted a valid positive LFD result providing a positivity rate of minimum 284/5076 (5.59%) and maximum 346/5076 (6.82%).

Outcome of confirmatory PCR testing

Of these 284 individuals, 263/284 (92.6%) undertook a confirmatory PCR test, and in 259/263 (98.5%) PCR tests a valid result was obtained. 21 staff members did not respond to the request to attend a confirmatory PCR test. Four PCR results were inconclusive or lost, and therefore were not included in the final analysis (Fig 2. A).

Fig. 2

Summary of paired SARS-CoV-2 lateral flow device (LFD) and polymerase chain reaction (PCR) results of healthcare workers during the study period.

The majority of paired PCR results were positive: 244/259 (94.21%). This yielded an estimated positive predictive value of 244/259, 94.21% (95% CI 90.73% to 96.43%) of the Innova LFD antigen test in staff during the study period. The prevalence of SARS CoV-2 infection in this cohort is estimated as 244/5076 (4.81%) during the nine-week study period.

The demographics of all staff members with a verified positive LFD result are presented in Table 1 . The majority of staff held patient-facing roles: 231/259 (89.19%) of those with confirmatory PCR results and 20/25 (80%) of those without. There was no significant variability in the gender, staff group or job roles between the group with a confirmatory PCR test result and the group without. The positive predictive value for the Innova LFD can therefore be assumed for the group without confirmatory PCR test results.

Table 1

Comparison of demographics between healthcare workers with positive SARS-CoV-2 lateral flow antigen results, with confirmatory PCR testing and without.

	Group with confirmatory PCR (n = 259)	Group without confirmatory PCR (n = 25)	P-value (95% CI)
Gender
Male	75 (28.96%)	4 (16%)	0.17 (−6.40–24.17)
Female	184 (71.04%)	21 (84%)	0.17 (−6.40–24.17))
Staff Group
Additional Clinical Services	44 (16.99%)	3 (12%)	0.52 (−13.42–14.31)
Administrative and Clerical	16 (6.18%)	4 (16%)	0.07 (−0.44–28.62)
Allied Health Professionals	14 (5.41%)	1 (4%)	0.76 (−14.28–6.19)
Domestic and security services	4 (1.54%)	0	0.53 (−11.81–3.90)
Medical and Dental	50 (19.31%)	4 (16%)	0.69 (−15.84–14.24)
Nursing and Midwifery	118 (45.56%)	12 (48%)	0.82 (−16.53–21.88)
Scientific and Technical	7 (2.70%)	0	0.41 (−10.69–5.47)
Support Services	3 (1.16%)	1 (4%)	0.25 (−1.11–18.40)
Volunteer	1 (0.39%)	0	0.75 (−12.93–2.16)
Unknown	2 (0.77%)	0	0.66 (−12.56–2.77)
Job Role
Administrator	9 (3.47%)	3 (12%)	0.04 (0.14–26.56)
Audiologist	2 (0.77%)	0	0.66 (−12.56–2.77)
Cleaner	3 (1.16%)	0	0.59 (−12.18–3.35)
Clinical Support Worker	16 (6.18%)	2 (8%)	0.72 (−5.00–18.95)
Consultant (Doctor)	15 (5.79%)	0	0.22 (−7.72–9.33)
Dietician	1 (0.39%)	0	0.75 (−12.93–2.16)
Healthcare Assistant	22 (8.49%)	1 (4%)	0.43 (−11.31–9.69)
Helpdesk Operator	1 (0.39%)	0	0.75 (−12.93–2.16)
Junior Doctor	34 (13.13%)	4 (16%)	0.69 (−7.80–21.86)
Manager	11 (4.25%)	3 (12%)	0.09 (−0.71–25.80)
Matron	3 (1.16%)	0	0.59 (−12.18–3.35)
Midwife	18 (6.95%)	0	0.17 (−6.60–10.72)
Nurse	90 (34.75%)	11 (44%)	0.36 (−9.09–28.98)
Nurse Practitioner	1 (0.39%)	0	0.75 (−12.93–2.16)
Operating Department Practitioner	3 (1.16%)	0	0.59 (−12.18–3.35)
Pharmacist	6 (2.32%)	0	0.44 (−11.06- 4.97)
Physician Associate	1 (0.39%)	0	0.75 (−12.93–2.16)
Radiographer	3 (1.16%)	0	0.59 (−12.18–3.35)
Therapist	15 (5.79%)	1 (4%)	0.71 (−13.92–6.63)
Volunteer	1 (0.39%)	0	0.75 (−12.93–2.16)
Unknown	3 (1.16%)	0	0.59 (−12.18–3.35)
Patient contact
Patient facing	231 (89.19%)	20 (80%)	0.17 (−2.78–28.59)
Non-patient facing	25 (9.65%)	5 (20%)	0.11 (−1.56–29.72)
Unknown	3 (1.16%)	0	0.59 (−12.18–3.35)

Time to confirmatory PCR testing

Of 259 staff with a confirmatory PCR test result, 104 (40.2%) had PCR on the same day. 100/259 staff (38.6%) had LFD and PCR one day apart, or within 36 h. 28/259 staff members (10.8%) had LFD and PCR tests performed two days apart and 8/259 (3.1%) three days. PCR and LFD were performed more than three days apart in 17/259 staff (6.6%). PCR results were processed within 24–48 h.

Symptomology of staff members

The symptom profiles of staff with positive confirmatory PCR results (true positives) were confirmed for 132/244 staff members (54.1%). Of those, 91/132 (68.9%) were symptomatic at the time of LFD testing. 65/91 (71.4%) had symptoms consistent with the PHE case definition of SARS-CoV-2, whereas 26/91 (28.6%) did not have any of the symptoms fitting the PHE case definition but had atypical symptoms including headache, malaise and coryzal symptoms. 41/132 (31.1%) were asymptomatic at the time of LFD testing. At least 18 of these (43.9%) went on to develop symptoms at a later time point.

Staff members with negative confirmatory PCR results (false positives) were contacted to establish symptomatology. Of these, 13/15 staff members responded (86.7%); 9/13 (76.9%) were asymptomatic, 1/13 (7.7%) had coryzal symptoms not matching the PHE case definition and 3/13 (23.1%) had symptoms consistent with the PHE case definition of COVID-19. One of these staff members had two negative LFD tests whilst symptomatic prior to a positive LFD test.

False negative LFD rate

During the study period, 521 HCWs had a positive PCR test result for SARS CoV-2. PCR tests were performed following a positive LFD test for those enrolled in staff self-testing (n = 244),or undertaken for other indications e.g. development of symptoms (n = 277) (Fig 2.B). Of these, 36 HCWs had a negative LFD result within the preceding 72 h, giving an estimated false negative rate of 36/521 (6.91%). A true negative predictive value cannot be calculated without paired PCR tests for those with negative LFD results.

Discussion

Evaluation in this large real-world cohort demonstrates a high positive predictive value of the Innova LFD for detection of SARS CoV-2 in a routine self-testing program among hospital staff when background prevalence is high. We find SARS-CoV-2 prevalence to be higher than the maximum prevalence in the local London borough during this surge period, which was 0.74% (0.36–1.49%) at its lowest and 3.5% (2.16–5.76%) at its peak. Symptom profiles at the time of testing reveal that LFD adoption detected COVID-19 in currently asymptomatic and pre-symptomatic true positive individuals (those with positive matched PCR tests), as well as in true positive individuals with mild symptoms not fitting the PHE case definition and therefore not qualifying for PCR testing.

Our findings corroborate those of Downs et al., who observed a similar positive predictive value of 96% for the Innova LFD antigen test when used for regular testing of asymptomatic hospital workers. Our study established a significantly higher confirmatory PCR test rate of 91%, whereas Downs et al. only acquired confirmatory PCR tests in 52%, providing robust additional evidence to support the high positive predictive value in this high-risk cohort. The higher uptake of PCR testing was likely due in part to the hospital's COVID-19 screening infrastructure. Staff testing was facilitated by clinical specialists, who ensured follow up of all positive LFD cases, engaging these individuals in PCR testing, subsequent result processing with review of symptomatology, management of positive individuals and household contacts.

Risk of nosocomial COVID-19 infection is high; initial studies in Wuhan revealed that the majority of HCW infections were contracted from patients or other HCWs, with only 12% contracting COVID-19 in the community. In our evaluation, regular LFD testing of staff members allowed early detection of COVID-19 infections prior to symptom onset, ensuring earlier isolation of infectious HCWs and therefore a likely reduction in the risk of nosocomial transmission to colleagues and patients. Regular LFD testing also allowed detection and isolation of asymptomatic individuals and individuals with atypical symptoms (two-thirds of this cohort) that would not qualify staff members for PCR testing under the current guidance. Without LFD testing, these infected staff members would have continued to work, posing a risk to other HCWs and patients alike.

Significant operator-dependent variability in LFD performance has been shown, with higher sensitivity of tests performed by trained patient-facing staff compared with lay people. Improved operator technique in this study, which evaluates LFD testing by hospital staff of whom the majority were in patient-facing roles, may account for the improved performance of the Innova LFD observed than in testing of the general population. , Despite increased experience in using diagnostic tests, a noteworthy number of staff members made errors in interpreting and recording their LFD results. Furthermore, a significant proportion of positive LFD results were observed in symptomatic staff, demonstrating use of the test beyond that of the intended asymptomatic screening tool. Although such use is expected in a real-world setting and allows earlier detection of paucisymptomatic individuals, increased viral load associated with presence of symptoms may contribute to the higher PPV than observed in studies of asymptomatic individuals. The PPV of the Innova LFD antigen test in this study is also likely impacted by the background high prevalence of COVID-19, as well as improved operator proficiency, and therefore may not translate into use for widespread testing of the public at periods of low-prevalence.

Limitations in the study design include weakness of the online data entry form; staff members were required to input their own identifiable details, LFD results and symptoms, with human error leading to exclusion of some results as staff were unable to be followed up. Only positive LFD results were confirmed with PCR testing, unless the staff member developed symptoms. Without matched PCR results for all negative LFD results, the true negative rate and therefore negative predictive value is not known. However, the clinical impact of this may be negligible; several studies have demonstrated that the LFD antigen tests detect individuals with high viral loads, and those that are missed have low viral loads and are less likely to be infectious. , In contrast, the PCR test detects small amounts of genetic material that may linger in the respiratory tract for weeks to months, and long after the person is infectious. Moreover, the specificity of the Innova LFD has shown to be consistently high suggesting that a negative LFD likely represents a true negative status. The safety of allowing staff members back to work on the basis of a negative LFD test is, however, still undetermined in a real-world setting. , ,

Another limitation of this study is the use of nasopharyngeal and throat swab PCR testing as the gold-standard confirmation of a positive case, which has an estimated sensitivity of 73.3% (CI 68•1–78.0%). Overall, 3/13 (23.1%) staff members with a positive LFD but negative PCR result had symptoms consistent with COVID-19 infection, potentially reflecting false negative PCR results given the test's limited sensitivity or an element of cross-reactivity with other respiratory viruses producing similar symptoms to COVID-19.

Given the high positive predictive value, this evaluation provides encouraging evidence to support the use of rapid antigen tests for screening and prevention of nosocomial transmission in this high-risk group during COVID-19 surges. As detailed, PCR testing should be employed for confirmation in asymptomatic individuals, as the false positive rate indicates that a significant economic cost would be incurred by the 10-day absence of staff isolating unnecessarily.

Further work is needed to establish the true negative predictive value of the LFD antigen test in this cohort. At the time of writing, plans for mass roll-out of LFD in general populations in England were in development. The utility of LFDs in populations and time periods with lower pre-test probability remains undetermined and should be robustly evaluated.

Declaration of Competing Interests

JH received research funding from CW+ Charity and the Westminster Medical School Research Trust, and received honoraria from Gilead. NM has received speaker fees from Beyer (2016) and Pfizer (2019) and received educational support from Eumedica (2016) and Baxter (2017). LSPM has consulted for or received speaker fees from bioMerieux (2013–2021), Pfizer (2018–2021), Eumedica (2016–2021), DNAelectronics (2015–18), Dairy Crest (2017–2018), Umovis Lab (2020–2021), Shionogi (2021), Pulmocide (2021), and received research grants from the National Institute for Health Research (2013–2021), CW+ Charity (2018–2021) and LifeArc (2020–2021). RJ has received honoraria, speaker fees, travel support and/or research grant funding from Gilead, ViiV Healthcare, BMS, Abbvie, Janssen and Merck. All other authors have no conflicts of interest to declare.