Point-of-care serological assays for delayed SARS-CoV-2 case identification among health-care workers in the UK: a prospective multicentre cohort study.

BACKGROUND: Health-care workers constitute a high-risk population for acquisition of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. Capacity for acute diagnosis via PCR testing was limited for individuals with mild to moderate SARS-CoV-2 infection in the early phase of the COVID-19 pandemic and a substantial proportion of health-care workers with suspected infection were not tested. We aimed to investigate the performance of point-of-care and laboratory serology assays and their utility in late case identification, and to estimate SARS-CoV-2 seroprevalence.
METHODS: We did a prospective multicentre cohort study between April 8 and June 12, 2020, in two phases. Symptomatic health-care workers with mild to moderate symptoms were eligible to participate 14 days after onset of COVID-19 symptoms, as per the Public Health England (PHE) case definition. Health-care workers were recruited to the asymptomatic cohort if they had not developed PHE-defined COVID-19 symptoms since Dec 1, 2019. In phase 1, two point-of-care lateral flow serological assays, the Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Bitotech, Poway, CA, USA) and the Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China), were evaluated for performance against a laboratory immunoassay (EDI Novel Coronavirus COVID-19 IgG ELISA kit [Epitope Diagnostics, San Diego, CA, USA]) in 300 samples from health-care workers and 100 pre-COVID-19 negative control samples. In phase 2 (n=6440), serosurveillance was done among 1299 (93·4%) of 1391 health-care workers reporting symptoms, and in a subset of asymptomatic health-care workers (405 [8·0%] of 5049).
FINDINGS: There was variation in test performance between the lateral flow serological assays; however, the Encode assay displayed reasonable IgG sensitivity (127 of 136; 93·4% [95% CI 87·8-96·9]) and specificity (99 of 100; 99·0% [94·6-100·0]) among PCR-proven cases and good agreement (282 of 300; 94·0% [91·3-96·7]) with the laboratory immunoassay. By contrast, the Onsite assay had reduced sensitivity (120 of 136; 88·2% [95% CI 81·6-93·1]) and specificity (94 of 100; 94·0% [87·4-97·8]) and agreement (254 of 300; 84·7% [80·6-88·7]). Five (7%) of 70 PCR-positive cases were negative across all assays. Late changes in lateral flow serological assay bands were recorded in 74 (9·3%) of 800 cassettes (35 [8·8%] of 400 Encode assays; 39 [9·8%] of 400 Onsite assays), but only seven (all Onsite assays) of these changes were concordant with the laboratory immunoassay. In phase 2, seroprevalence among the workforce was estimated to be 10·6% (95% CI 7·6-13·6) in asymptomatic health-care workers and 44·7% (42·0-47·4) in symptomatic health-care workers. Seroprevalence across the entire workforce was estimated at 18·0% (95% CI 17·0-18·9).
INTERPRETATION: Although a good positive predictive value was observed with both lateral flow serological assays and ELISA, this agreement only occurred if the pre-test probability was modified by a strict clinical case definition. Late development of lateral flow serological assay bands would preclude postal strategies and potentially home testing. Identification of false-negative results among health-care workers across all assays suggest caution in interpretation of IgG results at this stage; for now, testing is perhaps best delivered in a clinical setting, supported by government advice about physical distancing. FUNDING: None.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spread extensively following its identification in December, 2019, becoming a global pandemic by March, 2020. More than 13 800 000 cases have been reported and 593 000 deaths attributed to COVID-19 worldwide, as of July 18, 2020. Substantial public health isolation measures have been adopted in an attempt to slow the spread of infection. Case finding strategies have predominantly relied on PCR assays during the acute infection phase, through centralised specialist laboratories. In the UK, testing capacity in the early period of the COVID-19 pandemic was mostly limited to patients who were admitted to hospital with COVID-19 symptoms, and only later extended to include symptomatic health-care workers. Health-care workers constitute a population that is at substantially greater risk of contracting SARS-CoV-2 infection due to the rate and nature of exposure associated with clinical care of positive cases. Personal protective equipment (PPE) and stringent infection prevention and control measures aim to mitigate this risk and minimise both nosocomial infection of health-care workers and onward transmission. During the initial period, when the case rate was at its peak but PCR testing was not yet widely available, a large proportion of symptomatic health-care workers were not tested. Thus, SARS-CoV-2 prevalence among UK health-care workers remains largely unknown. Where the infection rate in asymptomatic health-care workers is similar to that seen in general community transmission, targeted testing of symptomatic individuals might be better placed to inform infection rates among health-care workers.

Evidence before this study

The evidence for the performance and limitations of various serological assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is scarce. We searched PubMed for academic publications and online search engines for relevant grey literature on May 1, 2020, and repeated the search on May 26, 2020. A summary provided by Wu and McGoogan in February, 2020, of characteristics and important lessons from China, highlighted the considerable risk to health-care workers of the potential for nosocomial infection. Studies by Hunter and colleagues and Treiber and colleagues provide further insight, through PCR testing, into the burden of COVID-19 in health-care workers in the UK. The capacity to test health-care workers during the early phase of the COVID-19 pandemic in the UK was limited but can now be mitigated by delayed case identification through serology testing, providing insight into the overall burden of SARS-CoV-2 infection among health-care workers. To date, suitably sized studies have not yet been done in the UK.

Added value of this study

This prospective, multicentre cohort study demonstrates the comparative utility of SARS-CoV-2 serology tests, looking at both lateral flow assays and ELISA approaches. Our study also provides one of the first large-scale insights into seroprevalence data in a cohort of health-care workers with high COVID-19 exposure in the UK. Methodologically, demonstration of colorimetric band intensity on lateral flow assays and its correlation with optical density on ELISA provides a degree of confidence in the interpretation of high-intensity bands but reinforces the limitations of interpretation of lateral flow assays (and risks identification of false positives) when the bands appear weak. Additionally, our study is the first to demonstrate the substantial risks associated with delayed reading of lateral flow assays (in terms of both false positives and false negatives) and is informative when considering delayed reading (eg, postal testing) as part of a testing strategy if shared at this early stage in the planning process.

Implications of all the available evidence

Variation in performance characteristics between assays highlights the urgent need for individual evaluation of the large number of commercial SARS-CoV-2 serology tests that have become rapidly available. Practically, the observation of false-negative serology results among health-care workers provides valuable information considering their messaging around interpretation of serology results at this early stage in the scale up of serological testing. False-negative results have a clear impact on the manner in which serological testing might be used to augment and support physical distancing policies, as well as implications for the development of large-scale testing pathways. Further research is required into the full scope of serological testing for SARS-CoV-2 infection and factors associated with failing to mount a detectable immune response to SARS-CoV-2 infection in otherwise healthy individuals. This study also demonstrates the potential limitations of single-target immunoassays for SARS-CoV-2 and should help inform future research studies, where further evaluation is required not just of alternative assays but also through the comparison of the various epitope targets that are currently available.

A variety of pathways to enhance case finding have been considered. These include point-of-care molecular platforms for acute phase testing, and laboratory ELISA or lateral flow serological assays for antibodies specific to SARS-CoV-2 for delayed case identification.4, 5, 6 Although lateral flow serological assays potentially offer rapid results in either the point-of-care setting or home reading or postal testing, concern exists around test performance characteristics, particularly in the first 2 weeks after onset of symptoms, as well as their poor positive predictive value when applied to a general population. By contrast, although laboratory-based ELISA kits might offer improved test performance characteristics, they have undergone only limited clinical evaluation.

We have previously evaluated the performance of lateral flow serological assays against PCR in moderate to severe SARS-CoV-2 infection. In the present study we aimed to explore the utility of these assays in a population of health-care workers with mild to moderate community-managed SARS-CoV-2 infection (all health-care workers delivering direct clinical care to high-acuity SARS-CoV-2-positive patients). We compared two lateral flow serological assays against laboratory-based ELISA and PCR, where available, doing an interval analysis of test performance characteristics and their suitability for delayed case identification. Subsequent screening of the remaining symptomatic health-care workers was completed to assess seroprevalence.

Methods

Study setting and design

A prospective multicentre SARS-CoV-2 serological testing programme was implemented on April 8, 2020, and data were collected until June 12, 2020, across two hospitals in London, UK, comprising 6440 employees. Health-care workers were eligible for serological testing if they had delivered direct clinical care to SARS-CoV-2-positive inpatients in cohort areas or isolation rooms involving aerosol-generating procedures; and had experienced mild to moderate symptoms matching the Public Health England (PHE) case definition for SARS-CoV-2 infection, including fever or cough, or both, with breathlessness or anosmia, or both, with onset at least 14 days before testing.

Specialist staff (redeployed from HIV and sexual health services) screened staff for clinical symptoms; collected demographic data (including age, sex, ethnicity, and job role); and carried out venepuncture for the serum sample facilitating inoculation of the point-of-care lateral flow serological assay and matched laboratory ELISA.

The study was done in two phases. Phase 1 involved an evaluation of serology performance characteristics, comparing the matched lateral flow serological assays with ELISA. Phase 2 comprised an estimate of seroprevalence in symptomatic and asymptomatic health-care workers through further lateral flow serological assay testing. An interval analysis was done once testing had occurred for at least 90% of the reported number of symptomatic health-care workers to human resources during the study period (reached on June 12, 2020). Health-care workers were recruited to the asymptomatic cohort if they had satisfied inclusion criteria as above but had not experienced any PHE-defined COVID-19 symptoms since Dec 1, 2019. Recruitment was consecutive. Statistical analysis was done with IBM SPSS Statistics (version 26). A STARD (Standards for Reporting of Diagnostic Accuracy Studies) checklist is available in the appendix (pp 5–6).

Phase 1: serological testing

Lateral flow serological assay testing was done with the Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Biotech, Poway, CA, USA) and the Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China). Lateral flow serological assays were completed as per the manufacturers' instruction leaflets. Lateral flow serological assays were read at 15 min by two clinical staff experienced in the use of point-of-care analysis (appendix p 4). Lateral flow serological assay readers, and subsequently ELISA staff, were masked with respect to any previous PCR results. A visual scoring system for evaluating immunochromatography rapid diagnostic kits has previously been described for chikungunya virus. Based on this system, lateral flow serological assays were recorded as positive or negative with readings visually scored as absent (0), very weak positive (1), weak positive (2), medium positive (3) or strong positive (4), and compared with ELISA optical density readings. SDs of values in each visual group were compared with overall results. The mean ranks were compared for significance by use of the Kruskal-Wallis test. Dunn's post-hoc tests were then done for pairwise comparisons and results reported as p values, with a significance threshold of p<0·050.

Comparator ELISA testing was done with the qualitative EDI Novel Coronavirus COVID-19 IgG ELISA kit (Epitope Diagnostics, San Diego, CA, USA), targeting IgG antibodies to the nucleocapsid protein of SARS-CoV-2. Post-marketing manufacturer information reported a sensitivity of 98·4% and specificity of 99·8%. ELISA was completed as per the manufacturer instruction leaflet and results were recorded as an optical density reading. Where equivocal results were found, confirmation testing was done with the Abbott SARS-CoV-2 IgG (anti-nucleocapsid) chemiluminescent microparticle immunoassay (Abbott Laboratories, Lake Bluff, IL, USA) on the ARCHITECT i1000SR immunoassay analyser and the final result included in the analysis.

Phase 1: determining test performance characteristics

To derive a measure of sensitivity, the results of lateral flow serological assays and ELISA were compared in 300 health-care workers who had previously received PCR testing (AusDiagnostics, Sydney, Australia) at initial presentation with COVID-19 symptoms. As per local hospital guidelines, PCR testing was done where possible between days 1 to 7 inclusive since onset of symptoms. A measure of specificity was derived through testing, with ELISA and lateral flow serological assays, of historical negative serum samples (50 samples from among patients with infectious or inflammatory presentations from August, 2018, and 50 maternal antenatal screening samples from August, 2019). A further evaluation was made on agreement between lateral flow serological assay results and ELISA through interpretation of lateral flow serological assay visual scores in relation to ELISA optical density readings, particularly very weak and weak positive bands, to identify potential issues with variability in reading of lateral flow serological assays. Lateral flow serological assay cassettes were observed at 15 min post-sampling, 2 h post-sampling, and 24 h post-sampling, and readings recorded. If a change was noted in the appearance of a band (negative to positive reading) or disappearance of a band (positive to negative reading) at 2 h or 24 h, a correlation was made against the ELISA result to establish the risk of late readings (eg, if postal return methods are used in conjunction with self-testing).

Phase 2: determining SARS-CoV-2 seroprevalence in health-care workers

Phase 1 of the study analysed ELISA against both lateral flow serological assays. In phase 2, the remaining health-care workers fulfilling the inclusion criteria were invited for serological testing with a lateral flow serological assay. To assess the significance of the sensitivity of the lateral flow serological assays (using an estimated sensitivity of 90%), we did a power calculation, setting α at 0·05 and β at 0·20. We estimated symptomatic infection to be 90% among health-care workers. We calculated that the study would require 132 PCR-positive health-care workers in phase 2. In order to achieve this sample size, based on self-reporting of PCR-positive cases to human resources, we estimated that testing coverage in phase 2 would need to capture at least 90% of all reported symptomatic health-care workers.

Additionally, following an update in government advice in mid-May, 2020, to offer testing to asymptomatic health-care workers, we tested a subset of asymptomatic health-care workers who were offered serology testing in order to more accurately estimate SARS-CoV-2 seroprevalence in the entire workforce.

Positive and negative predictive values based on test performance characteristics were then calculated if considered for use in three cohorts: in the UK general population (based on estimates of UK prevalence during our cohort's period of majority reported symptom onset of 2·7%); in the asymptomatic health-care worker population if general screening was to be applied, with a derived prevalence from a subset of asymptomatic health-care workers; and in health-care workers reporting PHE-defined symptoms during the study period, with a derived prevalence from serological testing of symptomatic health-care workers.

Role of the funding source

There was no funding source for this study. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

In phase 1 we compared matched two-way lateral flow serological assays against ELISA for 300 serum samples from health-care workers with an additional 100 historical pre-COVID-19-negative control samples (figure 1 ). The mean age of health-care workers in this cohort was 39·1 years (range 22·7–71·0) and 218 (72·7%) of 300 health-care workers were female. 70 (23·3%) of 300 health-care workers had a suitably timed positive PCR result before serology testing (table 1 ). In phase 2, serosurveillance was done among 1299 (93·4%) of 1391 health-care workers reporting symptoms and in 405 (8·0%) of 5049 asymptomatic health-care workers. All health-care workers were tested more than 14 days after onset of symptoms. 592 (45·6%) of 1299 health-care workers were tested more than 21 days (range 14–54) after symptom onset. 24 (6·0%) of 400 samples (18 of 300 health-care workers, six of 100 negative controls) returned equivocal ELISA results. Upon confirmation testing with the Abbott SARS-CoV-2 IgG assay, 12 of 18 samples from health-care workers were positive and all six control group samples were negative. All 400 samples were included in the analysis. No lateral flow serological assay cassette failures were noted.

Figure 1

Results of health-care workers and negative controls tested with IgG ELISA and Encode and Onsite split IgM/IgG antibody lateral flow serological assays in London, UK, from April to June, 2020

Phase 1: comparison of matched samples tested with the Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Biotech, Poway, CA, USA) and Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China) LFAs and ELISA (EDI Novel Coronavirus COVID-19 IgG ELISA kit [Epitope Diagnostics, San Diego, CA, USA]). 100 historical serum samples were evaluated to assess specificity (Table 2, Table 3). Equivocal results were retested with the Abbott SARS-CoV-2 IgG (anti-nucleocapsid) chemiluminescent microparticle immunoassay (Abbott Laboratories, Lake Bluff, IL, USA). Phase 2: further analysis of LFA testing offered to all symptomatic health-care workers, including initial LFAs tested in phase 1 (n=300) and phase 2 (n=1299). Further analysis of asymptomatic health-care workers with LFA testing (n=405). LFA=lateral flow serological assay. PHE=Public Health England.

Table 1

Sensitivity of ELISA, Encode LFA, and Onsite LFA in PCR-positive health-care workers in London, UK, in April–June, 2020

	Phase 1		Phase 2
	PCR positive (n=70)	Sensitivity	PCR positive (n=136)	Sensitivity
ELISA positive	60	85·7% (95% CI 75·3–92·9)	NA	NA
ELISA negative	10	..	NA	..
Encode LFA positive	65	92·9% (95 CI 84·1–97·6)	127	93·4% (95% CI 87·8–96·9)
Encode LFA negative	5	..	9	..
Onsite LFA positive	63	90·0% (95% CI 80·5–95·9)	120	88·2% (95% CI 81·6–93·1)
Onsite LFA negative	7	..	16	..

ELISA=EDI Novel Coronavirus COVID-19 IgG ELISA kit (Epitope Diagnostics, San Diego, CA, USA). NA=not available. Encode=Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China). LFA=lateral flow serological assay. Onsite=Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Bitotech, Poway, CA, USA). Where equivocal results were found on ELISA sampling, testing was repeated with the Abbott SARS-CoV-2 IgG (anti-nucleocapsid) chemiluminescent microparticle immunoassay (Abbott Laboratories, Lake Bluff, IL, USA). Phase 1: total PCR-positive matched health-care workers in phase 1 (70 of 300). Phase 2: matched LFA results in 136 PCR-positive health-care workers (AusDiagnostics, Sydney, Australia).

Table 2

Agreement for IgG detection between Encode LFA and Onsite LFA and ELISA matched samples

	IgG ELISA positive	IgG ELISA negative*	Historic negative serum samples (n=100)	Agreement with ELISA IgG	IgG specificity	IgG plus IgM specificity
Encode LFA positive	138	15	1	..	..	..
Encode LFA negative	3	144	99	94·0% (95% CI 91·3–96·7)	99·0% (95% CI 94·6–100·0)	98·0% (95% CI 93·0–99·8)
Onsite LFA positive	123	28	6	..	..	..
Onsite LFA negative	18	131	94	84·7% (95% CI 80·6–88·7)	94·0% (95% CI 87·4–97·8)	94·0% (95% CI 87·4–97·8)

Encode=Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China). LFA=lateral flow serological assay. Onsite=Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Bitotech, Poway, CA, USA). ELISA=EDI Novel Coronavirus COVID-19 IgG ELISA kit (Epitope Diagnostics, San Diego, CA, USA).

ELISA testing of historic negative samples showed 95 of 100 as negative: specificity 95·0% (95% CI 88·7–98·4).

Table 3

Agreement between the Encode and Onsite LFAs

	Encode LFA positive	Encode LFA negative
Onsite LFA positive	137	24
Onsite LFA negative	14	125
Encode LFA and Onsite LFA agreement	87·3% (95% CI 83·6–91·1)	87·3% (95% CI 83·6–91·1)

LFA=lateral flow serological assay. Encode=Encode SARS-CoV-2 IgM/IgG One Step Rapid Test Devices (Zhuhai Encode Medical Engineering, Zhuhai, China). Onsite=Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Biotech, Poway, CA, USA). Where equivocal results were found on ELISA (n=24), testing was repeated with the Abbott SARS-CoV-2 IgG (anti-nucleocapsid) chemiluminescent microparticle immunoassay (Abbott Laboratories, Lake Bluff, IL, USA). Specificity data are derived from testing of historical negative serum samples (n=100).

All 100 negative controls were included (mean age 38·7 years [range 2·0–80·1]; 79 [79·0%] of 100 individuals in the control group were female). Relevant virology and serology data are presented in the appendix (p 3).

Samples from all 300 health-care workers were run against both lateral flow serological assays and laboratory immunoassays. 141 (47%) of 300 samples had a positive IgG ELISA result; 138 (98%) of 141 were positive on the Encode lateral flow serological assay and 123 (87%) were positive on the Onsite lateral flow serological assay (figure 1). IgM-only positive cassettes were reported on the Encode lateral flow serological assay (eight [2·7%] of 300) and Onsite lateral flow serological assay (11 [3·7%] of 300). Of the 300 health-care workers evaluated, 159 (53%) had a negative IgG ELISA result; 144 (91%) of 159 were IgG negative on the Encode lateral flow serological assay and 131 (82%) of 159 were negative on the Onsite lateral flow serological assay (figure 1). Agreement between results for ELISA with Encode lateral flow serological assays was seen for 282 of 300 samples (94·0% [95% CI 91·3–96·7]), for ELISA with Onsite lateral flow serological assays agreement was seen for 254 of 300 samples (84·7% [80·6–88·7]) and for Encode with Onsite lateral flow serological assays agreement was seen for 262 of 300 samples (87·3% [83·6–91·1]; Table 2, Table 3 ).

70 (23·3%) of 300 health-care workers had positive SARS-CoV-2 PCR results from the time of initial symptom onset. 60 (85·7% [95% CI 75·3–92·9]) of these 70 samples were positive on laboratory ELISA (table 1). 65 (92·9% [95% CI 84·1–97·6]) of 70 samples were positive on the Encode lateral flow serological assay, whereas 63 (90·0% [80·5–95·9]) of 70 were positive with the Onsite lateral flow serological assay (table 1). All PCR-positive cases reported as IgG positive on ELISA were also positive on the Encode lateral flow serological assay. Five (7%) of 70 samples that were negative on laboratory ELISA were also negative on both lateral flow serological assays. 29 health-care workers had a negative PCR recorded. Of these, 27 (93·1%) of 29 were negative on ELISA and the Encode lateral flow serological assay. 26 (89·7%) of 29 were negative on the Onsite lateral flow serological assay.

95 of 100 ELISA-run samples were IgG negative (specificity 95·0% [95% CI 88·7–98·4]), 99 of 100 Encode lateral flow serological assays were IgG negative (specificity 99·0% [94·6–100·0]), and 94 of 100 Onsite lateral flow serological assays were IgG negative (specificity 94·0% [87·4–97·8]; table 2).

Where visual scores of lateral flow serological assays were recorded at 15 min for band visibility, these were compared to laboratory ELISA optical density readings (figure 2 ; appendix pp 1–2). Where the Encode assay was reported to have reactive bands (153 [51·0%] of 300), the appearance of bands was recorded as very weak (16 [10·5%] of 153), weak (51 [33·3%] of 153), medium (44 [28·8%] of 153), or strong (42 [27·5%] of 153). Three (2·0%) of 147 negative Encode assays were IgG positive on ELISA, and 15 (9·8%) of 153 positive Encode assays were negative on ELISA (seven very weak, six weak, and two medium; appendix p 1). Where the Onsite assay was reported to have reactive bands (151 [50·3%] of 300), the appearance of bands was recorded as very weak (30 [19·9%] of 151), weak (32 [21·2%] of 151), medium (41 [27·2%] of 151), or strong (48 [31·8%] of 151). 18 (12·1%) of 149 negative Onsite assays were positive on ELISA testing, and 28 (18·5%) of 151 positive Onsite assays were negative on ELISA testing (13 very weak, eight weak, six medium, one strong; appendix p 2). Kruskal-Wallis testing of visual scores for Encode and Onsite lateral flow serological assays against optical density showed a significant difference between groups (p<0·0001). SDs were calculated for each visual score and Dunn's pairwise tests were done between paired groups (figure 2).

Figure 2

Distribution of visual scores for Encode (A) and Onsite (B) LFAs against optical density readings of ELISA-matched samples

Readings of the Onsite CTK Biotech COVID-19 split IgG/IgM Rapid Test (CTK Biotech, Poway, CA, USA) and the Encode SARS-CoV-2 split IgM/IgG One Step Rapid Test Device (Zhuhai Encode Medical Engineering, Zhuhai, China) LFAs at 15 min plotted against ELISA (EDI Novel Coronavirus COVID-19 IgG ELISA kit [Epitope Diagnostics, San Diego, CA, USA]) optical density for all health-care worker matched samples (n=300). Scoring correlates to a negative (0), very weak positive (1), weak positive (2), medium positive (3), and strong positive (4) reading. Reference photographs selected from mean optical density value cassette for each score. SDs of Encode lateral flow serological assay values: negative (0·09), very weak (0·21), weak (0·26), medium (0·21), and strong (0·20) visual scores. Dunn's pairwise tests showed no significant difference between visual scores, comparing very weak to medium (p=0·35), medium to strong (p=0·064), very weak to weak (p=1·00), and weak to medium (p=1·00) visual scores. All other pairwise comparisons had a significant relationship with optical density (p<0·0001). SDs of Onsite lateral flow serological assay values: negative (0·17), very weak (0·21), weak (0·23), medium (0·29), and strong (0·30) visual scores. Dunn's pairwise tests showed no significant difference between visual scores comparing very weak to weak (p=1·00), weak to medium (p=0·062), and medium to strong (p=1·00) visual scores. All other pairwise comparisons had a significant relationship with optical density (p<0·0043).

All 800 lateral flow serological assay cassettes were read at 15 min, 2 h, and 24 h after sampling. At 2 h no change was noted from the initial readings. At 24 h a change was noted in 35 (8·8%) of 400 Encode lateral flow serological assay tests. 29 of 35 tests changed from negative to positive, and six of 35 changed from positive to negative. ELISA results were concordant with initial Encode lateral flow serological assay readings at 15 min. 39 (9·8%) of 400 Onsite lateral flow serological assays also demonstrated a change in readings at 24 h, with 15 of 39 cassettes changing from negative to positive (ELISA confirmed seven as positive and eight as negative) and 24 of 39 becoming unreadable.

In phase 2, 1299 health-care workers working in high-acuity areas were tested (including phase 1 participants) with a lateral flow serological assay (mean age 38·2 years [range 20·5–71·7]; 931 [71·7%] female). 581 (44·7%) of 1299 participants had a positive IgG band, and 684 (52·7%) were negative. 34 (2·6%) of 1299 health-care workers had an IgM-only band (figure 1). 1391 of 6440 health-care workers reported PHE-defined symptoms up until the date of analysis. Where IgG seroprevalence among symptomatic health-care workers was estimated at 44·7% (95% CI 42·0–47·4), this corresponded to 622 of 1391 individuals. Of the 1299 (93·4%) of 1391 health-care workers tested, 136 had a positive PCR result (table 1).

Of the 1299 symptomatic health-care workers, 842 (64·8%) reported fever, 1043 reported cough (80·3%) and 298 (22·9%) reported anosmia. χ2 testing for independent association of symptoms with seropositivity was significant for fever (χ2 9·01, p=0·0027) and anosmia (χ2 43·0, p<0·0001). A significant association was not found for cough (χ2 1·92, p=0·17).

405 asymptomatic health-care workers were tested (mean age 42·4 years [range 20·3–72·1]; 298 [73·6%] female). 43 (10·6%) of 405 asymptomatic health-care workers had a positive IgG band. 5049 health-care workers had not experienced symptoms throughout the study period. IgG seroprevalence among symptomatic health-care workers was estimated therefore at 10·6% (95% CI 7·6–13·6), corresponding to 535 individuals. Estimated IgG seropositivity for the workforce was calculated through the combination of estimates for symptomatic health-care workers (622 [44·7%] of 1391) and asymptomatic health-care workers (535 [10·6%] of 5049). This would correspond to an estimated seroprevalence across the entire workforce of 18·0% (95% CI 17·0–18·9).

Seroprevalence was considered in three separate cohorts as described. In the UK general population (2·7%), for Onsite the positive predictive value was 29·0% (95% CI 15·8–47·1) and negative predictive value was 99·7% (99·5–99·8); and for Encode the positive predictive value was 72·2% (26·9–94·8) and negative predictive value 99·8% (99·7–99·9). In the asymptomatic health-care worker cohort (10·6%), for Onsite the positive predictive value was 63·6% (95% CI 44·5–79·2) and negative predictive value was 98·5% (97·7–99·1); and for Encode the positive predictive value was 91·7% (61·2–98·7) and negative predictive value was 99·2% (98·5–99·6). In health-care workers reporting symptoms (44·7% of symptomatic health-care workers), for Onsite the positive predictive value was 92·2% (95% CI 84·5–96·3) and negative predictive value was 90·8% (86·2–94·0); and for Encode the positive predictive value was 98·7% (91·5–99·8) and negative predictive value was 94·9% (90·8–97·2).

Discussion

In this study we report good positive and negative predictive values for delayed serological testing in a carefully selected population with suspected SARS-CoV-2 infection. These findings have the potential to support case identification strategies, depending on the device used. Testing could be particularly useful in improving our understanding of the true prevalence of community managed, mild to moderate infection among high-risk health-care workers while informing the analysis of infection prevention and control strategies for any future waves of the ongoing COVID-19 pandemic.

Testing in this high-exposure cohort based in London, UK, has identified several key lessons that could substantially affect future plans for wider serological testing among health-care workers.

Although a wide range of rapid diagnostic point-of-care serological assays have emerged in a short period of time, individual assay analysis has been challenging in individuals with mild to moderate infection as numbers of so-called true-positive PCR-tested individuals available for comparison are limited. Concerns about variability in sensitivity between lateral flow serological assays are borne out in our data, when these assays were compared against laboratory ELISAs. We demonstrate good agreement for detection of IgG with the Encode lateral flow serological assay when our pre-test criteria are applied (94·0%); however, there was reduced agreement with the Onsite lateral flow serological assay (84·7%). This result would therefore limit the utility of the Onsite assay even within the high-risk health-care worker population, where a much higher pre-test probability exists than in the general population. Among the PCR-positive subgroup, results were concordant between individual assays, with the Encode lateral flow serological assay identifying 100%, and the Onsite 97%, of health-care workers who were also positive on ELISA. The failure of all three assays to identify PCR-positive health-care workers with PHE-defined symptoms highlights the limitations of relying on any single diagnostic platform, or perhaps identifies a cohort of individuals who did not mount a timely serological response.

Where reduced sensitivity among serological assays in the early infection period has been previously described,6, 14 it is clear that any use of such assays in supporting early return-to-work policies could carry considerable risk. Such policies or recommendations should therefore be actively discouraged. Additionally, the proportion of individuals who are seronegative on both ELISA and lateral flow serological assays, despite being PCR-positive, is of concern. Any use of serological testing should be accompanied with a reinforcement of PPE and infection, prevention and control advice, regardless of result, and therefore testing should be undertaken face to face in a clinical setting. Immunity inferred by the presence of antibodies is yet to be determined and clinical counselling of result interpretation at this stage as purely case identification is an important part of the testing process, preventing misunderstanding and incorrect behaviour modification (eg, relaxing physical distancing rules or reducing PPE). This assessment could be done at established UK National Health Service (NHS) facilities, as we have done here, or potentially done via mobile health units as demonstrated by early identification of HIV infection through point-of-care CD4 testing in South Africa.

We found visual scoring of immunochromatography lateral flow serological assay bands to correlate with ELISA optical density. More prominent bands were seen in positive cases and the weakest bands were seen in those initially reported as equivocal. We did, however, find laboratory ELISA-negative samples with very weak lateral flow serological assay bands, casting doubt on the ability to interpret these results. The risk of reporting false positives is likely to be increased further by non-expert reading, such as with home testing seroprevalence studies. Where bands have a strong or medium immunochromatography reading, however, ELISA was positive in almost all (84 [98%] of 86) cases for the Encode lateral flow serological assay, reflecting greater reliability. In comparison, seven of 89 Onsite cassettes with medium to strong bands or strong bands were ELISA negative. Across both lateral flow serological assays, the finding of IgM-only bands might represent greater capability among the lateral flow serological assays to identify positive cases, or false-positive or cross-reactive IgM results. Among the historical negative samples, reactivation of Epstein-Barr virus might have accounted for a false-positive result. Although manufacturers have reported no observed cross-reactivity with seasonal coronaviruses, the sample sizes in the present study were not large enough and further evaluation is required in a larger cohort of health-care workers. An apparent failure to seroconvert to IgG among those with confirmed SARS-CoV-2 infection has been documented in one study and this effect might account for some of the IgM-only cases. Although we observed IgG-positive results in PCR-positive individuals as late as 2 months after symptom onset, the limited PCR positive rate among health-care workers makes analysis of test performance as a function of time difficult to assess in this study. Further analyses with alternative laboratory ELISA platforms, including those with IgM targets as well as IgG spike-protein receptor-binding-domain targets (in addition to the nucleocapsid protein targets), are urgently required to determine the reasons for these discordant findings. Longitudinal serial testing will also be informative.

When considering the utility of lateral flow serological assays, one of the key benefits is the availability of real-time results at the point of testing. Where we investigated delayed reading of lateral flow serological assay cassettes, a considerable number changed at 24 h, becoming either entirely unreadable or leading to the reporting of a different result. This effect has previously been reported with a high incidence of true non-reactive results changing to weak positive when undergoing delayed reading of rapid antibody tests for HIV. We therefore strongly advise against any use of lateral flow serological assays in home testing programmes that involve non-expert reading of cassettes or return postage of samples for evaluation at extended time intervals.

Early work in PCR testing of health-care workers in the UK, through testing of symptomatic individuals but also through serial screening of asymptomatic individuals, suggests that infection rates in this cohort might reflect wider community transmission rather than primarily nosocomial infection.3, 18 This observation is in keeping with first reports of large-scale testing of health-care workers in China. Where PCR testing is not as widely available, serological testing offers an additional avenue for estimating infection rate among health-care workers. Where fewer than half of those working in the highest risk areas and presenting with PHE-defined symptoms had reactive results, estimation of total rates was similar to previously reported rates. However, we found some discordance in our asymptomatic cohort when compared to previously reported data. Serological screening done among health-care workers in Essen, Germany, found a seropositive rate of 1·6%, of which the majority of individuals did not report PHE-defined symptoms; yet in our study we reported a seroprevalence of 10·6%. Recent data from a study in Oxford, UK, however are supportive of our findings, where 1016 (10·7%) of 9456 IgG-positive results were reported in a large asymptomatic health-care worker cohort. Although all three studies were done during the same time period, the incidence rate and population density in the UK were considerably higher and such marked variation in seroconversion reflects the impact of geographical variation in exposure intensity and reinforces the need for granular serological data. In stark contrast to seroprevalence among individuals with PHE-defined symptoms (44·7%), the findings from our asymptomatic cohort are perhaps more in keeping with the observed peak among the general population in London (7·1%), with a slight increase reflecting the increased exposure faced by health-care workers. There is growing evidence, however, that those with mild disease are less likely to produce a detectable antibody response than those with more severe disease, and specifically those with lower respiratory tract involvement. A relative difference in exposure might also partly explain the observed variability in seroprevalence, with minimal doses of virus perhaps insufficient to induce an adaptive immune response, but instead dealt with by the innate immune system. In this context, until the differential antibody response and its neutralising capacity in SARS-CoV-2 are better understood, generalised testing of asymptomatic health-care workers, where positive predictive value is substantially reduced, should be done only with considerable caution alongside detailed advice about the current limitations for interpretation. When pre-test criteria are applied (14 days after onset of PHE-defined symptoms), the positive predictive value is substantially increased. Further investigations among this cohort with anti-receptor binding domain assays might provide additional information to the anti-nucleocapsid assay used here, where reduced sensitivity is a clear issue.

Although PCR is recognised as the gold standard in the diagnosis of acute SARS-CoV-2 infection, our study was limited to a subgroup analysis of PCR-positive health-care workers because of the reduced availability of PCR testing during the early phase of the pandemic. As PCR testing capacity was increased, we conversely noted a substantial decline in health-care workers presenting with acute phase symptoms. Where PCR data were available, reduced sensitivity for all three assays was observed compared to information provided by the manufacturer. This finding might reflect limitations in the use of these assays within individuals with mild to moderate infection, with many assay development characteristics reported in testing of inpatients with severe disease, and supports the requirement of individual assay evaluation within intended populations before widespread use. That our data demonstrate general agreement between lateral flow serological assays and PCR, however, is encouraging, but the reduced agreement among lateral flow serological assays in general is concerning. This potential discrepancy could be explained by the utilisation of separate assay targets (nucleocapsid or spike protein) in lateral flow serological assays. We were unable to discern further target information from both manufacturers. Where clear information remains unavailable for assay targets, as with many currently available commercial lateral flow serological assays, greater understanding of any neutralising effect of different antibodies will be even more important. Although our study was done at multiple sites, its capacity to reflect estimated prevalence in secondary care facilities in London alone and elsewhere was probably limited; appropriately powered studies are now required in locations with a lower estimated prevalence to understand the likely prevalence among health-care workers at a national level. This study is further limited by the use of an IgG-target laboratory ELISA to the nucleocapsid protein only. Additional analysis with a laboratory ELISA capable of detecting the spike protein, and in particular the receptor binding domain, would allow for further evaluation of the reliability of the observed results with lateral flow serological assays.

In conclusion, serological testing, whether with laboratory-based ELISA platforms or lateral flow serological assays designed for use at the point of care, offer the potential for late case identification among health-care workers with mild to moderate SARS-CoV-2 infection. We found variability in the performance of different lateral flow serological assays when compared against an IgG-target laboratory ELISA. Building on our previously reported work of moderate to severe SARS-CoV-2 infection among inpatients with PCR-confirmed SARS-CoV-2 infection, we found suitable performance characteristics from the Encode lateral flow serological assay device (and slightly less so with Onsite) but only if utilised in patients with PHE-defined symptoms at least 14 days following symptom onset. Although lateral flow serological assays might provide a clear mechanism to identify those health-care workers who have contracted SARS-CoV-2 infection, it is crucial to note that cassettes provide incorrect results if delayed reading is done and this must be a key consideration for any future plans for large-scale testing with lateral flow serological assays.

This online publication has been corrected. The corrected version first appeared at thelancet.com/respiratory on July 30, 2020

Data sharing

The data analysed during the current study and further details about the assays are available from the corresponding author on reasonable request, as long as this meets local ethical and research governance criteria.