Diagnostic accuracy of loop-mediated isothermal amplification coupled to nanopore sequencing (LamPORE) for the detection of SARS-CoV-2 infection at scale in symptomatic and asymptomatic populations.

OBJECTIVES: Rapid, high throughput diagnostics are a valuable tool, allowing the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in populations so as to identify and isolate people with asymptomatic and symptomatic infections. Reagent shortages and restricted access to high throughput testing solutions have limited the effectiveness of conventional assays such as quantitative RT-PCR (RT-qPCR), particularly throughout the first months of the coronavirus disease 2019 pandemic. We investigated the use of LamPORE, where loop-mediated isothermal amplification (LAMP) is coupled to nanopore sequencing technology, for the detection of SARS-CoV-2 in symptomatic and asymptomatic populations.
METHODS: In an asymptomatic prospective cohort, for 3 weeks in September 2020, health-care workers across four sites (Birmingham, Southampton, Basingstoke and Manchester) self-swabbed with nasopharyngeal swabs weekly and supplied a saliva specimen daily. These samples were tested for SARS-CoV-2 RNA using the Oxford Nanopore LamPORE system and a reference RT-qPCR assay on extracted sample RNA. A second retrospective cohort of 848 patients with influenza-like illness from March 2020 to June 2020 were similarly tested from nasopharyngeal swabs.
RESULTS: In the asymptomatic cohort a total of 1200 participants supplied 23 427 samples (3966 swab, 19 461 saliva) over a 3-week period. The incidence of SARS-CoV-2 detection using LamPORE was 0.95%. Diagnostic sensitivity and specificity of LamPORE was >99.5% (decreasing to approximately 98% when clustered estimation was used) in both swab and saliva asymptomatic samples when compared with the reference RT-qPCR test. In the retrospective symptomatic cohort, the incidence was 13.4% and the sensitivity and specificity were 100%.
CONCLUSIONS: LamPORE is a highly accurate methodology for the detection of SARS-CoV-2 in both symptomatic and asymptomatic population settings and can be used as an alternative to RT-qPCR.

Introduction

Coronavirus disease 2019, caused by an emergent novel betacoronavirus known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2); represents a public health emergency [1].

Rapid detection of infected cases in order to limit transmission, remains challenging as most validated methods use quantitative RT-PCR (RT-qPCR) [2]. Although considered the reference standard for diagnosis, RT-qPCR is laborious and can be difficult to scale up for mass-testing; in addition, competition for reagents/equipment from many laboratories may lead to widespread reagent shortages. Initially in the outbreak, laboratories throughout the UK used primers that were designed to target sequences within the RNA-dependent polymerase gene (RdRp) [3]; however, these lacked sensitivity [4]. RT-qPCR primer sets were introduced that targeted the envelope (E), nucleocapsid (N) and ORF1ab genes, which provided the necessary increased sensitivity [5,6].

Loop-mediated isothermal amplification (LAMP) offers an alternative to RT-qPCR [7]. This reaction typically takes 20–30 minutes, which is considerably quicker than PCR [8].

Nanopore sequencing allows rapid sequencing using protein nanopores embedded in a lipid membrane [9]. Nanopore sequencing technology allows all the advantages of conventional next-generation sequencing technology, especially the capacity to perform very high (limited only by the number of barcodes available) sample multiplexing within the same sequencing run.

LamPORE is a combination of LAMP and nanopore sequencing, developed by Oxford Nanopore Technologies (Oxford, UK) [10]. This technology has a theoretical maximum capacity of 15 000 samples per GridIon Mk 1 machine (Oxford Nanopore Technologies) per 24 hours, allowing scalability and high throughput.

Use of alternative sampling strategies, such as saliva, could theoretically increase capacity over swabbing (because of less pressure on the supply chain) and increase compliance because of the less invasive nature of sampling of saliva [11].

This study aimed to assess the assay performance characteristics of the LamPORE SARS-CoV-2 Detection Assay against the reference standard RT-qPCR for SARS-CoV-2 detection in both symptomatic and asymptomatic populations from multiple independent centres.

Materials and methods

Study design

The study consisted of a retrospective and prospective diagnostic accuracy study comparing the performance of LamPORE sequencing of the ORF1ab, N2 and E gene targets of SARS-CoV-2 against RT-qPCR of the ORF1ab and N1 gene targets of SARS-CoV-2 [12].

Participants

For the prospective study, 1200 health-care workers at high risk of asymptomatic transmission were recruited as part of a consented National Health Service (NHS) England and NHS Improvement service evaluation in September 2020. Prospective participants performed nasopharyngeal self-swabs at days 0, 7, 14 and 21 as well as daily saliva sampling for 21 days (Fig. 1 ). Participants were recruited from staff working within five sites: University Hospitals Birmingham NHS Foundation Trust, Birmingham Women's’ and Children's’ NHS Foundation Trust, University Hospital Southampton NHS Foundation Trust, Hampshire Hospitals NHS Foundation Trust and Manchester University NHS Foundation Trust. The retrospective study was undertaken by collating surplus samples from patients whose diagnostic samples had been sent to the Public Health England West Midlands laboratory for respiratory panel testing for influenza-like illness from January 2020 to June 2020 (National Research Ethics Service Committee West Midlands—South Birmingham 2002/201 Amendment Number 4).

Fig. 1

Graphical representation of recruitment strategy for collecting saliva and swabs. Day of study is shown below thick black horizontal line. Nasopharyngeal swab sampling timings are represented by thick red vertical arrows. Saliva sample timings are represented by thin black vertical arrows.

Test methods

Sampling

For swab-based tests, participants underwent self-directed nasopharyngeal swabbing using flocked swabs containing viral transport medium. For saliva-based tests, participants were instructed to dribble at least 1 mL of saliva into a universal specimen container without any additive. Samples were tested immediately if returned on the day of testing or if received on Friday they were stored for a maximum of 4 days at 4°C then tested.

Clinical material used for analytical performance

To assess the limit of detection and precision, a tenfold dilution series (from 20 000 copies/mL to 0.2 copies/mL) of droplet digital polymerase chain reaction (ddPCR) quantified SARS-CoV-2 was used and tested in triplicate. A panel of respiratory viruses (Zeptometrix Respiratory Panel R2; Zeptometrix, Buffalo, NY, USA) was used to assess the specificity of the LamPORE assay.

RNA extraction

For all samples at the Birmingham site, samples were heated to 56°C for a minimum of 30 minutes to inactivate live SARS-CoV-2 and. RNA extraction was performed. For every batch of RNA extraction performed (95 samples/batch) an RNA extraction control was used consisting of heat-inactivated SARS-CoV-2 grown on Vero-E6 cells (PHE SARS-CoV-2 England reference strain).

Reference test (RT-qPCR)

Single-step RT-qPCR against the ORF1ab and N1 gene targets of SARS-CoV-2 was carried out using the CerTest ViaSure SARS-CoV-2 real-time PCR kit (CerTest Biotech SL, Zaragoza, Spain) according to the manufacturer's instructions for use on ThermoFisher QuantStudio 5 or BioMolecular Systems MIC instruments, using 5 μL of extracted RNA per reaction [2,12].

Comparator test (LamPORE)

For each sample, 20 μL of RNA sample underwent amplification and sequencing using the LamPORE technique as per the manufacturer's protocol. A proprietary Guppy/VSEARCH/SnakeMake pipeline (algorithmic details as per James et al. [13]) that aligned reads to the viral target genes and a human β-actin (ACTB) internal control and reported results in absolute reads per sample per gene.

Analysis

Sample size calculation

Sample size was determined pragmatically, based on the incidence seen in the UK at the time of the study (1%). Sample size was calculated using R code (using R 3.6.3 [14]) from the methodology of Stark et al. [15] for binary diagnostic test outcomes (a = 0.05, b = 0.90) setting a base sensitivity and specificity of RT-qPCR of 95% and 99%, respectively. We aimed to be able to detect a change of sensitivity and specificity of 10% in LamPORE, respectively giving a sample size greater than 9600 in the prospective cohort.

Results interpretation

The readers of RT-qPCR and LamPORE tests were blinded to any clinical information relating to study participants. For the RT-qPCR reference assay, SARS-CoV-2 was said to be detected if the following conditions were met; amplification of the kit internal control, amplification of either the ORF1ab or N gene with a cycle threshold (Ct) < 38, detection of the positive control on the sample plate, detection of the RNA extraction control on the sample plate, and no SARS-CoV-2-specific amplification in the negative control. Ct values were calculated automatically using instrument software with automatic baseline setting calculated. All curves were manually inspected by two investigators to check for quality and inhibition of reaction.

LamPORE

Aligned read counts were generated via the LamPORE pipeline against ORF1ab (labelled AS1), E1 and N2 genes, as well as a human ACTB gene internal control. Any unaligned reads were marked as Undetermined. Samples were called positive if any of the SARS-CoV-2 target genes had >50 reads/sample, indeterminate if between 20 and 50 reads and negative if < 20 reads. ACTB gene counts were not used as part of the calling algorithm but were used to infer sufficient sampling.

Test results were compared using a 2 × 2 table and standard measures of sensitivity, specificity, positive predictive value and negative predictive value were calculated using R. If results from either the RT-qPCR or LamPORE test were missing or indeterminate then no comparison was made and the sample was removed from the analysis. Standard analyses of variability in diagnostic precision were made, and modifications were made to the analysis to carry out clustered analyses of diagnostic precision using clustered logistic regression with a sandwich estimator (Stata 16.1; StataCorp, College Station, TX, USA), on the basis that each separate study represented a cluster within the whole and may bias estimates of sensitivity and specificity [16].

Results

For the prospective asymptomatic study, a total of 1200 participants who were at work and reported to be well were recruited across the four sites (Birmingham n = 600, Southampton n = 200, Basingstoke/Winchester n = 200, Manchester n = 200). There were no adverse events. Sample flow is shown in Fig. 2 .

Fig. 2

Flowchart showing sample numbers at each stage. Pass = sample passed assay quality control; Indeterminate = sample passed quality control but did not have a clear result; Fail = sample failed assay quality control.

Analytical performance

LamPORE reliably detected SARS-CoV-2 to the 20 copies/mL of sample. SARS-CoV-2 reads were detected in the 0.2 copies/mL sample but this was below the threshold for calling a positive sample in LamPORE and were not detected with RT-qPCR (Table 1 , Fig. 3 ).

Table 1

Dilution series of SARS-CoV-2 and LamPORE

Concentration (copies/mL)	ORF1ab reads (median)	E1 reads (median)	N2 reads (median)	RT-qPCR result	ORF1ab Ct	N Ct
20 000	6429	808	2288	Detected	10.9	14.7
2000	1385	18	602	Detected	28.1	24.1
200	27	4	82	Detected	30.7	30.7
20	67	6	979	Detected	N/D	33
2	0	0	45	Not detected	N/D	N/D
0.2	16	0	0	Not detected	N/D	N/B

Abbreviations: Ct, cycle threshold; E1, envelope protein 1 gene; LamPORE, loop-mediated isothermal amplification (LAMP) coupled to nanopore sequencing technology; N2, nucleocapsid protein 2 gene; RT-qPCR, quantitative RT-PCR; SARS-CoV-2, severe acute respiratory symdrome coronavirus 2.

Fig. 3

ORF1ab (black), E1 (fuchsia) and N2 (taupe) reads in serial dilution series of SARS-CoV-2 for LamPORE. Detection threshold shown by red dotted line.

Intra-assay and inter-assay precision was calculated against the ORF1ab gene. For intra-assay precision on a single day, the standard deviation of ORF1ab was 50 reads with a coefficient of variation (CV) of ±2.3% (see Supplementary material, Table S1). For inter-assay precision across multiple days, the standard deviation of ORF1ab was 178 reads with a CV of ±7.8% (see Supplementary material, Table S2).

For reproducibility for 24 replicates the standard deviation for the ORF1ab gene was 128 reads with a CV of ±3.9% (see Supplementary material, Table S3).

For analytical specificity of the LamPORE assay, SARS-CoV-2 was not detected in any of the samples within the respiratory virus panel. In terms of quality control, the median number of reads reported aligned per sample to ACTB was 571 (range 76–7249 reads). The median ratio of mapped (to SARS-CoV-2 or ACTB) to unmapped reads was a median of 0.71 (interquartile range –0.63 to 0.79). There were generally lower levels of reads seen in extracted viral transport medium versus saliva in ACTB (mean 646 versus 206 reads, p < 0.001), ORF1ab (mean 146 versus 98 reads, p < 0.001) and E1 (14 versus 8 reads, p < 0.01) but not in N2 (22 versus 17 reads, p 0.06).

In order to understand the context in which LamPORE operates, a comparative experiment was carried out examining the sensitivity of our reference RT-qPCR assay compared with fluorimetric LAMP (Optigene LAMP kit) and LamPORE (see Supplementary material, Table 4). This showed that LamPORE detected three additional positive samples detected by RT-qPCR that were not detected by RT-LAMP.

Test results

All participants

In total, 23 427 samples were obtained from all participants, of which 22 401 were from the asymptomatic study and 848 were from the retrospective symptomatic cohort. Both LamPORE (comparator assay) and RT-qPCR (reference assays) were performed on all 23 427 samples (Table 2 ).

Table 2

Diagnostic assay performance tables for RT-qPCR versus LamPORE assay for whole cohort, asymptomatic cohort (swabs), asymptomatic cohort (saliva) and retrospective symptomatic cohort

Whole cohort
		RT-qPCR
		Positive	Negative
LamPORE	Positive	477	124	601
	Negative	2	22 824	22 826
		479	22 948
Sensitivity	Specificity	PPV	NPV
99.58% (95% CI 98.46%–99.95%)	99.46% (95% CI 99.36%–99.55%)	79.37% (95% CI 76.34%–82.10%)	99.99% (95% CI 99.9%–100%)
Asymptomatic cohort—swabs
		Positive	Negative
LamPORE	Positive	23	11	34
	Negative	0	3932	3932
		23	3943
Sensitivity	Specificity	PPV	NPV
100% (95% CI 85.2%–100%)	99.72% (95% CI 99.5%–99.7%)	67.65% (95% CI 53.7%–79.1%)	100%
Asymptomatic cohort—saliva
		Positive	Negative
LamPORE	Positive	187	112	299
	Negative	2	18 134	18 136
		189	18 246
Sensitivity	Specificity	PPV	NPV
98.9% (95% CI 96.2%–99.9%)	99.4% (95% CI 99.3%–99.5%)	62.5% (95% CI 58.1%–66.8%)	99.9% (95% CI 99.9%–100%)
Symptomatic cohort
		Positive	Negative
LamPORE	Positive	116	0	116
	Negative	0	752	752
		116	752	868
Sensitivity	Specificity	PPV	NPV
100% (95% CI 96.9%–100%)	100% (95% CI 99.51%–100%)	100%	100%

Abbreviations: LamPORE, loop-mediated isothermal amplification (LAMP) coupled to nanopore sequencing technology; RT-qPCR, quantitative RT-PCR.

Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for each cohort are shown at the bottom of each sub-table.

Of the 601 samples positive on the LamPORE assay, 477 were also positive and 124 were negative by RT-qPCR and in comparison with the reference assay 124 were false positives. Of the 22 826 negative samples, 22 824 were confirmed as negative by the RT-qPCR and there were two samples that were positive by RT-qPCR.

The diagnostic sensitivity of the LamPORE assay compared with the Certest ViaSure RT-qPCR assay was 99.58% (95% CI 98.46%–99.95%) and the diagnostic specificity (DSp) was 99.46% (95% CI 99.36%–99.55%). The positive predictive value (PPV) of the test with a tested population incidence of 2.04% was 79.37% (95% CI 76.34%–82.10%) and the negative predictive value (NPV) calculated with a prevalence of 2.04% was 99.99% (95% CI 99.9%–100.0%).

When modelled at 1% population prevalence the PPV dropped to 66.24% and at 0.1% population prevalence the PPV was 16.3%. NPV remained at >99.99% in all population scenarios.

When a clustered regression analysis was performed to calculate clustered sensitivity and specificity on all participants (when the cohorts were used as the cluster definition), sensitivity decreased to 98.52% (95% CI 94.75%–99.82%) and specificity was 97.39% (95% CI 97.14%–97.62%. Positive predictive value dropped to 22.62% (95% CI 21.03%–24.29%) and NPV to 99.99% (95% CI 99.95%–100%).

If sensitivity and specificity were calculated with regard to the cohort (n = 1200, see Supplementary material, Table S5) rather than individual test results, and either a positive or negative test was counted at any point in the 21 days of sampling, 66 were positive by LamPORE with an additional two participants detected by qPCR and 1031 testing negative by LamPORE with a further 101 testing negative by qPCR, giving a sensitivity of 97.06% (95% CI 89.78%–99.64%), a specificity of 91.08% (95% CI 89.26%–92.67%), a PPV of 39.52% (95% CI 35.07%–44.16%) and a negative predictive value of 99.81% (95% CI 99.25%–99.95%).

Asymptomatic cohort

For the asymptomatic cohort (Table 2) a total of 22 401 participant samples were tested, with 333 positive (34 swab, 299 saliva) samples being identified, of which 210 samples (23 swab, 187 saliva) were true positive and 123 samples (11 swab, 112 saliva) were false positive when compared with RT-qPCR. There were 22 068 negative samples in total, of which 22 026 samples (3932 swab, 18134 saliva) were true negative and two samples (both saliva) were false negatives. For this cohort there was a diagnostic sensitivity of 99.64% (95% CI 98.0%–99.9%), a diagnostic specificity of 99.48% (95% CI 99.38%–99.57%), a PPV of 69.44% (95% CI 53.7%–79.1%) and an NPV of 99.48% (95% CI 99.97%–100.00%). For the RT-qPCR assay, the mean ORF1ab Ct was 17.1 (range 16.2–37.2) and the N1 Ct was 14.3 (range 11.0–37.2).

Symptomatic cohort

There was complete agreement between the RT-qPCR and LamPORE assays for 116 positive samples and 752 negative samples, for this cohort resulting in a diagnostic sensitivity of 100%, diagnostic specificity of 100%, PPV of 100% and NPV of 100%. The incidence of SARS-CoV-2 was 13.4% over the study period.

Variability across time course

To understand the utility of LamPORE across the time course of infection, a single participant who was identified as the beginning of their infection, early in the study with a long time course (5 days) of positivity was studied with daily saliva sampling as per protocol (Fig. 4 ). Initially a high viral load, indicative of a Ct value of 19.5 was observed, which increased (indicating decreasing viral load) over the 5 days to 23.5 and then became undetectable at day 6. Only LamPORE N2 reads were detectable at day 1, but E1 reads became detectable at day 2 and ORF1ab reads at day 4.

Fig. 4

Line plot showing data from daily saliva sampling of a single participant reporting symptoms and their cycle threshold for the N1 gene (red dashed line, left y-axis, reverse order) and read count (right y-axis) for ORF1ab (green line), E1 (purple line) and N2 (orange line). Days since symptoms began shown on x-axis.

To understand how the assay performed across the whole spectrum of viral loads, we plotted Ct against numbers of reads for the ORF1ab and N2 genes (see Supplementary material, Figs S1 and S2). There was no strong correlation between number of reads and Ct values, we hypothesized that tjhis was a result of the nature of the non-linear amplification techniques used.

Discussion

We carried out a very large asymptomatic cohort study of health-care workers using a novel technology, LamPORE, comparing it with a reference RT-qPCR assay. We found that LamPORE has high sensitivity and specificity (>99%) in both the asymptomatic and symptomatic populations, directly comparable to RT-qPCR, and therefore has comparable predictive ability across a range of uses in varying levels of population prevalence. We studied a population with a wide range of viral loads as determined by cycle threshold, with LamPORE demonstrating good detection across the range.

LamPORE has the advantage that it is scalable [10] to allow testing of very large population levels because of the use of sample barcoding allowing pooling of up to 3500 samples on a single GridIon instrument. Due to the increased sensitivity of LAMP as part of the LamPORE system, it gives very high sensitivity for SARS-CoV-2. With combinatorial barcoding, as has been adopted in other population level assays [17] on larger flow cells (e.g. a Promethion flow cell), even greater sample multiplexing may be achievable. Another potential inherent advantage is the ability to multiplex gene targets allowing the detection of multiple respiratory pathogens [18] such as SARS-CoV-2, influenza and respiratory syncytial virus. It is not known what the upper limit of multiplexing of LAMP primers is, and they are considerably more complex to design than PCR primers [19]. Given the advantages of LAMP in terms of speed of amplification [8] and sensitivity of detection, an exploration of LAMP multiplexing is urgently required. Also, the assay chemistry uses different enzymes and methodologies to PCR, meaning a diversification of supplies and therefore potentially fewer reagent shortages in a pandemic setting. However, LAMP plus sequencing introduces several steps into the workflow, which means that LamPORE becomes inherently ‘non-linear’, i.e. the relationship between the genes amplified and sample viral load is not linear, so LamPORE may only be used to infer positivity rather than giving any measure of viral load.

A potential disadvantage of LamPORE is the differing workflow needed to prepare samples, including the LAMP step and library preparation, barcoding then sequencing. This requires more sample preparation steps than an RT-qPCR workflow.

During the testing of the asymptomatic cohort, we observed a number of false positives using LamPORE when compared with the RT-qPCR assay. There are a number of possible explanations for this observation. First, LAMP amplification is more sensitive than PCR amplification [20], so contamination risk is high; however, as the laboratories refined the technique, contamination issues seemed to resolve. Second, it is feasible that some of the samples are in fact, true positives as demonstrated by the ability of LamPORE to detect spiked, killed virus beyond the limit of detection of RT-qPCR. This may have useful implications for sample pooling [21], as greater sensitivity would allow more samples to be pooled and tested. Finally, the LamPORE protocol requires multiple manual liquid handling steps, which can lead to error and increases the number of opportunities for contamination to occur. We found two false-negative samples within the saliva cohort. These had high Ct values (ORF1ab Ct 36.5 and 37.1) in the PCR, and we hypothesize that this may have occurred because of low viral loads as the extracted RNA went through a freeze thaw before it was run on LamPORE.

In conclusion, we have demonstrated the accuracy of LamPORE across a range of population use cases, maintaining a high specificity and sensitivity, reproducibility and limit of detection, as well as working well on saliva samples, making it suitable for the detection of symptomatic and asymptomatic SARS-CoV-2 infections.

Author contributions

Conceptualization was by ADB, AR, SH, AD and DP; data were curated by ADB and CW; formal analysis was by ADB, AR, AW, DB, SH and AD. ADB was responsible for the funding acquisition and software. ADB, AR, AW, VF, SK, SH, AD, ZD and DP contributed to the methodology. Project administration was by SH, AD, ZD and DP; supervision and validation were by ADB, AR, SK and VF; and visualization was by ADB and AR. All authors contributed to the investigation, to writing the original draft, and to reviewing and editing.

Transparency declaration

ADB has received travel funding from the Oxford Nanopore Community Meeting 2019 from Oxford Nanopore. The other authors declare no competing interests.

Funding

The research in this study was funded by an unrestricted grant from the United Kingdom Department of Health and Social Care via NHS Test and Trace; and NHS England and NHS Improvement. ADB is currently supported by a Cancer Research UK Advanced Clinician Scientist award (C31641/A23923) and his laboratory is supported by CRUK Centre Birmingham (C17422/A25154) and the Birmingham Experimental Cancer Medicine Centre (C11497/A25127). The Department of Health and Social Care gave input into the study design and coordination but had no influence on the analysis and conclusions of the paper. This report presents independent research and the views expressed in this publication are those of the authors and not necessarily those of the NHS, or the Department of Health and Social Care.