Performance of lateral flow assays for SARS-CoV-2 compared to RT-qPCR.

Dear Editor

We have read with interest the study by Lamb et al. on the real-world performance of a lateral flow immunoassay antigen device (LFD) for regular COVID-19 testing of hospital workers. LFDs have found widespread application for broad-range screening, with subsequent confirmation of LFA-positive cases by RT-qPCR. While, in practice, an acceptably low number of positive LFD results turned out as false positives when validated with RT-qPCR, there is accumulating evidence that the number of positive cases missed, i.e., false negative results obtained with LFDs is unacceptably high. The LFD used in the study by Lamb et al. (Innova Medical Group, USA) had a high positive predictive value (PPV) when frequently used during periods of high prevalence of COVID-19. As the authors did not use the same sample for confirmation by RT-qPCR, however, it was not possible to directly link the test outcome to a certain cT value, or to evaluate the specificity.

We selected 10 common LFDs [Abbott (Panbio, Abbott Rapid Diagnostics Jena GmbH, Jena, Germany), Acon (Flowflex, Acon Laboratories Inc., CA, USA), Clungene (Hangzhou Clungene Biotech Co., Ltd., Tianjin, China), Joysbio (Joysbio (Tianjin) Biotechnology Co., Ltd., Tianjin, China), Lepu Medical (Beijing Lepu Medical Technology Co., Ltd., Peking, China), Nadal (nal von minden GmbH, Moers, Germany), Orient (Zhejiang Orient Gene Biotech Co., Ltd., Zhejiang, China), Realy Tech (Hangzhou Realy Tech Co., Ltd., Zhejiang, China), Roche (SD Biosensors, Gyeonggi-do, Republic of Korea), Siemens (Clinitest, Healgen Scientific Ltd., TX, USA)] and evaluated their performance using samples assigned for routine RT-qPCR screening of patients at the local university hospital between May and November 2021.

Fig. 1

Fig. 1

illustrates the predicted LFD results based on cT values by RT-qPCR. The gray lines and areas indicate the predicted probability of a positive outcome and the 95% confidence derived from the logistic regression of all samples (imputed cT-value for PCR-negative samples = 40.0). The red line indicates the predicted probabilities of PCR-positive samples only. Bars on the top and on the bottom indicate positive and negative LFD test results, respectively. Red bars indicate false results, while correct results are indicated in blue.

Table 1

Table 1

shows the cT thresholds at which 50% and 95% of the LFD test results become positive; in case of a50 and a95 all samples (imputed cT-value for PCR-negative samples = 40), in case of p50 and p95 only positive samples were considered. Brackets indicate the limit of the lower and upper 95% confidence interval.

LFA	TP	TN	FP	FN	a50	a95	p50	p95
Abott	76	38	15	43	32.1 [30.0; 34.3]	14.4 [14.0; NA]	30.2 [28.3; 32.5]	18.4 [14.7; NA]
Acon	39	55	0	80	23.5 [22.6; 24.4]	19.9 [18.4; 21.7]	23.5 [22.6; 24.4]	19.9 [18.4; 21.7]
Clungene	57	54	0	58	26.4 [25.4; 27.5]	21.0 [19.3; 23.1]	26.4 [25.4; 27.5]	20.9 [19.1; 23.0]
Joysbio	35	54	1	84	22.8 [21.6; 23.8]	16.2 [14.1; NA]	22.8 [21.8; 23.8]	18.0 [15.4; 20.4]
Lepu	53	48	7	66	25.7 [23.2; 27.8]	NA	25.4 [23.4; 27.6]	NA
Nadal	94	24	31	25	40.0 [NA; NA]	NA	40.0 [34.5; NA]	NA
Orient	69	54	1	50	28.3 [26.6; 30.0]	16.3 [14.1; NA]	29.0 [26.9; 31.7]	14.0 [13.9; NA]
Realy.Tech	59	49	6	60	27.4 [25.7; 29.0]	14.3 [13.9; NA]	26.7 [25.4; 28.3]	17.6 [14.5; NA]
Roche	62	49	6	57	27.9 [26.3; 29.6]	15.6 [14.0; NA]	27.2 [25.8; 28.7]	18.6 [15.4; 21.7]
Siemens	53	55	0	66	25.7 [24.5; 26.9]	19.0 [16.6; 21.6]	25.7 [24.5; 26.9]	18.9 [16.2; 21.5]

All clinical specimens were anonymized nasopharyngeal swabs in universal transport medium (119 positive and 55 negative) performed by trained personnel. All tests were performed in parallel using aliquots of the same samples. Samples were stored for a maximum of 48 h at 4 °C. The distribution of the cT-values of our samples was as follows: cT < 20, n = 16; 20–25, n = 31; 25–30, n = 31; 30–35; n = 29; >35; n = 12. Highest sensitivity was achieved by Nadal (79%), which however yielded the highest number of false positive results (31) and hence the lowest specificity (44%). Comparatively high sensitivity was achieved by Abbott (62%) and Orient (56%). Lowest sensitivity was achieved by Joysbio (33%) and Acon (33%). In the subset of samples with cT-values of < 25, sensitivity ranged from 68% to 91% (Nadal 94%). Siemens, Clungene and Acon showed no false positive results.

Binomial logistic regression analysis was applied to model the probability of obtaining a positive LFD result depending on the cT-value assuming 40 for PCR-negative tests or neglecting negative test results. We estimated the confidence intervals (CI) via nonparametric bootstrapping with 100.000 iterations.

In accordance with Lamb et al. and similar other studies, our study confirms that LFDs are suitable to identify potential super-spreaders with low cT values and to prevent large cluster formation, e.g., in hospitals , , , however, regardless of the manufacturer, they are not likely to contribute significantly to control the infection dynamics caused by carriers of low viral load or by asymptomatic individuals, including those at early stages of infection.

It should be noted that cT values can vary among clinical laboratories, depending on the method for RNA extraction and the PCR kits used. As an example, in a study involving 123 participating certified laboratories in Austria including our lab, a variation in cT values of ± 4.7 cycles was observed.

Samples in our study were exclusively obtained via nasopharyngeal swabs by professionals, whereas other techniques, such as the common anterior nasal swab or incorrect swabbing by non-trained personal may significantly decrease in vitro virus concentration, and thus can increase the cT-values by several orders of magnitude6, 7, 8 leading to a further loss of sensitivity. In the light of the repeated emergence of SARS-CoV-2 variants, one may also be worried about decreased sensitivity due to altered surface proteins, which are recognized be the LFD antibodies.

Based on our data, we conclude that LFDs cannot be recommended for general broad-range screening for SARS-CoV-2 infection in an asymptomatic population. At high infection dynamics however, insufficient PCR test capacities lead to logistic difficulties, an increased time-to-result, as well as decreased sensitivity related to sample pooling. In such settings, the short time-to-result of LFDs may support the timely identification and isolation of individuals with high viral load to dampen the dynamics of spread, as suggested by Lamb et al..

Declaration of Competing Interest

The authors have no conflict of interest to declare.