Accuracy of a nucleocapsid protein antigen rapid test in the diagnosis of SARS-CoV-2 infection.

OBJECTIVES: Rapid, reliable and easy-to-implement diagnostics that can be adapted in early severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) diagnosis are critical to combat the epidemic. SARS-CoV-2 nucleocapsid protein (NP) is an ideal target for viral antigen-based detection. A rapid and convenient method was developed based on fluorescence immunochromatographic (FIC) assay to detect the SARS-CoV-2 NP antigen. However, the accuracy of this diagnostic method needs to be examined.
METHODS: This prospective study was carried out between 10 and 15 February 2020 in seven hospitals in Wuhan and one hospital in Chongqing, China. Participants with clinically suspected SARS-CoV-2 infection were enrolled. NP antigen testing by FIC assay and nucleic acid (NA) testing by real-time reverse transcriptase PCR (RT-PCR) were performed simultaneously in a blinded manner with the same nasopharyngeal swab sample. The diagnostic accuracy of NP antigen testing was calculated by taking NA testing of RT-PCR as the reference standard, in which samples with a cycle threshold (Ct) value of ≤40 were interpreted as positive for SARS-CoV-2.
RESULTS: A total of 253 participants were enrolled; two participants were excluded from the analyses because of invalid NP testing results. Of 251 participants (99.2%) included in the diagnostic accuracy analysis, 201 (80.1%) had a Ct value of ≤40. With Ct value 40 as the cutoff of NA testing, the sensitivity, specificity and percentage agreement of the FIC assay was 75.6% (95% confidence interval, 69.0-81.3), 100% (95% confidence interval, 91.1-100) and 80.5% (95% confidence interval, 75.1-84.9) respectively.
CONCLUSIONS: With RT-PCR assay as the reference standard, NP antigen testing by FIC assay shows high specificity and relatively high sensitivity in SARS-CoV-2 diagnosis in the early phase of infection.

Introduction

Acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has spread to more than 200 countries, causing thousands of deaths due to coronavirus disease 2019 (COVID-19). A timely and accurate diagnosis of the SARS-CoV-2 infection is the prerequisite for quarantine and treatment. The explosive increasing number of SARS-CoV-2 infections currently warrants novel strategies to substantially improve diagnostic capacity [1].

Nucleic acid (NA) testing by real-time reverse transcriptase PCR (RT-PCR) is widely used for the pathogenic diagnosis of COVID-19. However, NA testing needs a well-equipped molecular diagnostic laboratory with trained staff and expensive equipment. Importantly, it has a long turnaround time that inevitably limits the scaling up of the testing capability [2]. Antibody-based testing has the potential to be used to increase diagnostic capacity. However, antibody responses to a pathogen do not appear in the early stage of infection. In SARS-CoV-2 infection, recent data show that the median seroconversion time of IgM and IgG was approximately 18 and 20 days after exposure respectively [[3], [4], [5], [6]]. Therefore, antibody-based serologic assays cannot achieve diagnosis early in the onset of infection.

Viral antigen detection may be a candidate strategy to achieve early diagnosis of SARS-CoV-2 infection. Nucleocapsid protein (NP) is one of the predominantly expressed structural proteins and has been confirmed as an ideal target for early diagnostic detection in SARS-CoV infection [7], which broke out in 2003 and has high genetic similarity to SARS-CoV-2 [8]. Briefly, among several viral proteins, the NP antigen is one of the best early diagnostic markers in SARS-CoV and can be detected up to 1 day before the appearance of clinical symptoms [9].

We developed a fluorescence immunochromatographic (FIC) assay to detect the NP antigen of SARS-CoV-2 specifically for a rapid laboratory test (Supplementary Fig. S1). However, data on the diagnostic test accuracy of SARS-CoV-2 NP antigen have not yet been reported. Here we report the sensitivity and specificity of a SARS-CoV-2 NP antigen–based FIC assay. This assay can detect SARS-CoV-2 infection in 10 minutes without costly equipment.

Methods

Study design

This study was designed as prospective research to examine the diagnostic test accuracy of the FIC assay to detect SARS-CoV-2 NP antigen on a population with symptomatic suspected COVID-19. This study was approved by the ethics committees of participating hospitals. The participants with clinically suspected SARS-CoV-2 infection were enrolled onto the study from eight centres in China between 10 and 15 February 2020. These centres included seven hospitals in Wuhan and one centre in Chongqing, China (Supplementary Methods).

Participants

Participants in this study were hospitalized patients or outpatients with suspected COVID-19 symptoms who were consecutively recruited, regardless of age or sex. A suspected COVID-19 case was defined according to the Guidelines of Diagnosis and Treatment of COVID-19 (5th version), released by the National Health Commission of China. Briefly, individuals who had related epidemiologic history and more than one of symptoms including fever, cough, shortness of breath, myalgia and fatigue were defined as suspected COVID-19 patients. Those who were unable to or did not provide verbal consent were excluded from this study. The cases confirmed on the basis of the Guidelines for Diagnosis and Treatment of Novel Coronavirus Pneumonia (5th version) to have COVID-19 were retrospectively collected (Supplementary Methods).

Sample collection

For each suspected participant enrolled, nasopharyngeal swabs in extraction buffer were collected by the medical staff from each COVID-19 designated hospital, with a part of the swab extraction buffer used for NA testing by RT-PCR and its counterpart used for NP antigen testing by FIC assay. The NA testing and NP antigen testing were performed simultaneously by trained study members in two central biosafety level 3 (P3 biosafe) laboratories in Wuhan and Chongqing respectively. The laboratory testing results were blinded to each other during the tests.

NA testing by RT-PCR assay

The RT-PCR assays were used to detect viral RNA by targeting the SARS-CoV-2 ORF1 ab and N gene region of SARS-CoV-2 [10]. Briefly, nasopharyngeal swab samples were immersed in 500 μL extraction buffer, and NAs were extracted using the High Pure Viral Nucleic Acid Kit. RT-PCR was performed using the TaqMan One-Step RT-PCR Kit (Da An Gene, Guangzhou, China), which is approved by the China Food and Drug Administration for the ABI Prism 7500 or Light Cycler 480 real-time PCR systems. According to the kit's instructions, samples with C  ≤ 40 were interpreted as being SARS-CoV-2 positive.

NP antigen testing by FIC assay

We developed an FIC assay to specifically detect the NP antigen of SARS-CoV-2 (Supplementary Fig. S1). The NP antigen detection assay uses the principle of double-antibody sandwich FIC analysis technology [11] to qualitatively detect SARS-CoV-2 NP antigen in human nasopharyngeal and throat swabs samples. Briefly, a nitrocellulose membrane with fluorescent microparticle–labeled SARS-CoV-2 NP-specific antibody (mouse anti–SARS-CoV-2 NP M1) was used to detect the NP antigen. This antibody was produced by vaccination of mice with long peptides containing SARS-CoV-2 NP-specific epitopes. If the sample contains NP antigen, a double-antibody sandwich is formed and a fluorescent signal detected. However, no fluorescence reaction line is formed when SARS-CoV-2–negative samples are detected. Regardless of whether or not the sample contains NP antigen, the control area forms a fluorescence reaction line as the assay control (Supplementary Fig. S1A). The fluorescent results were read by an immunofluorescence analyser. A total of 100 μL nasopharyngeal swab sample extraction buffer was added to the sample well of the test card. After a 10-minute reaction, the card was inserted into an immunofluorescence analyser to automatically determine a positive or negative result by comparing the detection value to the reference cutoff value that was set as the internal parameter of the kit's ID chip (Supplementary Fig. S1). Samples with detected fluorescent signals higher than the cutoff value and positive internal control were interpreted as SARS-CoV-2 positive by the analyser; if the internal control of a sample failed, the result was interpreted as ‘invalid’ by the analyser. The cutoff value was determined by testing 100 nasopharyngeal swab samples of healthy individuals and calculated as the mean value of the fluorescence signal with standard deviation [12].

Statistical analysis

Two different members of the study team (B.D. and J.Z.) did data entry; the data entered were then reconciled. NA testing by RT-PCR served as the reference standard, with sample C value ≤ 40 interpreted as SARS-CoV-2 positive. Sensitivity was defined as the proportion of NA testing samples that tested positive by the NP antigen testing assay; specificity was defined for each specimen type as the proportion of NA testing–negative samples that tested negative by the NP antigen testing assay. Samples that generated an invalid rapid diagnostic test result were excluded from sensitivity and specificity calculations. A 95% confidence interval was provided by the Wilson score method. Statistical analyses were carried out by R software [13]. This study has been reported according to the STARD (Standards for Reporting Diagnostic Accuracy Studies) guidelines (Supplementary Table S1).

Results

A total of 253 participants were enrolled; two participants were excluded from the analyses because of invalid NP testing results. Thus, a total of 251 participants (99.2%) were included in the diagnostic accuracy. The cohort comprised 122 male (48.6%) and 129 female (51.4%) subjects aged 16 to 75 years (mean, 40.2 years). Detailed symptoms of participants were not recorded. A schematic of this study is provided in Supplementary Fig. S2. A total of 201 participants (80.1%) had C  ≤ 40; of these, 155 (81.7%) had C  ≤ 37 and 46 (18.3%) had 37 < C  ≤ 40 (Table 1 ). Finally, a total of 194 participants (96.5%) who had C  ≤ 40 were definitely diagnosed as COVID-19 by epidemiology and by clinical examinations, according to the Guidelines for Diagnosis and Treatment of COVID-19 (5th version), released by the National Health Commission of China (Supplementary Fig. S2).

Table 1

Nucleic acid testing results of 251 patients

Cutoff Ct value for RT-PCR	n (%)
Ct > 40	50 (19.9)
Ct ≤ 40	201 (80.1)
37 < Ct ≤ 40	46 (18.3)
Ct ≤ 37	155 (81.7)

Cutoff is according to recommendation for COVID-19 diagnosis of Chinese Center for Disease Control. Samples with C values > 40 were considered to be SARS-CoV-2 negative; C ≤ 37, SARS-CoV-2 positive; and 37 < C ≤ 40, indeterminate, requiring second sampling and repeat RT-PCR assay.

COVID-19, coronavirus disease 2019; C, cycle threshold; RT-PCR, real-time reverse transcriptase PCR; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

With a C value of 40 as the cutoff of NA testing, the sensitivity, specificity and accuracy of NP antigen detection were 75.6% (95% confidence interval, 69.0–81.3), 100% (95% confidence interval, 91.1–100) and 80.5% (95% confidence interval, 75.1–84.9) respectively (Table 2 ). Notably, all the 152 NP antigen–positive results corroborated with NA tests and were finally diagnosed as COVID-19 (Supplementary Fig. S2).

Table 2

Performance of NP antigen–based FIC assay with RT-PCR results as reference standard

Characteristic	Ct ≤ 40 as NA testing positivea	Ct ≤ 37 as NA testing positiveb
Prevalence (%)	80.1 (201/251)	61.7 (155/251)
95% CI	74.5–84.7	55.4–67.7
Sensitivity (%)	75.6 (152/201)	91.0 (141/155)
95% CI	69.0–81.3	85.0–94.8
Specificity (%)	100 (50/50)	88.5 (85/96)
95% CI	91.1–100	80.0–93.9
Positive predictive value (%)	100 (152/152)	92.8 (141/152)
95% CI	96.9–100	87.1–96.2
Negative predictive value (%)	50.5 (50/99)	85.9 (85/99)
95% CI	40.3–60.6	77.1–91.8
Agreement (%)	80.5 (202/251)	90.0 (226/251)
95% CI	75.1–84.9	85.7–93.2

CI, confidence interval; COVID-19, coronavirus disease 2019; C, cycle threshold; FIC, fluorescence immunochromatographic; NA, nucleic acid; NP, nucleocapsid protein; RT-PCR, real-time reverse transcriptase PCR; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

According to PCR kit instructions, samples with C value ≤ 40 were interpreted as SARS-CoV-2 positive; C > 40, SARS-CoV-2 negative.

According to the recommendations for COVID-19 diagnosis of the Chinese Center for Disease Control, samples with C values of ≤37 were interpreted as SARS-CoV-2 positive; samples with 37 < C ≤ 40 were considered indeterminate, requiring second sampling and repeat RT-PCR assay.

Discussion

The COVID-19 pandemic is a global public health concern; however, diagnosis is difficult because of its nonspecific clinical manifestations and symptoms. Rapid, reliable and easy-to-implement diagnostic tools that can be applied to SARS-CoV-2 are an urgent requirement. Here we show that compared to RT-PCR, NP antigen–based assay has high specificity and relatively high sensitivity in the diagnosis of SARS-CoV-2 in the early phase of infection. In addition, despite a lack of complete clinical information, we noticed that a proportion of NP antigen detection assay–positive samples were of these patients who had fever, fatigue or cough onset within 1 day.

Various techniques may be used to detect virus antigens. On the one hand, compared to the colloidal gold immunochromatography assay, which may be used at the point of care, the experimental operation of the FIC assay is slower and less convenient; however, it has higher diagnostic sensitivity [14,15]. On the other hand, compared to NA testing by RT-PCR or next-generation sequencing, this method is a fast and reliable way to identify SARS-CoV-2–infected individuals. In practice, NP antigen–based FIC assay may not be used to replace NA testing, but its combination with RT-PCR will substantially improve diagnostic capabilities.

The present study has some limitations. Firstly, the sample size of this study was small. Secondly, the detailed medical information of some participants in this study was not available, so data could not be correlated with symptoms or disease course. Thirdly, participants in this study were suspected COVID-19 patients with symptoms; asymptomatic individuals were not included. Fourthly, the overall prevalence of COVID-19 in this population was high (77.2%, 194/251), with the background prevalence during the study still not reported. Thus, the applicability of the test in other settings, such as screening, should be evaluated carefully. Also, further investigation with different sample types is warranted.

Transparency declaration

All authors report no conflicts of interest relevant to this article.