Identification of Natural SARS-CoV-2 Infection in Seroprevalence Studies Among Vaccinated Populations.

Most SARS-CoV-2 antibody assays cannot distinguish between antibodies that developed after natural infection and those that developed after vaccination. We assessed the accuracy of a nucleocapsid-containing assay in identifying natural infection among vaccinated individuals. A longitudinal cohort composed of health care workers in the Minneapolis/St. Paul area was enrolled. Two rounds of seroprevalence studies separated by 1 month were conducted from November 2020 to January 2021 among 81 participants. Capillary blood from rounds 1 and 2 was tested for IgG antibodies against spike proteins by enzyme-linked immunosorbent assay (spike-only assay). During round 2, IgGs reactive to SARS-CoV-2 nucleocapsid protein (nucleocapsid-containing assay) were assessed. Vaccination status at round 2 was determined by self-report. Area under the curve was computed to determine the discriminatory ability of the nucleocapsid-containing assay for identification of recent infection. Participants had a mean age of 40 years (range, 23 to 66 years); 83% were female. Round 1 seroprevalence was 9.5%. Before round 2 testing, 46% reported vaccination. Among those not recently infected, in comparing vaccinated vs unvaccinated individuals, elevated levels of spike 1 (P<.001) and spike 2 (P=.01) were observed, whereas nucleocapsid levels were not statistically significantly different (P=.90). Among all participants, nucleocapsid response predicted recent infection with an area under the curve of 0.93 (95% CI, 0.88 to 0.99). Among individuals vaccinated more than 10 days before antibody testing, the specificity of the nucleocapsid-containing assay was 92%, whereas the specificity of the spike-only assay was 0%. An IgG assay identifying reactivity to nucleocapsid protein is an accurate predictor of natural infection among a partially vaccinated population, whereas a spike-only assay performed poorly.

Identification of SARS-CoV-2 infection by antibody assays is important for monitoring natural infection rates. Currently authorized antibody tests measure antibody reactivity to spike proteins, yet these develop in response to both infection and vaccination. , Consequently, ongoing antibody studies will be unable to accurately differentiate prior SARS-CoV-2 infection from vaccination against SARS-CoV-2 in populations with high vaccination coverage.

Current vaccines are not expected to elicit a nucleocapsid response. Consequently, antibody tests that target nucleocapsid proteins can potentially identify prior infection among vaccinated individuals. The purpose of this study was to compare the accuracy of a nucleocapsid-containing assay vs a spike protein–only assay in the identification of prior SARS-CoV-2 infection among a sample of health care workers (HCWs) in the United States.

Methods

A sample of HCWs located in the Minneapolis/St. Paul, Minnesota, metropolitan area was enrolled, and 2 rounds of seroprevalence studies were conducted from November 2020 to January 2021. A subsample of participants (N=81) for whom excess blood was available for testing with both a spike-only assay and a nucleocapsid-containing assay are presently included. The study was approved by the University of Minnesota Institutional Review Board. All participants provided informed consent.

Biospecimen Collection and Antibody Assay

Participants were invited to participate in 2 rounds of seroprevalence studies (Figure 1 ), and all participants participated in both rounds. Samples were home collected using Neoteryx Mitra 10 μL samplers by volumetric absorption of capillary blood from a finger stick. The first capillary blood specimens were collected between November 22, 2020, and December 18, 2020, and the second specimens were collected from December 26, 2020, to January 8, 2021; median time between round 1 and round 2 specimen collections was 28.5 days (range, 12 to 44 days). All round 1 specimens were collected from participants before vaccinations had become available.

Figure 1

Overview of study design against the backdrop of the Minnesota (MN) active infection curve. PCR, polymerase chain reaction.

To distinguish antibodies developed after vaccination from those of natural infection, 2 assays were used. The spike-only assay comprises Quansys Q-Plex SARS-CoV-2 Human IgG (4-Plex), a qualitative chemiluminescent enzyme-linked immunosorbent assay that measures human IgGs reactive to SARS-CoV-2 spike subunit S1 (S1) and spike subunit S2 (S2). A spike-only positive call requires both S1 and S2 antibody levels to exceed assay cutoffs. The second assay measures human IgGs reactive to SARS-CoV-2 nucleocapsid (hereafter, nucleocapsid-containing assay), absent from vaccines, to determine natural infection. The assays were developed by Quansys Biosciences.

Round 1 samples were tested with the spike-only assay, and round 2 samples were tested with both the spike-only assay and the nucleocapsid-containing assay. Assay details are presented in the Supplemental Materials, available online at http://www.mayoclinicproceedings.org.

Statistical Analyses

Recent infection with SARS-CoV-2 was defined as either a positive result from the round 1 spike-only assay (all round 1 assays were performed before vaccination) or a self-report of recent infection. Self-reported vaccination (vaccinations in the study contain only the spike protein) status (yes/no, dose and date) was assessed from questionnaires. Participants were further classified as having been vaccinated 0 to 7 days, 7 to 10 days, or more than 10 days before collection of capillary blood during round 2.

Kruskal-Wallis tests compared median IgG response to nucleocapsid protein from the nucleocapsid-containing assay at round 2 by vaccination and infection status. Receiver operating characteristic curves were constructed, and area under the curve (AUC) was computed to determine the discriminatory ability of round 2 IgG reactivity to nucleocapsid for identification of infection. The optimal nucleocapsid cut point was defined by the Youden value. The accuracy of identifying infection from the Youden optimal nucleocapsid level was then compared with the accuracy of the spike-only assay using round 2 samples.

The accuracy was also estimated for the nucleocapsid-containing assay using the cut point defined by the Youden value (described before) in comparison to concurrent round 2 results from the spike-only assay among unvaccinated individuals. In this analysis, infection was defined by results from the concurrent round 2 spike-only assay. These results were then compared against results from the round 2 nucleocapsid-containing assay.

Results

The mean age of participants was 40 years (range, 23 to 66 years); 83% were female, 95% White, 2.5% Hispanic, and 2.5% Asian. Comorbidities included asthma (14%), coronary artery disease (1%), and type 2 diabetes (5%), and 2.5% reported being immunocompromised. No participants were vaccinated before round 1 testing, whereas 46% of participants reported vaccination before the round 2 testing. Among those vaccinated before round 2, the mean time between vaccination and antibody testing was 9 days (range, 1 to 34 days). Round 1 seroprevalence (by the spike-only assay) was 9.5%.

Median values of S1, S2, and nucleocapsid were statistically significantly elevated among infected vs uninfected participants (all P values <.0001; Figure 2 ). Among individuals who were not infected, elevated levels of S1 (P=.0005) and S2 (P=.01) were observed among vaccinated compared with unvaccinated participants (Figure 2A, 2B), whereas nucleocapsid levels were not statistically significantly different by vaccination status (P=.90; Figure 2C).

Figure 2

IgG antibody titers against spike 1 protein (A), spike 2 protein (B), and nucleocapsid protein (C). Respective P values for any difference in spike 1, spike 2, or nucleocapsid response by vaccination status among participants without recent infection are .0005 (A), .01 (B), and .90 (C). In D, the area under the curve (AUC) for the continuous nucleocapsid variable is shown. The AUC for the continuous nucleocapsid levels is 0.93 (95% CI, 0.88 to 0.99). The optimal nucleocapsid cut point (Youden value) is 0.42 and yields an AUC of 0.89 (95% CI, 0.82 to 0.96).

Among round 2 samples, nucleocapsid response predicted infection with an AUC of 0.93 (95% CI, 0.88 to 0.99; Figure 2D). Respective AUC values for S1 and S2 were 0.81 (95% CI, 0.72 to 0.90) and 0.89 (95% CI, 0.81 to 0.96). The optimal nucleocapsid cut point based on the Youden value was 0.42 and yielded an AUC of 0.89 (95% CI, 0.82 to 0.96) with a sensitivity of 90% and a specificity of 88%. Among the 37 with vaccination before round 2 biospecimen collection, the optimal nucleocapsid cutoff remained at 0.42 with an AUC of 0.95 (95% CI, 0.88 to 1.0), a sensitivity of 100%, and a specificity of 96%.

Among unvaccinated individuals, sensitivities for the spike-only and nucleocapsid-containing assays were 95% and 85%, respectively (Table ). Among vaccinated individuals, both assays had 100% sensitivity. Among unvaccinated individuals, the specificity of the spike-only assay was 86%, whereas among individuals vaccinated more than 10 days before round 2, testing specificity decreased to 0%. In contrast, the specificity of the nucleocapsid assay increases from 86% to 90% among unvaccinated vs vaccinated individuals. Regarding participants with apparent false-positive findings for the round 2 nucleocapsid-containing assay (ie, positive result for the nucleocapsid-containing assay during round 2 and a negative result from the spike-only assay at round 1), the median time between round 1 and round 2 testing was 33.5 days, raising the potential that these individuals were truly infected in the interim.

Table

Comparison of SARS-CoV-2 IgG Antibody Test Performance and Predictive Values Between a Spike Protein–Only Assay and a Nucleocapsid-Containing Assay According to Prior Vaccination Statusa

	Spike-only assayb		Nucleocapsid-containing assayc		Test performance difference
	Infectedd	Not infectedd	Infectedd	Not infectedd
Among all participants (N=81)
Assay +	29 (97%)	24 (47%)	27 (90%)	6 (12%)
Assay −	1 (3%)	27 (53%)	3 (10%)	45 (88%)
Sensitivity	97%		90%		−7%
Specificity	53%		88%		33%
PPV	55%		82%		27%
NPV	96%		94%		−2%
No vaccination before round 2 testing (n=41)
Assay +	19 (95%)	3 (14%)	17 (85%)	3 (14%)
Assay −	1 (0%)	18 (86%)	3 (11%)	18 (86%)
Sensitivity	95%		85%		−10%
Specificity	86%		86%		0%
PPV	86%		85%		−1%
NPV	95%		86%		−9%
Vaccinated within 7 days before round 2 testing (n=15)
Assay +	6 (100%)	2 (22%)	6 (100%)	2 (22%)
Assay −	0 (0%)	7 (78%)	0 (0%)	7 (78%)
Sensitivity	100%		100%		0%
Specificity	78%		78%		0%
PPV	75%		75%		0%
NPV	100%		100%		0%
Vaccinated 8 to 10 days before round 2 testing (n=7)
Assay +	1 (100%)	4 (67%)	1 (100%)	0 (0%)
Assay −	0 (0%)	2 (33%)	0 (0%)	6 (100%)
Sensitivity	100%		100%		0%
Specificity	33%		100%		67%
PPV	25%		100%		75%
NPV	100%		100%		0%
Vaccinated >10 days before round 2 testing (n=15)
Assay +	1 (100%)	14 (100%)	1 (100%)	1 (7%)
Assay −	0 (0%)	0 (0%)	0 (0%)	13 (92%)
Sensitivity	100%		100%		0%
Specificity	0%		92%		92%
PPV	7%		50%		43%
NPV	—		100%		—

Three individuals did not have vaccination date information and are not included in results presented by time since vaccination.

NPV, negative predictive value; PPV, positive predictive value.

A positive test result was defined as a reactive call Quansys Q-Plex SARS-CoV-2 Human IgG (4-Plex) at round 2.

A positive test result was defined as a round 2 nucleocapsid response ≥0.42 per Methods and Figure 2.

“Infected” and “Not infected” refer to SARS-CoV-2 “true” recent infection status based on antibody testing results and self-report in all participants during round 1 of testing (before vaccination). Testing was conducted with Quansys Q-Plex SARS-CoV-2 Human IgG (4-Plex). Reactivity by this assay or self-report recent infection is the definition of true recent infection status in this table.

The Supplemental Table (available online at http://www.mayoclinicproceedings.org) presents assay results among unvaccinated individuals for the nucleocapsid-containing assay compared with the “gold standard” spike-only assay test results from the same blood collection time at round 2; this minimizes the potential for false-positive findings. Sensitivity and specificity were 90% and 100%, and as expected, the specificity improves from 86% to 100% because of fewer false-positive findings.

Discussion

We evaluated the ability of an IgG antibody assay that assesses reactivity to the nucleocapsid protein to identify previous SARS-CoV-2 infection among a population of HCWs, with and without infection and before and after vaccination. This is the first investigation of this nature to our knowledge. We found that in the context of vaccination, nucleocapsid response is highly predictive of infection. Among vaccinated individuals, the nucleocapsid assay had a substantially higher specificity and positive predictive value compared with the spike-only assay.

Results in the Table are limited because infection was determined with use of an antibody test conducted approximately 1 month before round 2 testing, and it is possible that participants appearing as a false-positive in the Table were truly infected between round 1 and round 2 and developed an antibody response. Therefore, our findings underestimate specificity for the nucleocapsid-containing assay. Our results are also not generalizable to the general population.

There is likely to be high value in using nucleocapsid assays for public health surveillance efforts as SARS-CoV-2 infection will probably become endemic, and ongoing surveillance studies will be necessary. This is particularly relevant against the backdrop of vaccination, where assays targeting the spike protein cannot distinguish natural infection from vaccination. From a clinical perspective, there is potential utility in antibody assays for detecting undiagnosed SARS-CoV-2 infection and assessing post-COVID conditions such as multi-inflammatory syndrome in children.

Conclusion

An IgG assay identifying reactivity to nucleocapsid protein was an accurate predictor of recent infection among a population of vaccinated HCWs. These findings suggest that in the era of SARS-CoV-2 vaccination, seroprevalence studies monitoring natural infection will require assays that capture reactivity to the nucleocapsid protein.