Limitations of Molecular and Antigen Test Performance for SARS-CoV-2 in Symptomatic and Asymptomatic COVID-19 Contacts.

COVID-19 has brought unprecedented attention to the crucial role of diagnostics in pandemic control. We compared severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) test performance by sample type and modality in close contacts of SARS-CoV-2 cases. Close contacts of SARS-CoV-2-positive individuals were enrolled after informed consent. Clinician-collected nasopharyngeal (NP) swabs in viral transport media (VTM) were tested with a routine clinical reference nucleic acid test (NAT) and PerkinElmer real-time reverse transcription-PCR (RT-PCR) assay; positive samples were tested for infectivity using a VeroE6TMPRSS2 cell culture model. Self-collected passive drool was also tested using the PerkinElmer RT-PCR assay. For the first 4 months of study, midturbinate swabs were tested using the BD Veritor rapid antigen test. Between 17 November 2020 and 1 October 2021, 235 close contacts of SARS-CoV-2 cases were recruited, including 95 with symptoms (82% symptomatic for ≤5 days) and 140 asymptomatic individuals. Reference NATs were positive for 53 (22.6%) participants; 24/50 (48%) were culture positive. PerkinElmer testing of NP and saliva samples identified an additional 28 (11.9%) SARS-CoV-2 cases who tested negative by reference NAT. Antigen tests performed for 99 close contacts showed 83% positive percent agreement (PPA) with reference NAT among early symptomatic persons, but 18% PPA in others; antigen tests in 8 of 11 (72.7%) culture-positive participants were positive. Contacts of SARS-CoV-2 cases may be falsely negative early after contact, but more sensitive platforms may identify these cases. Repeat or serial SARS-CoV-2 testing with both antigen and molecular assays may be warranted for individuals with high pretest probability for infection.

INTRODUCTION

As global COVID-19 cases exceed 250 million by December 2021 (1), severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) diagnosis remains a critically important public health priority for individuals to know their status and, from a public health standpoint, to understand the amount of circulating virus and risk of infection. Sustained demand for SARS-CoV-2 testing has accelerated the development and deployment of multiple competing testing technologies, sample types, and approaches (2, 3). Rapid diagnostic tests that use a lateral flow assay (LFA) to detect SARS-CoV-2 antigen offer scale, convenience, and potential for home use but suffer from reduced sensitivity compared to nucleic acid tests (NAT) such as real-time reverse transcription-PCR (RT-PCR) (4).

In early symptomatic disease—within 5 days of symptom onset—infected persons are likely to have high viral loads such that both antigen and molecular tests are likely to perform well. For individuals who are close contacts (5), the viral burden of the index case has been associated with risk of transmission (6). However, in the presymptomatic phase when viral burdens are still low (7), false-negative antigen tests can occur. A recent study from the Netherlands in close contacts showed that only 64% of SARS-CoV-2 molecular-test-positive close contacts were identified by antigen testing; antigen test sensitivity was lower in asymptomatic individuals (58.7%) than in symptomatic individuals (84.2%) (8).

We sought to evaluate the performances of molecular and antigen testing in symptomatic and asymptomatic close contacts of SARS-CoV-2-confirmed individuals. We also sought to assess the sensitivity of both molecular and antigen tests in nasal swabs and molecular tests in saliva to diagnose close contacts with infectious virus in nasopharyngeal (NP) swab viral transport media (VTM) using the VeroE6TMPRS2 cell model.

MATERIALS AND METHODS

Patient population and study design.

Participants were recruited through posters, online advertisement, and social media written in English and Spanish. Direct outreach targeted close contacts of persons who tested positive for SARS-CoV-2. Under a partial waiver of HIPAA authorization, study staff called patients who tested positive for SARS-CoV-2 at any Johns Hopkins Medical System site to provide study information to their close contacts, defined as individuals who spent >15 min at <6 feet in the 5 days after symptom onset or test positivity (9). Interested close contacts were able to participate by contacting a dedicated study phone number. Study staff obtained informed consent from willing and eligible participants using a script administered either over the phone or in person. Participants were scheduled for same- or next-day testing at one of two outdoor outpatient testing sites. Two weeks after specimen collection, the study staff called the participant to administer a brief questionnaire, which assessed ongoing or new COVID-19 symptoms, hospitalization, and any other SARS-CoV-2 test results that the participant may have obtained after enrollment. The study was approved by the Johns Hopkins School of Medicine Institutional Review Board.

Sample collection.

NP swabs were collected by a trained nurse on all participants and immediately inoculated into VTM. Study staff additionally observed self-collection of a midturbinate swab for antigen testing and 2 mL of passive drool saliva in a sterile urine cup. All specimens were transported on ice to the clinical laboratory using a medical courier service. The transit time from sample collection sites to the laboratory was 15 to 20 min, and samples were processed immediately upon receipt.

Reference nucleic acid testing.

All molecular testing took place in the Johns Hopkins Molecular Virology Laboratory, which is Clinical Laboratory Improvement Amendments of 1988 (CLIA) certified. In order to provide results to participants without delay, NATs were performed as part of the routine clinical workflow on a range of available platforms. The NAT performed and resulted to patients in real-time was considered to be the reference NAT. For each of the following assays, testing was performed per manufacturer Food and Drug Administration emergency use authorization (EUA) instructions: NeuMoDx SARS-CoV-2 assay (NeuMoDx, Ann Arbor, MI) (10), Roche cobas 6800 (Roche Molecular Systems, Pleasanton, CA), RealStar SARS-CoV-2 RT-PCR (Altona Diagnostics, Hamburg, Germany), Hologic Aptima SARS-CoV-2 assay (Hologic, Bedford, MA) (11), PerkinElmer new coronavirus nucleic acid detection kit (PerkinElmer, Inc. Austin, TX) (12), and BD Veritor SARS-CoV-2 (Becton, Dickinson, Sparks, MD); results were interpreted using the Veritor Plus analyzer (13). The reference clinical assay performed on fresh NP swab VTM was the NeuMoDx, Roche, Aptima, Cepheid Xpert, or Altona assay. Remnant specimen was aliquoted and stored frozen at −80°C and then 300 μL thawed for extraction followed by RT-PCR on the PerkinElmer instrument. Similarly, fresh saliva was aliquoted, frozen, and then, in batches, 300 μL of saliva was processed using the same methods. BD Veritor testing occurred within 6 h of collection, with all midturbinate swabs stored on ice. At the time of test performance and interpretation, staff were blinded to results of other testing modalities and sample types. Antigen testing was performed in a separate laboratory space independent of molecular test facilities.

Virus culture from nasal swabs.

VeroE6TMPRSS2 cells were grown in complete medium (CM) consisting of Dulbecco modified Eagle medium (DMEM) with 10% fetal bovine serum (FBS) (Gibco), 1 mM glutamine (Invitrogen), 1 mM sodium pyruvate (Invitrogen), 100 U/mL of penicillin (Invitrogen), and 100 μg/mL of streptomycin (Invitrogen) (14). Viral infectivity was assessed on VeroE6TMPRSS2 cells as previously described using infection media (IM) (identical to CM except the FBS is reduced to 2.5%) (15). When a cytopathic effect was visible in >50% of cells in a given well, the supernatant was harvested. The presence of SARS-CoV-2 was confirmed through reverse transcription-quantitative PCR (RT-qPCR) as described previously by extracting RNA from the cell culture supernatant using the Qiagen viral RNA isolation kit and performing RT-qPCR using the N1 and N2 SARS-CoV-2-specific primers and probes in addition to primers and probes for the human RNaseP gene using synthetic RNA target sequences to establish a standard curve (16).

Statistical analysis.

Clinical characteristics of participants with positive clinical reference test results were compared to participants with negative results. Categorical variables were compared using the chi-square test or exact Fisher test for comparisons, including groups with <5 expected frequencies. Continuous variables were compared using the Kruskal-Wallis test. As SARS-CoV-2 molecular tests are highly specific, in cases of discordant molecular results by modality (standard clinical reference test or PerkinElmer RT-PCR) or sample type (NP or saliva), participants who tested positive by any molecular test were considered to have SARS-CoV-2. Simple frequencies were calculated for the proportion of infectious samples (as measured by cell culture) that were positive by antigen and molecular testing. Antigen test performance characteristics were reported as positive and negative percent agreement with the clinical reference test and calculated as simple proportions with two-sided 95% confidence intervals (CI).

RESULTS

Clinical characteristics among all participants by SARS-CoV-2 diagnosis.

Between 17 November 2020 and 1 October 2021, 327 participants were consented, among whom 301 presented for sample collection and provided information to evaluate their status as a close contact of a laboratory-confirmed SARS-CoV-2 case (Fig. 1; see also Fig. S1 in the supplemental material). There were 235 close contacts (upon presentation, n = 66 were deemed not to meet close contact criteria) (Fig. 1) who comprise the analyzed cohort, among whom 95 were symptomatic and 140 were asymptomatic. Of the symptomatic contacts, 82% (78/95) had symptoms for ≤5 days. Table 1 shows the baseline characteristics of the participants. Overall, the median age was 38.0 (interquartile range [IQR], 29.0 to 50.5), and 52.3% were male. There were 148 (63%) participants who reported having health care coverage. The standard clinical reference in NP samples was positive for 53 (23%) participants. Those who were SARS-CoV-2 test positive compared to those who were negative were more likely to be unvaccinated (83.0% versus 71.4%; P = 0.050), black (22.6% versus 15.9%), and of Hispanic ethnicity (54.7% versus 39.6%). Among the cohort of COVID-19 contacts, 32.1% of those who were test positive were asymptomatic, which may include contacts in the early, presymptomatic phase or asymptomatic infections. The most common symptoms among the NP swab SARS-CoV-2-positive patients were cough (41.5%), runny nose (41.5%), muscle aches (35.8%), fever or chills (32.1%), and scratchy throat (26.4%).

FIG 1

Participants enrolled and molecular test results.

TABLE 1

Clinical characteristics among close contacts stratified by SARS-CoV-2 reference test result

Clinical characteristica	Total no. (%) contacts	No. (%) reference test negative	No. (%) reference test positive	P value
	n = 235	n = 182	n = 53
Age, median (IQR)	38.0 (29.0–50.5)	37.0 (29.2–50.8)	41.0 (27.0–49.0)	0.495
Male	123 (52.3)	95 (52.2)	28 (52.8)	1.000
Race and ethnicity				0.068
Asian	3 (1.3)	3 (1.6)	0 (0.0)
Black or African American	41 (17.4)	29 (15.9)	12 (22.6)
Hispanic	101 (43.0)	72 (39.6)	29 (54.7)
Other	2 (0.9)	2 (1.1)	0 (0.0)
White	88 (37.4)	76 (41.8)	12 (22.6)
Vaccine status				0.050
Fully vaccinated	43 (18.3)b	39 (21.4)	4 (7.5)
Not vaccinated	174 (74.0)	130 (71.4)	44 (83.0)
Partially vaccinated	18 (7.7)	13 (7.1)	5 (9.4)
Tobacco use	23 (9.8)	17 (9.3)	6 (11.3)	0.870
Health care coverage	148 (63.0)	117 (64.3)	31 (58.5)	0.544
Diabetes	15 (6.4)	11 (6.0)	4 (7.5)	0.750
Kidney failure	2 (0.9)	1 (0.5)	1 (1.9)	0.401
Hypertension	37 (15.7)	26 (14.3)	11 (20.8)	0.356
Cancer, under active treatment	1 (0.4)	1 (0.5)	0 (0.0)	1.000
Asthma or COPD	22 (9.4)	17 (9.3)	5 (9.4)	1.000
Symptom onset				<0.001
Asymptomatic	140 (59.6)	123 (67.6)	17 (32.1)
Symptomatic, onset >5 days or unknown	17 (7.2)	9 (4.9)	8 (15.1)
Symptomatic, onset within 5 days	78 (33.2)	50 (27.5)	28 (52.8)
Scratchy throat	32 (13.6)	18 (9.9)	14 (26.4)	0.004
Painful sore throat	25 (10.6)	14 (7.7)	11 (20.8)	0.014
Cough worse than usual	40 (17.0)	18 (9.9)	22 (41.5)	<0.001
Runny nose	45 (19.1)	23 (12.6)	22 (41.5)	<0.001
Symptoms of fever or chills	25 (10.6)	8 (4.4)	17 (32.1)	<0.001
Temperature >38.0°C	13 (5.5)	4 (2.2)	9 (17.0)	<0.001
Muscle aches	31 (13.2)	12 (6.6)	19 (35.8)	<0.001
Nausea, vomiting, diarrhea	17 (7.2)	10 (5.5)	7 (13.2)	0.071
Shortness of breath	20 (8.5)	9 (4.9)	11 (20.8)	0.001
Unable to taste or smell	11 (4.7)	2 (1.1)	9 (17.0)	<0.001
Red or painful eyes	13 (5.5)	5 (2.7)	8 (15.1)	0.002

IQR, interquartile range; COPD, chronic obstructive pulmonary disease.

Among asymptomatic close contacts, 28 (20%) were fully vaccinated; among symptomatic close contacts, 15 (16%) were fully vaccinated.

Molecular test and viral culture results.

The NP swab reference NAT was positive for 22.6% (53/235) of participants who were close contacts of SARS-CoV-2 cases (Table 2; see Fig. S1). Among 50 of 53 reference-positive samples which underwent viral culture, 25 (50%) were culture positive. Remnant frozen NP swab VTM for 158 of the 182 SARS-CoV-2 close contacts who tested negative by the reference test were retested by PerkinElmer RT-PCR and positive for 13 (8%) samples with a median cycle threshold (C) of 36.8 (IQR, 36.0 to 38.6) (see Table S1 in the supplemental material). Saliva testing using PerkinElmer RT-PCR was performed for 186 (79%) close contacts, including 143 of 182 (79%) reference test-negative participants. Saliva was positive for 37 of 43 (86%) reference-positive participants and 17 of 143 (12%) reference-negative participants. Combined, PerkinElmer RT-PCR of NP and saliva samples identified SARS-CoV-2 in an additional 21 asymptomatic close contacts and 7 symptomatic close contacts not identified by the NP reference test. The median C value of symptomatic close contacts was lower than that of asymptomatic close contacts for both NP (22.0 with an IQR of 17.7 to 28.5 versus 34.8 with an IQR of 31.4 to 36.9) and saliva (25.8 with an IQR of 22.5 to 31.5 versus 33.4 with an IQR of 26.5 to 36.7) samples. Among the 67 participants that tested positive by any NAT and had a saliva NAT result, the sensitivity of saliva NAT was 80.6% and NP was 77.6%. Among symptomatic patients, patients with systemic symptoms of fever and muscle ache were more likely to test positive by both NP and saliva samples, and patients with runny nose were more likely to test positive by NP only.

TABLE 2

SARS-CoV-2 testing among contacts by presence of symptoms

Assay and result	Total no. (%)	No. (%) asymptomatic	No. (%) symptomatic	P value
Reference NAT, NP	235	140	95	<0.001
Positive	53 (22.6)	17 (12.1)	36 (37.9)
Negative	182 (77.4)	123 (87.9)	59 (62.1)
Reference CT value (IQR), NP (n = 51)a	24.7 (20.4–31.1)	30.9 (25.3–32.2)	21.8 (19.4–30.0)	0.002
Viral cultureb	50	16	34	0.077
Positive for COVID-19	25 (50.0)	5 (31.2)	20 (58.8)
Negative for COVID-19	23 (46.0)	11 (68.8)	12 (35.3)
Inconclusive	2 (4.0)	0 (0.0)	2 (5.9)
PerkinElmer, NPc	209	129	80	0.004
Positive	61 (29.2)	28 (21.7)	33 (41.2)
Negative	148 (70.8)	101 (78.3)	47 (58.8)
PerkinElmer CT value (IQR), NP	30.0 (19.9–34.3)	34.8 (31.4–36.9)	22.0 (17.7–28.5)	<0.001
PerkinElmer, Salivad	186	108	78	0.010
Positive	54 (29.0)	23 (21.3)	31 (39.7)
Negative	132 (71.0)	85 (78.7)	47 (60.3)
PerkinElmer CT value (IQR), Saliva	28.9 (23.0–34.8)	33.4 (26.5–36.7)	25.8 (22.5–31.5)	0.020
BD Veritor (antigen), midturbinate	99	55	44	<0.001
Positive	14 (14.1)	1 (1.8)	13 (29.5)
Negative	85 (85.9)	54 (98.2)	31 (70.5)

Reference NAT for 2 participants was performed on Panther; C values not available.

Evaluable culture results were not available for 3 of the reference NAT-positive samples.

Twenty-six samples were insufficient for PerkinElmer testing.

Forty-nine participants either did not produce saliva or had insufficient quantities to test.

Antigen test results.

From 17 November 2020 to 11 March 2021, the initial 126 participants underwent antigen testing of a midturbinate swab using the BD Veritor lateral flow assay, and among these participants, 99 were close contacts. Among close contacts who underwent antigen testing, 55 (56%) were asymptomatic, 35 (35%) were symptomatic and within the first 5 days of symptom onset, and 9 (9%) were symptomatic but beyond 5 days of symptom onset. Among symptomatic close contacts in the first 5 days of symptom onset (intended use case for the BD Veritor test), the positive percent agreement (PPA) was 83.3% (95% CI, 50.9 to 97.1) and the negative percent agreement was 95.7% (95% CI, 76.0 to 99.8) compared to the reference molecular test. However, among asymptomatic persons or persons after 5 days of symptom onset, the PPA was 18.2% (95% CI, 3.2 to 52.2). The median reference test C for participants who tested positive by reference test and negative by BD Veritor was 31.6 for asymptomatic participants, 30.8 for symptomatic in the first 5 days of symptom onset, and 30.3 for participants beyond 5 days of symptom onset (Fig. 2A). Among the 11 close contacts that were culture positive with BD Veritor results, 8 (72.7%) were positive by BD Veritor (Fig. 2B). Among the 3 that were missed, 2 were asymptomatic.

FIG 2

Antigen test positivity of contacts. (A) Cycle threshold (C) values stratified by asymptomatic, symptom onset of ≤5 days, and symptom onset of >5 days for the RT-PCR-positive reference assays, for which there were antigen lateral flow results (n = 99). (B) Culture, antigen result, and C of reference assay for all RT-PCR-positive NP samples arrayed by early symptomatic (blue shaded), symptomatic (white), and asymptomatic (gray).

DISCUSSION

In this study, we report the relative performance of multiple molecular test approaches in a large number of known SARS-CoV-2 NAT-confirmed case contacts. As expected, a high proportion (22.6%) of these contacts were reference lab NP NAT positive. However, an additional 11.6% were positive when both NP VTM and saliva were tested using a more sensitive platform; more than half of asymptomatic close contacts tested falsely negative using a high-complexity reference lab test. Patients with symptomatic SARS-CoV-2 were less likely to be missed by standard NAT testing, and there were no missed cases among patients who were not close contacts. Although prior data has shown that approximately half of patients who eventually test positive for SARS-CoV-2 initially test negative and test performance is worse in the first few days after exposure and before symptom onset (17, 18), work describing the potential to detect additional cases using more sensitive assays and additional specimen types is limited.

The FDA requires that SARS-CoV-2 NATs demonstrate concordance with existing authorized tests to gain EUA (19), which gives the impression that the clinical performance of all EUA SARS-CoV-2 NATs are approximately equivalent. However, among SARS-CoV-2 NATs with FDA EUA, there is a 100-fold difference between the highest and lowest levels of detection (LOD) reported using comparable FDA reference materials (20). In this study, testing using the PerkinElmer NAT, which has the lowest reported LOD, identified many cases missed by standard reference tests. Another source of missed positives were samples that tested positive in saliva but not NP samples. Similar to prior work, saliva was not more or less sensitive than NP samples, but it did identify a population of SARS-CoV-2 cases that was only partially overlapping with those found using NP samples (21–23). If testing resources were not scarce, a strategy of performing high sensitivity NAT on both saliva and NP specimens may be considered to exclude infection in patients with the highest pretest probability for infection.

Although antigen testing identified the majority of culture-positive contacts, a few patients with culture-positive SARS-CoV-2 were missed. It is expected that antigen testing will miss a proportion of SARS-CoV-2 cases, particularly those with lower viral burden (identified by higher C values) and particularly among asymptomatic individuals; a recent meta-analysis reported a sensitivity of 58.1% (95% CI, 40.2% to 74.1%) (24), and more focused studies including asymptomatic close contacts reported a sensitivity of 59 to 82% compared to NAT (8, 25). However, the hope has been that persons with potential to transmit SARS-CoV-2 may be excluded, at least in the moment, by a negative antigen test (15, 26, 27). Used as intended, in the first 5 days of symptom onset, only one patient with culture-positive SARS-CoV-2 tested negative by antigen testing, but two asymptomatic patients who tested negative by antigen tests had positive cultures. Recent data from our group and others have shown that serial testing may be a strategy to overcome the lack of sensitivity in presymptomatic patients (23, 28). Daily antigen testing is now being used by some school systems to forego quarantine after exposure (29, 30).

False negative SARS-CoV-2 test results for asymptomatic close contacts may provide false reassurance, leading to relaxation of isolation measures and greater onward transmission. Our findings suggest that for asymptomatic close contacts with a high pretest probability for infection, testing at a single time point with a standard reference NAT misses some cases, and negative antigen testing does not exclude transmissibility. SARS-CoV-2 exposures that are to household members, indoors, and prolonged pose a higher risk of transmission (31), which may warrant additional vigilance, including repeat testing for close contacts with high pretest probability for infection. Newer SARS-CoV-2 variants, including delta and omicron, have higher capacity for transmission than earlier circulating genotypes (32, 33). Therefore, the consequences for each false-negative test on community transmission may be greater now that currently circulating variants are more transmissible than earlier circulating SARS-CoV-2.

Limitations include the possibility that other commercially available antigen tests may outperform the one chosen for use in this study, although direct comparisons of the Veritor antigen test used in this study has shown similar performance characteristics to others (8, 34). Similarly, differences in the analytical sensitivity of SARS-CoV-2 test platforms may impact the results if this study were to be replicated using alternative SARS-CoV-2 NAT. The antigen tests were not performed as intended at the point of collection, which could reduce their sensitivity. Most of the positive cases were from unvaccinated persons prior to circulation of delta and omicron variants—as more vaccinated persons test positive for COVID, new data regarding the relationship between pretest probability of infection, test performance characteristics, SARS-CoV-2 variant, and culture positivity may be required. Data regarding timing of participant exposure to an individual with COVID were not available, which limits the ability to identify subgroups of close contacts with better or worse test performance.

Ongoing SARS-CoV-2 transmission with increasingly contagious variants highlights the continued importance of widespread and accurate testing even as the pandemic approaches its third year. Our findings in a large cohort of close contacts demonstrate the limitations of cross-sectional antigen testing to exclude transmissibility and standard NAT to diagnose COVID-19. Close contacts of individuals with COVID-19 should isolate and may need to test more than once, especially those with highest risk exposures such as household contacts. Future work is necessary to improve test performance characteristics or implement serial testing to identify patients with SARS-CoV-2 when they are most likely to have transmissible disease.