COVID-19 Vaccination Breakthrough Infections in a Real-World Setting: Using Community Reporters to Evaluate Vaccine Effectiveness.

Purpose: Coronavirus disease 2019 (COVID-19) has highlighted the need for new methods of pharmacovigilance. Here, we use adult community volunteers to obtain systematic information on vaccine effectiveness and the nature and severity of breakthrough infections.
Methods: Between December 15, 2020 and September 16, 2021, 11,826 unpaid community-based volunteers reported the following information to an on-line registry: COVID-19 test results, vaccination (Pfizer, Moderna, or Johnson & Johnson) and COVID-19 symptoms. COVID-19 infections were described based on vaccination status at the time of infection: 1) fully vaccinated, 2) partially vaccinated (received first of two-dose vaccines or were <14 days post-final dose), or 3) unvaccinated.
Results: Among 8554 participants who received any COVID-19 vaccine, COVID-19 infections were reported by 74 (1.0%) of those who were fully vaccinated and 198 (2.3%) of those who were partially vaccinated at the time of infection. Among the 74 participants who reported a breakthrough infection after full vaccination, the median time from vaccination to reported positive test result was 104.5 days (interquartile range: 77-135 days), with no difference among vaccine manufacturers. One quarter (25.7%) of breakthrough infections in the fully vaccinated cases were asymptomatic and most (>97%) fully vaccinated participants reported no symptoms or only mild symptoms compared to 89.3% of the unvaccinated cases. Only 1.4% of fully vaccinated participants reported experiencing at least 3 moderate-to-severe symptoms compared to 7.8% in the unvaccinated.
Conclusion: Person-generated health data, also referred to as patient-reported outcomes, is a useful approach for quantifying breakthrough infections and their severity and for comparing vaccines. Trial Registration: Clinicaltrials.gov NCT04368065, EU PAS Register EUPAS36240.

Background

The three Coronavirus disease 2019 (COVID-19) vaccines authorized by the United States Food and Drug Administration (FDA) as of this writing have been found to be safe and highly efficacious in preventing symptomatic, laboratory-confirmed COVID-19 infections in randomized controlled trials.1–3 But, as is not uncommon with even highly effective vaccines, these vaccines confer infection-permissive or “non-sterilizing” immunity, which may permit infection while preventing symptoms, and especially severe symptoms.4

Infection in vaccinated persons is referred to as a “breakthrough” infection, defined by the US Centers for Disease Control and Prevention (CDC) as those who have detectable SARS-CoV-2 RNA or antigen on a respiratory specimen collected ≥14 days after completion of a vaccine regimen.1 While cases of COVID-19 do occur after vaccination, an excess of breakthrough infections could suggest a lack of vaccine effectiveness, consistent with the FDA definition of an adverse event as including “failure to produce the expected pharmacologic action”.5 For COVID, these concerns may be exacerbated by the emergence of viral mutations conferring possible vaccine escape, including some known variants of concern, making prompt detection of lack of effectiveness a public health imperative.6

While some serious safety concerns remain for COVID-19 vaccines, eg, anaphylaxis7,8 and Guillain-Barre Syndrome,9 lack of vaccine effectiveness is of particular importance given the potential risk of severe COVID-19 disease and mortality. Such information may also be useful for addressing the hesitancy occasioned by the accelerated development of these vaccines.10,11. While there are some reports quantifying the extent of breakthrough infections and severity of any associated symptoms based on surveillance or electronic medical record data,12–19 person-generated data, sometimes called patient-reported health data or patient reported outcomes provide additional insights, especially for symptoms that may not occasion medical care.

The objective of this work was to describe breakthrough infections in adults who received any of the three COVID-19 vaccines currently authorized for use in the US (BioNTech (Pfizer), Moderna, Inc. (Moderna), and Janssen Pharmaceuticals (Johnson & Johnson (J&J)) or who were not vaccinated. We describe the number of infections reported and participant-reported COVID-19 symptom information, stratified by vaccination status (fully vaccinated, partially vaccinated, and unvaccinated at the time of infection), illustrating how data generated by community reporters (here, people with confirmed COVID-19 infections) can be used for pharmacovigilance.

Materials and Methods

We leveraged data from a web-based registry, the COVID-19 Active Registry Experience (CARE, see ). CARE was launched on April 2, 2020 to study COVID-19 symptoms and severity outside of the hospital setting to examine what factors, if any, mitigate the impact of infection with COVID-19.20–22 The protocol and web technology are updated periodically, with the COVID-19 vaccination questions reported here added on December 15, 2020. Participants are recruited via social media, targeting adult US residents, and are not compensated for participation. All provide informed consent online. Participants are asked to complete surveys once a week for the first month after enrollment, and then twice a month after the first month. They report information on their demographics, medical history, COVID-19-like symptoms and severity,20 COVID-19 tests, COVID-19 vaccination, and use of prescription and non-prescription medications and dietary supplements in the enrollment and follow-up surveys. Those who report having been vaccinated are also asked to provide the date of vaccination and side effects for first and second doses, as appropriate, manufacturer, and lot number. Participants are instructed to refer to their vaccination card to enhance data accuracy. This study was approved by an Institutional Review Board (Advarra) and registered with Clinicaltrials.gov NCT04368065, and the study complies with the Declaration of Helsinki.

Surveys were submitted by 11,927 participants between December 15, 2020 and September 16, 2021. We excluded 101 reports of unauthorized vaccines, receipt of doses from more than one manufacturer, inconsistent dates, or vaccine dates earlier than June 2020. The analytic population consisted of 11,826 participants, including 8554 who had ever received a Pfizer, Moderna, or J&J vaccine, and 3272 who were not vaccinated during the study period. Among the 8554 participants, 7532 completed vaccination (ie, two doses of Pfizer or Moderna, or one dose of J&J), and 1022 had an incomplete vaccination regimen (ie, one dose of Pfizer or Moderna). Vaccination status was determined at the time of their positive COVID-19 test (Figure 1). For those who reported multiple positive COVID-19 test results, we considered tests within 90 days of each other as the same infection23 and used the first positive test to mark the start of the infection. Only a few participants (21 [0.2%] out of 11,826) have reported positive tests more than 90 days apart; for these participants we used the first positive test of the most recent course of infection. For participants who never received any COVID-19 vaccines during follow-up, we examined all positive tests occurring after December 15, 2020. Thus, participants only had one test result selected for this analysis and one corresponding vaccination status at the time of that test result, with no minimum requirement of follow-up time. For the analytic sample (n = 11,826), the average follow-up time is 73 days (median: 32 days, range: 0–502 days).

Figure 1

COVID-19 breakthrough infection journey by manufacturer of COVID vaccine (A Janssen; B Pfizer-BioNTech or Moderna).

To assess potential confounding of other characteristics on COVID-19 infection symptoms, we performed two multivariate logistic regression models on these two outcomes respectively: (1) reporting at least one symptom (vs no symptom), and (2) reporting ≥1 moderate/severe symptom (vs no moderate/severe symptom). Covariates included in the models included age, gender, education, race, ethnicity, BMI category, smoking status, influenza vaccination status, cancer treatment status (yes/no), and any history of other medical condition (yes/no).

Results

Characteristics of Study Participants

Of 11,826 participants in our study population, 8554 reported receiving at least one dose of an FDA-authorized vaccine and 3272 did not report any vaccination. The average age among the study participants was 46.5 years (range: 18–91 years). The majority were female (82.9%), with some college or higher education (87.6%), White (86.5%), non-smokers (88.7%), and also had received prior vaccination for influenza (67.5%) (Table 1). Compared with non-vaccinated participants, those who received any vaccination were older (47.1% of ≥50 years vs 31.4%), had higher proportion of pregnant women (14.3% vs 0.0%), completed 4-year college or higher degree (60% vs 38.1%), were White (89.9% vs 77.5%) and non-Hispanic or Latino (93.5% vs 83.23%), non-smoker (90.4% vs 84.31%) and reported history of medical conditions such as depression and cardiovascular diseases.

Table 1

Characteristics at Enrollment of Study Participants*

	All Study Participants N=11,826	By Vaccination Status
		Participants who Received at Least 1 COVID-19 Vaccine N=8554	Participants who Did Not Receive Any COVID-19 Vaccine N=3272
Age in years (N)	11,823	8553	3270
Mean (SD)	46.5 (15.2)	47.9 (15.42)	42.8 (13.8)
Age Group (years, n[%])	11,823	8553	3270
18–29	1601 (13.5)	994 (11.6)	607 (18.6)
30–39	2932 (24.8)	2094 (24.5)	838 (25.6)
40–49	2238 (18.9)	1439 (16.8)	799 (24.4)
50–59	2161 (18.3)	1590 (18.6)	571 (17.5)
≥60	2891 (24.5)	2436 (28.5)	455 (13.9)
Gender, n (%)	11,825	8554	3271
Female	9800 (82.9)	7137 (83.4)	2663 (81.4)
Male	1772 (15.0)	1200 (14.0)	572 (17.5)
Transgender/Other/ Not Disclosed	253 (2.1)	217 (2.5)	36 (1.1)
Pregnant, n (%)	9800	7137	2663
Yes	1019 (10.4)	1019 (14.3)	0 (0.0)
Education, n (%)	11,803	8538	3265
High school or less	1466 (12.4)	782 (9.2)	684 (21.0)
Some college	3966 (33.6)	2629 (30.8)	1337 (41.0)
4-yr college degree	2865 (24.3)	2212 (25.9)	653 (20)
>4yr college degree	3506 (29.7)	2915 (34.1)	591 (18.1)
Race, n (%)	11,805	8514	3264
White	10,208 (86.5)	7680 (89.9)	2528 (77.5)
Black or African American	384 (3.3)	149 (1.7)	235 (7.2)
Other**	1213 (10.3)	712 (8.3)	501 (15.4)
Ethnicity, n (%)	11,705	8491	3214
Hispanic or Latino	1092 (9.3)	553 (6.5)	539 (16.8)
BMI, n (%)	11,607	8399	3208
Underweight or Normal weight (15.0 ≤ BMI< 25.0)	3156 (27.19)	2240 (26.7)	916 (28.6)
Overweight (25.0 ≤ BMI < 30.0)	3146 (27.1)	2328 (27.7)	818 (25.5)
Obese (30.0 ≤ BMI ≤ 40.0)	3871 (33.4)	2820 (33.4)	1051 (32.8)
Severe Obesity (BMI>40.0)	1434 (12.4)	1011 (12.0)	423 (13.2)
Smoker, n (%)	11,371	8217	3154
Yes	1284 (11.29)	789 (9.6)	495 (15.7)
Vaccinated for Influenza, n (%)	10,954	8157	2797
Yes	7393 (67.5)	6012 (73.7)	1381 (49.4)
Medical Conditions, n (%)	11,826	8554	3272
Anxiety	1963 (16.6)	1459 (17.1)	504 (15.4)
Autoimmune disease	775 (6.55)	593 (6.9)	182 (5.6)
Blood disorder	74 (0.63)	69 (0.8)	5 (0.15)
Cardiovascular disease	320 (2.7)	300 (3.5)	20 (0.61)
Depression	2251 (19.03)	1742 (20.4)	509 (15.6)
Diabetes	973 (8.2)	712 (8.3)	261 (8.0)
Hypertension	2416 (20.4)	1794 (21.0)	622 (19.0)
Insomnia or trouble sleeping	678 (5.7)	512 (6.0)	166 (5.1)
Kidney disease	34 (0.3)	31 (0.4)	3 (0.09)
Lung disease	841 (7.1)	625 (7.3)	216 (6.6)
Organ transplant	13 (0.1)	13 (0.2)	0 (0.0)
Whether on Treatment for Cancer, n (%)	11,764	8529	3235
Yes	123 (1.1)	99 (1.2)	24 (0.7)

Notes: *Percentages in this table are column percentages. **Including 226 Asians, 67 American Indian or Alaska Native, 12 Native Hawaiian or other Pacific Islander, and 908 Other.

Among vaccinated participants, comparison by vaccine manufacturer showed no difference in gender, race, and ethnicity, but slight differences in age and some comorbid conditions: age: 49.4 Moderna, 47.0 J&J, and 46.9 Pfizer (p < 0.01); cardiovascular disease: 4.3% Moderna vs 3.8% J&J and 2.8% Pfizer (p < 0.01); diabetes: 9.3% Moderna vs 8.0% J&J and 7.6% Pfizer (p = 0.02); hypertension: 23.9% Moderna vs 17.6% J&J and 19.1% Pfizer (p < 0.001); insomnia: 6.7% Moderna vs 7.1% J&J and 5.2% Pfizer (p < 0.01).

COVID-19 Infected Study Participants

The counts of participants who reported a positive COVID-19 test by vaccination status at the time of infection are shown in Figure 2 and Table 2. Among the 7532 fully vaccinated participants, 74 (1.0%) participants reported an infection, a proportion that was generally similar across manufacturers (n= 6 (0.9%) for J&J, n = 52 (1.4%) for Pfizer, and n = 16 (0.5%) for Moderna). The proportion of participants who reported an infection when they were partially vaccinated was 2.3% (n = 198), also comparable across manufacturers with 0.2% for J&J, 2.4% for Pfizer, and 2.6% for Moderna.

Figure 2

Flowchart of study sample and participant counts with COVID-19 infection.

Table 2

Number of Participants with COVID-19 Infections by Vaccination Status and Characteristics of Infection

	Total	Ever Vaccinated			Never Vaccinated
		J & J	Pfizer	Moderna
Total number of study participants	11,826	652	4322	3580	3272
Ever vaccinated	8554	652	4322	3580
o Completed vaccine regimen	7532	652	3788	3092
– Completed two doses of Pfizer/Moderna	6880		3788	3092
o Received only one dose of Pfizer/Moderna	1022		534	488
Not vaccinated	3272				3272
Total number of infected participants	3290	108	743	581	1858
Infection while fully vaccinated (ie, after 14 days of last dose)*, n (%)	74 (1.0)	6 (0.9)	52 (1.4)	16 (0.5)
Infected while partially vaccinated€, n(%)	198 (2.3)	1 (0.2)	103 (2.4)	94 (2.6)
o After final dose, awaiting presumed immunity (0–13 days after final dose)	15 (0.2)	1 (0.2)	12 (0.3)	2 (0.1)
o Between two doses	117 (1.7)		64 (1.7)	53 (1.7)
o After 1st dose and did not receive 2nd dose	66 (6.5)		27 (5.1)	39 (8.0)
Infected while unvaccinated/before vaccinationΩ, n(%)	3018 (25.5)	101 (15.5)	588 (13.6)	471 (13.2)	1858 (56.8)
o Infection prior to first dose	1160 (13.6)	101 (15.5)	588 (13.6)	471 (13.2)
o Infection and never vaccinated	1858 (56.8)				1858 (56.8)
Time interval from COVID-19 vaccination to infection
For infections after fully vaccinated, number of days since presumed immunity (N)	74	6	52	16	N/A
Mean (SD)	101.4 (44.99)	78.7 (61.17)	100.1 (42.82)	113.9 (44.64)	N/A
Median [IQR]	104.5 [77.0–135.0]	91.5 [10.0–127.0]	104.0 [75.0–132.5]	108.5 [88.5–148.0]	N/A
Min - Max	2–196	2–150	12–196	14–190	N/A
For infections after partially vaccinated, number of days since first dose before receiving the 2nd dose, or days since completed doses and before reaching full immunity) (N)	198	1	103	94	N/A
Mean (SD)	11.3 (21.84)	8.0 (-)	10.4 (14.24)	12.4 (28.03)	N/A
Median [IQR]	8.0 [5.0–12.0]	8.0 [8.0–8.0]	8.0 [5.0–13.0]	8.0 [5.0–12.0]	N/A
Min - Max	0–223	8–8	0–128	0–223	N/A

Notes: *Percentages based on 7532 participants who had completed doses. €Percentages based on 8554 participants who received any vaccination. ΩPercentages based on all 11,826 participants.

Among 74 participants who reported a breakthrough infection after full vaccination, the median time from 14 days post-final vaccination dose to reported positive test result was 104.5 days, Interquartile range (IQR): 77–135 days, with no significant differences across vaccine manufactures: 91.5 days for J&J, 104.0 days for Pfizer, and 108.5 days for Moderna (p-value = 0.44) (Table 2). Among the 198 people infected after partial vaccination, the median time from the first dose to infection was 8.0 days (IQR: 5.0–12.0 days).

We also looked at the infection counts by age group (Table 3). Participants aged 60 years or more had slightly lower infection rates than younger participants at most time points of vaccine status.

Table 3

Number of Participants with COVID-19 Infections by Age Group

	Total	By Age Group
		18–29 Years	30–39 Years	40–49 Years	50–59 Years	60+ Years
Total number of study participants	11,826	1601	2932	2238	2161	2891
Ever vaccinated	8554	994	2094	1439	1590	2436
o Completed vaccine regimen	7532	827	1873	1224	1375	2233
– Completed two doses of Pfizer/Moderna	6880	734	1746	1104	1225	2071
o Received only one dose of Pfizer/Moderna	1022	167	221	215	215	203
Not vaccinated	3272	607	838	799	571	455
Total number of infected participants	3290	594	907	784	587	418
Infection while fully vaccinated (ie, after 14 days of last dose)*, n (%)	74(1.0%)	12(1.5%)	22(1.2%)	14(1.1%)	8(0.6%)	18(0.8%)
Infected while partially vaccinated€, n(%)	198(2.3%)	25(2.5%)	59(2.8%)	43(3.0%)	40(2.5%)	31(1.3%)
o After final dose, awaiting presumed immunity (0–13 days after final dose)	15(0.2%)	2(0.2%)	6(0.3%)	3(0.2%)	2(0.1%)	2(0.1%)
o Between two doses	117(1.7%)	12(1.6%)	33(1.9%)	28(2.5%)	28(2.3%)	16(0.8%)
o After 1st dose and did not receive 2nd dose	66(6.5%)	11(6.6%)	20(9.0%)	12(5.6%)	10(4.7%)	13(6.4%)
Infected while unvaccinated/before vaccinationΩ, n(%)	3018(25.5%)	557(34.8%)	826(28.2%)	927(41.4%)	539(24.9%)	369(12.8%)
o Infection prior to first dose	1160(13.6%)	146(14.7%)	266(12.7%)	266(18.5%)	265(16.7%)	217(8.9%)
o Infection and never vaccinated	1858(56.8%)	411(67.7%)	560(66.8%)	461(57.7%)	274(48.0%)	152(33.4%)
Time interval from COVID-19 vaccination to infection
For infections after fully vaccinated, number of days since presumed immunity (N)	74	12	22	12	8	18
Mean (SD)	101.4 (44.99)	96.3 (46.90)	109.3 (50.00)	92.2 (45.14)	99.4 (37.72)	103.0 (43.19)
Median [IQR]	104.5 [77.0–135.0]	110.5 [46.5–138.0]	105.5 [77.0–143.0]	98.0 [77.0–120.0]	105.0 [99.0–113.5]	104.0 [79.0–127.0]
Min - Max	2–196	29–150	2–196	10–150	14–146	13–178
For infections after partially vaccinated, number of days since first dose before receiving the 2nd dose, or days since completed doses and before reaching full immunity) (N)	198	25	59	43	40	31
Mean (SD)	11.3 (21.84)	6.8 (3.77)	9.0 (5.36)	12.3 (18.74)	17.9 (42.42)	9.8 (12.05)
Median [IQR]	8.0 [5.0–12.0]	6.0 [4.0–9.0]	8.0 [5.0–12.0]	10.0 [5.0–14.0]	7.0 [4.0–11.5]	7.0 [3.0–13.0]
Min - Max	0–223	0–15	0–28	1–128	0–223	0–66

 Notes: *Percentages based on 7532 participants who had completed doses. € Percentages based on 8554 participants who received any vaccination. Ω Percentages based on all 11,826 participants.

Among 74 people infected after full vaccination, their average age was 44.5 years (Table 4). Most were female (79.7%), White (91.9%), and 40.5% had above a college degree. There were small differences in age across the three vaccination status groups of infected people, with 24.3% of people aged ≥60 years in the fully vaccinated group and 15.7% and 12.2% in partially vaccinated and unvaccinated, respectively. Among the partially vaccinated and unvaccinated, there were fewer male participants (13.1% for both partially vaccinated and unvaccinated versus 17.6% among fully vaccinated) and they had less education (31.8% and 20.3%, were college-educated versus 40.5% among fully vaccinated). Other demographics and medical characteristics including hypertension, diabetes, obesity, were similarly distributed.

Table 4

Characteristics of COVID-19 Positive Participants at Enrollment *

	All COVID-19 Infections N=3290	Infected After Full Vaccination N=74	Infected After Partial Vaccination N=198	Infected While Not Vaccinated N=3018
Age in years (N)	3290	74	198	3018
Mean (SD)	42.3 (13.40)	44.5 (14.98)	44.3 (13.03)	42.2 (13.37)
Age Group (years, n[%])	3290	74	198	3018
18–29	594 (18.1)	12 (16.2)	25 (12.6)	557 (18.5)
30–39	907 (27.6)	22 (29.7)	59 (29.8)	826 (27.4)
40–49	784 (23.8)	14 (18.9)	43 (21.7)	727 (24.1)
50–59	587 (17.8)	8 (10.8)	40 (20.2)	539 (17.9)
≥60	418 (12.7)	18 (24.3)	31 (15.7)	369 (12.2)
Gender, n(%)	3290	74	198	3018
Female	2821 (85.7)	59 (79.7)	170 (85.9)	2592 (85.9)
Male	434 (13.2)	13 (17.6)	26 (13.1)	395 (13.1)
Transgender/Other/ Not Disclosed	35 (1.1)	2 (2.7)	2 (1)	31 (1.0)
Pregnant, n(%)	2821	59	170	2592
Yes	73 (2.6)	12 (20.3)	3 (1.8)	58 (2.2)
Education, n(%)	3289	74	198	3017
High school or less	564 (17.2)	4 (5.4)	21 (10.6)	539 (17.9)
Some college	1304 (39.7)	21 (28.4)	70 (35.4)	1213 (40.2)
4-yr college degree	715 (21.7)	19 (25.7)	44 (22.2)	652 (21.6)
>4yr college degree	706 (21.5)	30 (40.5)	63 (31.8)	613 (20.3)
Race, n(%)	3286	74	198	3014
Black or African American	202 (6.2)	3 (4.1)	6 (3.0)	193 (6.4)
White	2640 (80.3)	68 (91.9)	179 (90.4)	2393 (79.4)
Other**	444 (13.5)	3 (4.1)	13 (6.6)	428 (14.2)
Ethnicity	3245	74	195	2976
Hispanic or Latino	538 (16.6)	5 (6.8)	20 (10.3)	513 (17.2)
BMI, n(%)	3242	73	198	2971
Underweight or Normal weight (15.0 ≤ BMI< 25.0)	795 (24.5)	13 (17.8)	36 (18.2)	746 (25.1)
Overweight (25.0 ≤ BMI < 30.0)	825 (25.5)	22 (30.1)	53 (26.8)	750 (25.2)
Obese (30.0 ≤ BMI ≤ 40.0)	1145 (35.2)	28 (38.4)	79 (39.9)	1038 (34.9)
Severe Obesity (BMI>40.0)	477 (14.7)	10 (13.7)	30 (15.2)	437 (14.7)
Smoking status, n(%)	3211	73	197	2941
Smoker	342 (10.7)	8 (11.0)	18 (9.1)	316 (10.7)
Vaccinated previously for Influenza, n(%)	3231	74	194	2963
Vaccinated	1867 (57.8)	58 (78.4)	162 (83.5)	1647 (55.6)
Medical Conditions, n(%)	3290	74	198	3018
Anxiety	608 (18.5)	12 (16.2)	41 (20.7)	555 (18.4)
Autoimmune disease	192 (5.8)	6 (8.1)	14 (7.1)	172 (5.7)
Blood disorder	10 (0.3)	0	1 (0.5)	9 (0.3)
Cardiovascular disease	19 (0.6)	3 (4.1)	2 (1.0)	14 (0.5)
Depression	594 (18.1)	12 (16.2)	57 (28.8)	525 (17.4)
Diabetes	238 (7.2)	5 (6.8)	13 (6.6)	220 (7.3)
Hypertension	634 (19.3)	13 (17.6)	51 (25.8)	570 (18.9)
Insomnia or trouble sleeping	210 (6.4)	3 (4.1)	14 (7.1)	193 (6.4)
Kidney disease	4 (0.1)	0	0	4 (0.1)
Lung disease	221 (6.7)	4 (5.4)	14 (7.1)	203 (6.7)
Organ transplant	1 (0.03)	0	0	1 (0.03)
Whether on Treatment for Cancer, n (%)	3272	74	197	3001
Yes	29 (0.9)	1 (1.4)	3 (1.5)	25 (0.8)

Notes: *Percentages in this table are column percentages. **Including 69 Asians, 26 American Indian or Alaska Native, 4 Native Hawaiian or other Pacific Islander, and 345 Other.

COVID-19 Symptoms and Impact Among Vaccinated vs Unvaccinated

The fully vaccinated group with breakthrough infections had lower proportions of every COVID-19 symptom compared to the partially vaccinated and unvaccinated groups, including headache, aches and pains, decreased sense of taste and smell, chills, and diarrhea (Table 5 and Figure 3). Fully vaccinated breakthrough cases reported fewer mean number of COVID-19 symptoms (4.1) when compared to partially vaccinated (5.8) or unvaccinated cases (5.3).

Table 5

COVID-19 Infection Symptoms Among COVID-19 Positive Participants*

	All COVID-19 Infections N=3290	Infected After Full Vaccination N=74	Infected After Partial Vaccination N=198	Infected While Not Vaccinated N=3018
COVID-19 symptoms, n (%)
Aches and Pains	1381 (42.0)	18 (24.3)	76 (38.4)	1287 (42.6)
Chills	850 (25.8)	12 (16.2)	58 (29.3)	780 (25.8)
Decreased Sense of Smell	1469 (44.7)	23 (31.1)	84 (42.4)	1362 (45.1)
Cough	1606 (48.8)	31 (41.9)	108 (54.6)	1467 (48.6)
Decreased Sense of Taste	1349 (41.0)	25 (33.8)	77 (38.9)	1247 (41.3)
Diarrhea	784 (23.8)	10 (13.5)	48 (24.2)	726 (24.1)
Fatigue	2051 (62.3)	44 (59.5)	136 (68.7)	1871 (62.0)
Fever	680 (20.7)	8 (10.8)	46 (23.2)	626 (20.7)
Nausea	712 (21.6)	12 (16.2)	44 (22.2)	656 (21.7)
Nasal Congestion	1677 (51.0)	32 (43.2)	105 (53.0)	1540 (51.0)
Runny Nose	1054 (32.0)	20 (27.0)	73 (36.9)	961 (31.8)
Shortness of Breath	906 (27.5)	16 (21.6)	53 (26.8)	837 (27.7)
Sore Throat	906 (27.5)	19 (25.7)	67 (33.8)	820 (27.2)
Vomiting	164 (5.0)	2 (2.7)	12 (6.1)	150 (5.0)
Headache	1744 (53.0)	31 (41.9)	117 (59.1)	1596 (52.9)
Number of symptoms
Mean (SD)	5.3 (3.87)	4.1 (3.73)	5.8 (3.90)	5.3 (3.87)
Median [IQR]	5.0 [2.0–8.0]	4.0 [0.0–6.0]	5.0 [2.0–9.0]	5.0 [2.0–8.0]
Min-Max	0–15	0–14	0–15	0–15
Category of number of symptoms, n (%)
0 symptoms	542 (16.5)	19 (25.7)	27 (13.6)	496 (16.4)
1–2 symptoms	418 (12.7)	10 (13.5)	27 (13.6)	381 (12.6)
3+ symptoms	2330 (70.8)	45 (60.8)	144 (72.7)	2141 (70.9)
Category of number of moderate to severe symptoms, n(%)
0 moderate or severe symptoms	2945 (89.7)	72 (97.3)	182 (92.4)	2691 (89.3)
1–2 moderate or severe symptoms	91 (2.8)	1 (1.4)	3 (1.5)	87 (2.9)
3+ moderate or severe symptoms	247 (7.5)	1 (1.4)	12 (6.1)	234 (7.8)
Hospitalization
Hospitalized	54 (1.6)	1 (1.4)	4 (2)	49 (1.6)

Note: *Percentages in this table are column percentages.

Figure 3

Percentage of COVID-19 infected participants with symptoms by vaccination status.

Approximately one quarter (25.7%) of fully vaccinated breakthrough cases reported no symptoms compared with 13.6% and 16.4% among partially vaccinated and unvaccinated cases, respectively (Figure 4). Fully vaccinated breakthrough cases were also more likely to report only mild symptoms and not report moderate or severe symptoms (97.3%) compared with partially vaccinated or unvaccinated cases (92.4% and 89.3%, respectively) (Table 5 and Figure 5). Hospitalization due to COVID infection was rare, with 1.4% reporting hospitalization among the 74 infected after full vaccination, 1.6% among the 198 infected after partial vaccination, and 2.0% among the 3018 infected while unvaccinated, respectively (Table 5 and Figure 6).

Figure 4

Number of symptoms among COVID-19 infected participants by vaccination status.

Figure 5

Number of moderate or severe symptoms among COVID-19 infected participants by vaccination status.

Figure 6

Percentage of hospitalized participants among COVID-19 infected by vaccination status.

When accounting for the effects of potential confounders, results of both models suggested a protective effect of full vaccination on symptoms although the associations were not statistically significant (Odds Ratios < 1, p-values = 0.13 and 0.24 for association between vaccination status and reporting any symptom or reporting any moderate/severe symptom, respectively, and ).

Discussion

Systematic collection of person-generated health data as illustrated by the CARE registry demonstrates the value of coupling patient symptom information with COVID-19 test data to provide meaningful and timely information about vaccine effectiveness. This study corroborates other work showing that vaccination affords significant protection against severe symptoms of COVID-19 disease, and that it also confers protection against infection overall.12–18 Here 1.0% of all participants reported infections after full vaccination, consistent with the low rates reported elsewhere.24 Fully vaccinated individuals who experienced breakthrough infections reported fewer symptoms than partially vaccinated or unvaccinated at the time of study, with only a small fraction of fully vaccinated cases reporting moderate or severe symptoms and about one quarter experienced no symptoms at all. Although the multivariate logistic regression models suggested protection against symptom occurrence, the lack of statistical significance could be due to the relatively small study size and the large number of covariates.

In this study, there were a few differences between those who reported COVID-19 infection after full vaccination and those who reported infection while partially vaccinated. Those fully vaccinated had a slightly higher proportion of infection among males, individuals aged 60 years or above and those with at least a college education. These older and more educated participants may have been vaccinated earlier, and therefore had a longer “at risk” time for infection and/or potentially waning immunity after being fully vaccinated. Infection counts by age group in Table 3 showed that participants aged 60 years or more had slightly lower infection rates than younger participants at most time points of vaccine status. However, the results should be cautiously interpreted given the small sample sizes.

Also of interest, looking at all fully vaccinated participants, the median time to breakthrough infection was 105 days, ranging from 2 to 196 days. Other studies have found similarly long time to breakthrough infections, including a study in Massachusetts that reported a median time of 86 days after final vaccine dose.13,25 Interestingly, in this study the average time to breakthrough infection by manufacturer was not significantly different between manufacturers, with 91.5, 104.0, and 108.5 days for J&J, Pfizer, and Moderna, respectively. The proportions of participants reporting infections after completing the vaccination regimen were low, but they were notably higher (1.4%) for participants vaccinated with Pfizer as compared to J&J (0.9%) and Moderna (0.5%).

Among the 198 people infected after partial vaccination, the median time to infection was 8.0 days. The proportion of participants reporting infection within 14 days of completing at least one vaccination were similar for those vaccinated with Pfizer (2.4%) and Moderna (2.6%), and lower for those vaccinated for J&J (0.2%). Although this average time is much shorter than those who were fully vaccinated, it should be noted that the window for having infection while partially vaccinated was, by definition, shorter for J&J with a window of only 14 days until full immunity, compared to about 28 days between doses plus 14 days post-2nd dose (42 days) for those receiving Pfizer or Moderna.

Of note, several factors that might be expected to be associated with an increased risk of infection, such as hypertension, diabetes, and obesity, were not associated with COVID-19 infection in the fully vaccinated. This observation might offer encouraging mechanistic support for vaccine effectiveness.

This study has a number of strengths that enhance its information value. The rich symptom data permits more contextualization of the effectiveness of the vaccine than simply reporting a positive or negative COVID-19 test result. Vaccine reporting is likely to be more complete than the vaccination status recorded in most real-world data sources such as insurance claims data and/or electronic health record datasets, considering how vaccines were distributed in the United States and the lack of any single accessible central source of vaccination status information. Similarly, COVID-19 test results are also likely to be more complete than in many real-world databases, since participants were systematically queried to see if they were tested at every bi-weekly follow-up.

Nonetheless it is important to keep in mind that, like other online registries that survey volunteers without intervention or collaboration from medical care providers, the CARE registry relies on participants to contribute information completely, faithfully, and accurately. Missing data, especially outcomes of interest, are possible. Also, there was no independent validation of COVID-19 test result reporting, nor were there any clinical assessments of these person-generated symptoms. However, CARE emphasized the importance of data accuracy for example, by instructing participants to refer to their vaccination card when entering vaccination history. Furthermore, the validity of reports from lay people on medication use has been demonstrated,26 suggesting that reporting quality is likely to apply at least as well to vaccine reporting.

There also may be underreporting of breakthrough cases and misclassification of their timing and severity. Participation in follow-up surveys depends on sustained personal interest, and participants may have tested positive after reporting their vaccine information but not completed any follow-up surveys. It is also possible that individuals with asymptomatic breakthrough infections were not aware of their infection in the absence of widespread systematic surveillance testing in most parts of the US at the time these data were collected. Very serious cases of COVID-19 that required hospitalization or resulted in death may be underreported. Analytically, if only one positive test was reported, we assumed the positive test date reflected the start of the infection, whereas the infection may have started earlier and gone undetected prior to vaccination initiation.

The representativeness and generalizability of observational, volunteer-driven studies like CARE might be questioned. CARE has broad geographic representation, with participants from all 50 states but CARE participants are not a systematic sample of the US adult population. CARE participants are older, more likely to be female, and have less racial and ethnic diversity than adults living in the US. Respondents include those with internet access and sufficient availability and interest to participate in surveys. Projects like this tend to attract more females as well as those “worried well”, with higher participation among those with autoimmune disorders as well as anxiety, depression and insomnia. Nonetheless, it is unlikely that comparisons made here between vaccine manufacturers were biased as differential participation by vaccine manufacturers is unlikely, especially since there no isolated widespread new events drawing attention to one of these vaccines or another.

Conclusion

The systematic collection of person-generated health data shows that only a small proportion of vaccinated people became infected after vaccination, a majority of whom were asymptomatic or had fewer and milder symptoms than the unvaccinated, and that while cases of COVID-19 may occur in vaccinated individuals, these are uncommon and generally mild in nature. This efficient approach of direct-to-patient research can be used obtain information not readily available in most real-world data sources, and can be used effectively to study adverse events as well as medical product effectiveness.