Factors indicating intention to vaccinate with a COVID-19 vaccine among older U.S. adults.

BACKGROUND: The success of vaccination efforts to curb the COVID-19 pandemic will require broad public uptake of immunization and highlights the importance of understanding factors associated with willingness to receive a vaccine.
METHODS: U.S. adults aged 65 and older enrolled in the HeartlineTM clinical study were invited to complete a COVID-19 vaccine assessment through the HeartlineTM mobile application between November 6-20, 2020. Factors associated with willingness to receive a COVID-19 vaccine were evaluated using an ordered logistic regression as well as a Random Forest classification algorithm.
RESULTS: Among 9,106 study participants, 81.3% (n = 7402) responded and had available demographic data. The majority (91.3%) reported a willingness to be vaccinated. Factors most strongly associated with vaccine willingness were beliefs about the safety and efficacy of COVID-19 vaccines and vaccines in general. Women and Black or African American respondents reported lower willingness to vaccinate. Among those less willing to get vaccinated, 66.2% said that they would talk with their health provider before making a decision. During the study, positive results from the first COVID-19 vaccine outcome study were released; vaccine willingness increased after this report.
CONCLUSIONS: Even among older adults at high-risk for COVID-19 complications who are participating in a longitudinal clinical study, 1 in 11 reported lack of willingness to receive COVID-19 vaccine in November 2020. Variability in vaccine willingness by gender, race, education, and income suggests the potential for uneven vaccine uptake. Education by health providers directed toward assuaging concerns about vaccine safety and efficacy can help improve vaccine acceptance among those less willing. TRIAL REGISTRATION: Clinicaltrials.gov NCT04276441.

Introduction

With the recent FDA Emergency Use Authorization (EUA) for three COVID-19 vaccines, significant attention is now being placed on whether sufficient numbers of the public will be willing to be immunized to control the pandemic and how to ensure the public is adequately informed about the vaccine [1-3]. Surveys in the United States showed an initial decline in reported willingness to receive a COVID-19 vaccine [4-9], though some recent data appear more promising [5, 10, 11].

Individual perceptions about vaccines and about COVID-19 can strongly influence the decision to vaccinate against COVID-19 and are likely more associated with vaccine behaviors than demographics alone [9, 12–14]. Vaccine uptake in older adults is of particular importance as increasing age is the leading risk factor for mortality and complications from COVID-19 infections [15, 16]. Guidelines have been developed that identify priority populations for vaccination (CDC.gov), with recommendations including those older than 65 years. Understanding who is least likely to vaccinate as well as potentially modifiable factors influencing their decisions, is important to develop public health strategies to overcome vaccine hesitancy, especially among older and higher risk populations [15].

Smartphone-based research can facilitate rapid data collection for timely research questions. To accelerate public health’s understanding of current perspectives on vaccinations, we surveyed subjects already participating in a smartphone-based clinical trial, the HeartlineTM Study, a virtual clinical study that is enrolling U.S. adults age 65 years and older. These participants are at higher risk of COVID-19 related morbidity and mortality due to their age, and thus have a large potential benefit from immunization. We hypothesized that older adults in our sample population would have a high willingness to vaccinate and sought to understand factors associated with those less willing. In order to understand factors associated with and indicative of willingness to vaccinate in this higher risk population, we deployed a vaccine survey to all participants through the HeartlineTM platform in November, 2020.

Materials and methods

Study population and data collection

On February 25, 2020, enrollment began into The HeartlineTM Study (clinicaltrials.gov, NCT04276441 and https://heartline.com/), a large heart health clinical study in the United States. Eligibility requirements include age 65 and older, possessing an iPhone 6s or later, current Original Medicare beneficiary, U.S. resident, and English-speaking (see clinicaltrials.gov, NCT04276441 for complete inclusion and exclusion criteria). The HeartlineTM Study is investigating if wearable and custom-built mobile app technologies can enable earlier detection of atrial fibrillation (AF), reduce the incidence of clinical events, and improve adherence with oral anticoagulants in those with AF. The study is completely virtual, without the need for study site visits, and is conducted on the HeartlineTM app. The study platform is designed to introduce novel surveys as needed throughout the study. The study protocol was approved by Western Institutional Review Board and all participants submit a signed informed consent to analyze de-identified data through the HeartlineTM app.

Willingness to vaccinate against COVID-19 was assessed through an optional survey through the HeartlineTM app between November 6 and 20, 2020. The assessment was framed by the World Health Organization recommended Capability, Opportunity, Motivation and Behavior model (COM-B) model for addressing vaccine hesitancy and acceptance [17]. The assessment included questions regarding beliefs about vaccines in general, beliefs about COVID-19 and the COVID-19 vaccine, and opinions on vaccine dosing and potential side effects (see S1 Table for the complete assessment). The assessment was offered to all study participants. Demographic data were collected at the time of study enrollment, which included race and gender by self-report. The primary outcome of the study was self-reported willingness to receive a COVID-19 vaccine. This was captured on a four-point scale (very willing, somewhat willing, not very willing, not at all willing).

Statistical analysis and model

To evaluate factors associated with vaccine willingness, we applied two different analytic approaches. First, determinants of willingness to be vaccinated were evaluated using an ordered univariate logistic regression model with the 4 levels of willingness to be vaccinated as the outcome while adjusting for race (Asian, Black or African American, White and other) and gender. Next, to identify a set of determinants to separate those who are willing to vaccinate from those who are not, recursive feature elimination in combination with a Random Forest classification algorithm [18] was performed. In this analysis, willingness to vaccinate was classified into two categories: willing to vaccinate (combining very willing with somewhat willing) and not willing to vaccinate (combining not at all willing with not very willing). The algorithm was trained using a dataset that was randomly split by stratum (either not willing or willing to be vaccinated), into a 2/3 training set (n = 4935 with 91.3% willing to vaccinate) and a 1/3 different hold-out set to verify model performance (n = 2467 with 91.3% willing to vaccinate).

For the machine learning models, categorical question and answer combinations from the survey and demographic variables, including gender, age, body mass index (BMI), race, income and education, were One-Hot encoded (i.e. converted to dummy variables). This resulted in a total of 85 features (S2 Table) that were used for the construction of the Random Forest. The model was tuned with respect to terminal node size and mtry (number of features available for splitting at each tree node), with class imbalance being addressed using stratified re-sampling for each of the 4001 trees. The most relevant features were determined using a recursive feature elimination approach. Starting with a model trained using the complete set of features, normalized permutation importance scores were determined and the bottom 4% of features were removed. The resulting model with all features (n = 85) or the recursively reduced models were evaluated using the hold-out dataset. The model with the minimum set of features was selected based on maintaining a high balanced accuracy on the hold-out dataset (i.e., the average of specificity and sensitivity), compared to the full model.

Shortly after the November survey was offered, Pfizer announced a first interim analysis reporting >90% efficacy of their vaccine candidate on November 9, 2020 [19]. We investigated if that news had an impact on willingness to vaccinate in our population and compared those who answered the survey before the 9th to those who answered after (excluding those who answered on the 9th itself, and those who answered on or after November 16th, the Moderna announcement [20]) using an ordered logistic regression model with four-level willingness to be vaccinated as the outcome and an indicator variable for the time period (before and after November 9th) and gender as predictors. Statistical analyses were performed using R (3.5.1 and 3.6.1) and r-packages randomForest (4.6–14) [18], caret (6.0–84) [21] and MASS (7.3–53) [22].

Results

Willingness to vaccinate against COVID-19

The assessment was offered to 9,106 participants with 7,621 (83.6%) completing the survey. Excluding participants (n = 219) missing demographic data left 7,402 participants (81.3%) for the study analyses (Table 1). Overall, 63.6% of participants reported they were very willing to receive a COVID-19 vaccine, 27.8% were somewhat willing, 6.0% were not very willing, and 2.6% were not at all willing. Treated as a dichotomous response, 91.3% were considered “willing” to be immunized while 8.7% were “unwilling” (Table 1).

Table 1

Demographics of the participants completing the vaccine assessment overall and stratified by vaccine willingness.

		Total	Unwilling to Receive Vaccine	Willing to Receive Vaccine
Subjects (N, %)		7402 (100%)	642 (8.7%)	6760 (91.3%)
Gender (N, %)	Women	3423 (46.2%)	414 (12.1%)	3009 (87.9%)
	Men	3979 (53.8%)	228 (5.7%)	3751 (94.3%)
Age (yr, mean ± SD)		70.8 ± 4.7	70.2 ± 4.4	70.9 ± 4.8
Age (N, %)	[65,70)	3509 (47.4%)	329 (9.4%)	3180 (90.6%)
	[70,75)	2378 (32.1%)	216 (9.1%)	2162 (90.9%)
	[75,80)	1093 (14.8%)	73 (6.7%)	1020 (93.3%)
	[80,85)	337 (4.6%)	18 (5.3%)	319 (94.7%)
	[85,90)	71 (1.0%)	6 (8.5%)	65 (91.5%)
	[90,95)	12 (0.2%	0 (0.0%)	12 (100%)
	[95,100)	2 (0.0%)	0 (0.0%)	2 (100%)
BMI (kg/m2, mean ± SD) †		27.5 ± 5.3	27.9 ± 5.6	27.4 ± 5.3
BMI (N, %)	Under weight (<18.5)	90 (1.2%)	4 (4.4%)	86 (95.6%)
	Normal weight (18.5–25)	2525 (34.1%)	209 (8.3%)	2316 (91.7%)
	Overweight (25.0–30)	2850 (38.5%)	229 (8.0%)	2621 (92.0%)
	Obese (> = 30)	1937 (26.2%)	200 (10.3%)	1737 (89.7%)
Race (N, %) ‡	American Indian or Alaska Native	21 (0.3%)	1 (4.8%)	20 (95.2%)
	Asian	238 (3.2%)	21 (8.8%)	217 (91.2%)
	Black or African American	183 (2.5%)	49 (26.8%)	134 (73.2%)
	Native Hawaiian or Other Pacific Islander	8 (0.1%)	3 (37.5%)	5 (62.5%)
	Two or more	44 (0.6%)	3 (6.8%)	41 (93.2%)
	White	6794 (91.8%)	544 (8.0%)	6250 (92.0%)
	Prefer not to answer	114 (1.5%)	21 (18.4%)	93 (81.6%)
Education (N, %)	Some high school or less	18 (0.2%)	3 (16.7%)	15 (83.3%)
	High school diploma or equivalent (GED)	280 (3.8%)	41 (14.6%)	239 (85.4%)
	Some college education	1104 (14.9%)	152 (13.8%)	952 (86.2%)
	Associate degree (e.g. AA, AS)	509 (6.9%)	66 (13.0%)	443 (87.0%)
	Bachelor’s degree (e.g. BA, BS)	2285 (30.9%)	173 (7.6%)	2112 (92.4%)
	Master’s degree (e.g. MA, MS, MEd)	2179 (29.4%)	148 (6.8%)	2031 (93.2%)
	Doctorate (e.g. PhD, EdD)	480 (6.5%)	29 (6.0%)	451 (94.0%)
	Professional degree (e.g. MD, DDS, DVM)	499 (6.7%)	23 (4.6%)	476 (95.4%)
	Prefer not to answer	48 (0.6%)	7 (14.6%)	41 (85.4%)
Income (N, %)	Under $30,000	356 (4.8%)	56 (15.7%)	300 (84.3%)
	$30,000-$39,999	308 (4.2%)	43 (14.0%)	265 (86.0%)
	$40,000-$49,999	343 (4.6%)	55 (16.0%)	288 (84.0%)
	$50,000-$59,999	442 (6.0%)	45 (10.2%)	397 (89.8%)
	$60,000-$74,999	747 (10.1%)	79 (10.6%)	668 (89.4%)
	$75,000-$99,999	1115 (15.1%)	78 (7.0%)	1037 (93.0%)
	$100,000-$149,999	1485 (20.1%)	85 (5.7%)	1400 (94.3%)
	$150,000-$200,000	688 (9.3%)	30 (4.4%)	658 (95.6%)
	Above $200,000	632 (8.5%)	26 (4.1%)	606 (95.9%)
	Prefer not to answer	1286 (17.4%)	145 (11.3%)	1141 (88.7%)

Plus-minus values are means ± standard deviations (SD). Overall column percentages represent % of overall sample (column percent). Percentages in willing and unwilling columns represent row %.

† The body-mass index is the weight in kilograms divided by the square of the height in meters.

‡ Race was reported by the participants, who could select more than one category.

Fig 1 shows results of univariable evaluations of demographic factors associated with vaccine willingness. Black or African American race was most strongly associated with decreased odds of vaccine willingness (odds ratio 0.24, 95% CI 0.18–0.31) (Fig 1). A total of 26.8% of Black or African American participants noted they were not very willing or not at all willing to vaccinate, compared with 8.0% of white participants (Table 1). Women were also less willing to be vaccinated as shown in Table 1 by 12.1% of women and 5.7% of men being not very or not at all willing to vaccinate against COVID-19 (odds ratio 0.49, 95% CI 0.45–0.54) (Fig 1). Income and education are also associated with willingness to be vaccinated, with higher income and higher education being associated with a higher willingness to be vaccinated (Fig 1).

Fig 1

Forest plot of willingness to vaccinate by demographics.

Shown are Odds Ratios (95% CI) for willingness to vaccinate for the different demographic characteristics. Odds Ratios were calculated using ordered logistic regression model with the 4 levels of willingness to be vaccinated as the outcome while adjusting for gender and race. Reference for each category is indicated by an open circle. na indicate not sufficient subjects for this category. ‘Native Hawaiian or Other Pacific Islander’, ‘Two or more’, and ‘Prefer not to answer’ are combined in ‘Other’.

Surveyed beliefs about COVID-19 and the COVID-19 vaccine (Fig 2), as well as beliefs about vaccines in general (S1 Fig), were found to be strongly associated with willingness to vaccinate. Regarding COVID-19, the most strongly associated beliefs included that the COVID-19 vaccine will help protect “myself and others” (odds ratio 38.6, 95% CI 32.4–46.1), the COVID-19 vaccine would be safe and effective (odds ratio 21.6, 95% CI 18.9–24.7), and being comfortable with short term side effects such as prolonged injection site pain (odds ratio 10.9, 95% CI 9.1–13.1). These beliefs were consistently important across participants who were White, Asian, African American or Black (S2 Fig).

Fig 2

Forest plot of willingness to vaccinate by survey response.

Shown are Odds Ratios (95% CI) for willingness to vaccinate for the different demographic characteristics. Odds Ratios were calculated using ordered logistic regression model with the 4 levels of willingness to be vaccinated as the outcome while adjusting for gender and race. Reference for each survey question is the option ‘neutral’ and is indicated by an open circle. For example subjects who agreed with the question ‘I am comfortable taking a COVID-19 vaccine that has short term side effects such as stomach pain or nausea if the vaccine efficiently prevents COVID-19.’ are 3.0 times more likely to be more willing as compared to those who selected ‘neutral’.

Information seeking

The vast majority of those who would be willing to vaccinate indicated they would talk to their healthcare provider (HCP) or staff before deciding whether or not to receive the vaccine (91.4% of women and 88.9% of men) (S3 Table). The majority of those who indicated they would not be willing to be vaccinated (68.4% of women and 62.3% of men) also indicated they would talk to their healthcare provider before deciding (S3 Table).

Specifically, we categorized the proportions of those who would talk to their HCP before deciding among those not at all willing and not very willing, by gender and race (Table 2). Of those not very willing, greater than 75% would talk to their HCP first (75.0% White, N = 376; 85.3% Black or African American, N = 34). Even among those not at all willing to vaccinate, around half would talk to their HCP before deciding (48.2% White, N = 168; 53.3% Black or African American, N = 15).

Table 2

Talking to Healthcare provider within levels of willing to vaccinate.

Race	Gender	Variable	Not at all willing	Not very willing	Somewhat willing	Very willing
Asian	Women	Talk with HCP (No)	0 (0.0%)	5 (45.5%)	9 (22.0%)	1 (2.0%)
		Talk with HCP (Yes)	1 (100%)	6 (54.5%)	32 (78.0%)	48 (98.0%)
	Men	Talk with HCP (No)	3 (100%)	5 (83.3%)	7 (18.9%)	17 (18.9%)
		Talk with HCP (Yes)	0 (0.0%)	1 (16.7%)	30 (81.1%)	73 (81.1%)
Black or African American	Women	Talk with HCP (No)	6 (46.2%)	2 (9.1%)	2 (4.2%)	1 (3.6%)
		Talk with HCP (Yes)	7 (53.8%)	20 (90.9%)	46 (95.8%)	27 (96.4%)
	Men	Talk with HCP (No)	1 (50.0%)	3 (25.0%)	2 (5.7%)	3 (13.0%)
		Talk with HCP (Yes)	1 (50.0%)	9 (75.0%)	33 (94.3%)	20 (87.0%)
White	Women	Talk with HCP (No)	56 (48.7%)	56 (23.8%)	83 (8.2%)	157 (9.0%)
		Talk with HCP (Yes)	59 (51.3%)	179 (76.2%)	930 (91.8%)	1596 (91.0%)
	Men	Talk with HCP (No)	31 (58.5%)	38 (27.0%)	72 (8.7%)	299 (11.3%)
		Talk with HCP (Yes)	22 (41.5%)	103 (73.0%)	755 (91.3%)	2358 (88.7%)

An overview of those who would talk to their Healthcare Provider (‘I would talk to my healthcare provider when considering a COVID-19 vaccine, before deciding whether or not to receive the vaccine’) among the 4 levels of willingness, by race and gender. Note: due to small numbers, Native Hawaiian or Other Pacific Islander (n = 8), American Indian or Alaska Native (n = 21), Two or more (n = 44) and Prefer not to Answer (n = 114) were excluded from the table.

Participant characteristics and beliefs indicative of vaccine willingness

To identify a set of determinants that separate those who are willing and unwilling to be vaccinated, we constructed a Random Forest classification algorithm. From the survey and the available self-reported demographic data, we extracted a list of features (S2 and S3 Tables) to be used in the first model. This initial model using all 85 features resulted in 90.2% balanced accuracy (average of 90.7% sensitivity and 89.7% specificity) when applied to the hold-out dataset. When testing the recursively reduced models, the balanced accuracy remained near constant up to the model with 9 remaining features (89.5% balanced accuracy with 87.4% sensitivity and 91.6% specificity). A further reduction (removing the least important feature from the set of 9) resulted in a 12.3 percent point reduction in balanced accuracy primarily due to misclassification of the not willing to vaccinate (Specificity = 55.6%).

The features coming out of our model (Fig 3) revealed five main indicators: belief whether the COVID-19 vaccine would be safe and effective, belief whether a COVID-19 vaccine would help protect “myself and others,” degree of comfort with potential short term side effects from a COVID-19 vaccine, the belief whether vaccines in general are safe and important, and whether the respondent was Black or African American.

Fig 3

Result of recursive feature elimination algorithms.

Random Forest classification algorithm was constructed to identify a set of determinants able to separate those who are not willing to vaccinate from those who are. The model started with a list of 85 features and predicted the willingness of subjects in the hold-out dataset with 90.2% balanced accuracy (solid line), which is an average of 90.7% sensitivity (dashed line closed circles) and 89.7% specificity (dashed line open circles). The balanced accuracy remained near constant when testing the recursively reduced models, up to the model with 9 remaining features (i.e. 5 questions with a total of 9 answers, see inserted table). This final model showed an 89.5% balanced accuracy with 87.4% sensitivity and 91.6% specificity. Further reduction, removing the least important feature from the set of 9 (i.e. ‘Neutral’ to ‘Once approved, I believe a COVID-19 vaccine will help protect myself and others’), resulted in a 12.3 percent point reduction in balanced accuracy primarily due to misclassification of the not willing to vaccinate (Specificity = 55.6%).

Impact of positive vaccine news

Willingness to vaccinate increased after November 9th when results of the Pfizer Phase 3 vaccine trial were released (odds ratio 1.41, 95% CI 1.21–1.65; S4 Table). Prior to that date, 2.8% of adults were not at all willing to receive a COVID-19 vaccine and 6.3% were “not very” willing; after this these decreased to 1.4% and 5.9%, respectively. No meaningful difference was found in gender, age, BMI, education, income or race between the population who completed the survey before the Pfizer announcement on November 9th and those completing the survey after, between November 10th and 15th.

Notably, Moderna also released positive preliminary data regarding their vaccine on November 16th [20]. By this date 95% (7065 out of the 7,402) of the respondents had already completed the survey. To test if a similar effect of the Moderna news was present, surveys completed after the Pfizer announcement (November 9) and before the Moderna announcement (November 16) were compared to those completed after that announcement (November 17–20). A similar effect was observed (odds ratio of 1.27, 95%CI 0.91–1.80), though not statistically significant due to small sample sizes.

Discussion

The recent approval of vaccines for COVID-19 has increased the focus on the need to maximize public vaccine acceptance. Vaccine uptake in older adults is of particular importance as increasing age is the leading risk factor for mortality and complications from COVID-19 infections [15, 16]. In this large survey of adults age 65 and older conducted in the United States in November 2020, the vast majority (91%) of adults reported that they are willing to receive a COVID-19 vaccine. However, even in this population of people with sufficient health literacy and trust in the healthcare system to participate in a digital clinical trial, 1 in 11 reported unwillingness to receive a vaccine. More concerningly, rates of vaccine unwillingness were higher in Black or African American adults and those at lower income and education levels, suggesting the potential for uneven vaccine uptake in at-risk communities.

In this study, Black or African American race was the only factor associated with willingness to immunize after accounting for beliefs about the vaccine; 1 in 4 Black or African American participants reported unwillingness to receive a vaccine. This is slightly lower than previous reports of ~40% or more unwilling [8, 9, 11, 23, 24], but is consistent with other studies that show that willingness among Black or African Americans is improving [10]. We also found lower rates of reported vaccine willingness in women compared with men, a factor that has been shown in other studies [8, 9, 11, 14, 23]. Socioeconomic factors, including higher education and higher income, were also associated with increased reported willingness to be vaccinated, similar to other studies [9, 11]. Notably, these findings contrast with characteristics of adults who refuse vaccines for their children. In studies of vaccine exemptions, higher income and education are often associated with higher rates of vaccine refusal [25]. Given that lower income communities and communities of color are at higher risk of COVID-19, vaccine hesitancy in these groups is of particular concern. Even if overall vaccine uptake is high, clustering of unimmunized persons can lead to continued circulation of vaccine-preventable diseases [26].

The strongest factors associated with and indicative of vaccine willingness in this population were beliefs about the COVID-19 vaccine’s safety and efficacy, and a more altruistic belief whether a COVID-19 vaccine will help protect “myself and others." Vaccine safety and efficacy are among the most important factors reported influencing likeliness to vaccinate among Americans [9, 27] and Americans 65 and over [11]. Among those disinclined to vaccinate, the majority stated that they would discuss their decision with their healthcare provider, providing an important opportunity for education. Despite differences in rates of vaccine willingness by gender and race, the associations between beliefs about a COVID-19 vaccine and reported willingness to receive a vaccine were consistent across subgroups. In addition to broader public education campaigns about COVID-19, specific efforts should be made to facilitate healthcare patient-provider communication about COVID-19 vaccine focused on vaccine safety and efficacy.

While the study was not designed to determine the impact of vaccine-related news, the release of results from Pfizer’s Phase 3 vaccine trial occurring in the middle of the survey [19] provided an opportunity to evaluate the impact of positive vaccine news on vaccine attitudes. Increased vaccine willingness was seen after the trial results were released, suggesting that public willingness to vaccinate may continue to rise as additional positive data are released. These data are supported by a survey experiment demonstrating those who were shown hypothetical messaging about the safety/efficacy of a vaccine were more likely, compared to a control group, to report they would take the vaccine [27]. On the other hand, given the potential for vaccine related data to shift perception, there remains the possibility that negative news stories about the vaccine (including reports of adverse events) may negatively impact vaccine willingness.

The high willingness among participants in our population to be vaccinated is similar to that reported in other assessments in older adults, though higher than the general population. Recently reported results from the Kaiser Family Foundation found that 85% of those 65 and older were willing to receive a vaccine [10]. Older adults consistently express a higher willingness to accept a COVID-19 vaccine than younger populations [3, 5, 6, 8, 11, 14]. Reasons for this may include a higher perceived risk of COVID-19 illness compared to younger adults [28], prior experience with vaccine preventable diseases and mass vaccination campaigns for diseases such as polio, or comfort with routine immunizations due to being recommended for influenza vaccine. In contrast, increasing age (over 50 or 60) does not appear to be a significant factor to predict willingness to engage in other preventive measures for COVID-19 including willingness to isolate [29]. In fact, certain behaviors known to be effective against COVID-19 spread, like mask wearing, were actually negatively correlated with age in that study (through age 80 years) [29]. In addition to the reasons mentioned above, other possible explanations for higher willingness to vaccinate in our study could include a high level of trust of healthcare providers, who are generally strongly pro-vaccination, and a selection bias from those willing and eligible to participate in the HeartlineTM Study towards more openness to vaccination.

This study demonstrates the power of the digital platform used in the HeartlineTM Study to rapidly generate real-world data. There are key features of this platform that enable rapid data generation: a properly constructed informed consent form that permits ad hoc survey deployment for data collection; a mobile app capable of pushing content and gathering variable but structured data; a back-end data structure that enables rapid analysis; and an app design that keeps participants highly engaged throughout the entire study, as evidenced by 83.6% of participants who took the survey. This platform opens the possibility of studies that investigate multiple effects over time or serial interventions in a population.

These findings should be interpreted in the context of several potential limitations and the implications of this study’s findings should be understood in the context of the particular population recruited into the HeartlineTM Study. There is an inherent bias that comes with studying populations who have chosen to enroll into a clinical study and where recruitment is done through a mobile App platform. Bias could have been introduced in HeartlineTM, such as selection of those with access to iPhones, which are premium mobile hardware, those familiar with app-based technologies, and those who were willing to share extensive amounts of their health information digitally. In addition, those with interest in mobile applications related to their health may be healthier and/or more health-conscious, and therefore may be more likely to vaccinate. HeartlineTM participants have less representation of Black or African Americans (2.5% in this study vs 9% of 65 and older U.S. population) [30] and Asians (3.2% in this study vs 4% of 65 and older U.S population) [30] compared to the general U.S. 65+ population and skew higher in education and income, which may have led to an overestimate of vaccine acceptance compared with the general population. Nevertheless, finding that 1 in 11 patients in this selected group are unwilling to be vaccinated may portend even lower rates in a broader population. Next, our analysis of the impact of vaccine-related news occurred during a time period when infections continued to rise; whether changes in attitudes were due to vaccine news or due to other factors is unclear. We also can not rule out that differences in vaccine willingness may be due to differences in the populations who completed the surveys at different time periods, though we are reassured that the demographic characteristics of the population did not appear to differ temporally. Finally, we asked about vaccine willingness, which may not directly translate into behavior, particularly if attitudes shift over time. We also highlight several strengths, including a high response rate (>80%), a short period of data collection (2 weeks), and the ability to deploy the survey rapidly in response to the pandemic.

In a 65 and older U.S. population, most are willing to be vaccinated, with Black or African American participants and females significantly less willing. The majority would be willing to discuss their concerns with their providers, who could leverage the beliefs identified here in tailoring a message to encourage vaccination. Potential health care policy implications from these findings include using providers as a key leverage point and ensure appropriate resources for them to reach and educate this older, at risk population to address vaccine hesitancy. In addition, policies to start vaccinations with older populations first could result in faster adoption and make vaccination more acceptable to those who prefer to ‘wait and see’ [7]. Developing implementable strategies to consistently communicate the potential benefits of vaccination, including their safety and efficacy, could improve acceptance and help speed the efforts to thwart this global pandemic.

Forest plot of willingness to vaccinate by survey response (general vaccine beliefs).

Shown are Odds Ratios (95% CI) for willingness to vaccinate. Odds Ratios were calculated using ordered logistic regression model with the 4 levels of willingness to be vaccinated as the outcome while adjusting for gender and race.

(PDF)

Click here for additional data file.

Forest plots of willingness to vaccinate by COVID-19 beliefs and race.

Shown are Odds Ratios (95% CI) for willingness to vaccinate for the survey by race: White (A), Asian (B), and Black (C). Odds Ratios were calculated using ordered logistic regression model with the 4 levels of willingness to be vaccinated as the outcome while adjusting for gender and race. Reference for each survey question is the option ‘neutral’ and is indicated by an open circle. na indicates insufficient subjects for this category.

(PDF)

Click here for additional data file.

Vaccine hesitancy assessment.

(DOCX)

Click here for additional data file.

Recursive feature elimination algorithm feature definitions.

(DOCX)

Click here for additional data file.

Willingness to be vaccinated by model features from survey.

(DOCX)

Click here for additional data file.

Impact of positive vaccine efficacy news by level of willing to vaccinate.

November 9, 2020, Pfizer announced a first interim analysis reporting >90% efficacy of their vaccine candidate. The impact of this news on willingness was evaluated among those who answered the survey before the 9th to those who answered thereafter (excluding those who answered on the 9th itself). Those after November 9th were more willing to vaccinate (odds ratio 1.41, 95%CI 1.21–1.65).

(DOCX)

Click here for additional data file.

17 Feb 2021

PONE-D-21-00518

Factors indicating intention to vaccinate with a COVID-19 vaccine among older U.S. Adults

PLOS ONE

Dear Dr. Navar,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Both reviewers agree that the paper is well written and addresses a relevant and timely question. However, they also raise a number of concerns and provide comments and suggestions that, if properly addressed, will help to further improve the quality of the paper. I also outline my comments below.

First, both reviewers believe that more should be done to connect your research with the literature. They provide useful references, and you will certainly find recent work very relevant to the topic. Moreover, R2 is less satisfied than R1 regarding the importance of the research, and I encourage you to address this consideration. One way to do it could be to provide more information about the “high risk” that you refer to on page 4, which increases the relevance of improving our knowledge about elderly people and their responses to the crisis.

Second, and it is a crucial point: the reviewers raise a number of concerns with the timing. I see three issues. As mentioned by R1, you should address the news around the Moderna’s vaccine. R2 details some concerns about the representativeness of the sample in regard of the ‘treatment’ (i.e pre/post Pfizer news). Finally, I would also like you to address the possibility that the treatment (or the effect of the news) might have potential heterogenous effects across subsamples (for examples, groups that are less likely to trust scientists).

Third, on top of the representativeness in the pre/post-Pfizer news samples, both reviewers raise fair questions about the whole sample with clear comments related to respondents’ income, the opt-in panel from an iPhone app, the external validity, etc.

See also the other concerns outlined quite clearly by the reviewers below.

Please submit your revised manuscript by Mar 14 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Jean-François Daoust

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.Thank you for including your ethics statement:

"clinicaltrials.gov, NCT04276441

Approved by WIRB. Study number 1260314. IRB Tracking Number 20191385.

All study participants provided informed consent to analyze de-identified data, including surveys delivered through the HeartlineTM app. ".

a) Please amend your current ethics statement to include the full name of the ethics committee/institutional review board(s) that approved your specific study.

b)Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

Once you have amended this/these statement(s) in the Methods section of the manuscript, please add the same text to the “Ethics Statement” field of the submission form (via “Edit Submission”).

For additional information about PLOS ONE ethical requirements for human subjects research, please refer to http://journals.plos.org/plosone/s/submission-guidelines#loc-human-subjects-research.

3.We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

4. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

5.Thank you for stating the following in the Competing Interests section:

"I have read the journal's policy and the authors of this manuscript have the following competing interests:

JN, MK, GJW, JW, TS, JS, PB, RZ, NE, JT, RJ are employees of Janssen and Johnson and Johnson.

JS reports personal fees from Amgen, personal fees from Bayer, personal fees from Merck, personal fees from Novartis, personal fees from Janssen, personal fees from Myokardia, personal fees from Blue Cross Blue Shield of Kansas City,  outside the submitted work;  In addition, Dr. Spertus has a patent Copyright to the KCCQ with royalties paid and Equity in Health Outcomes Sciences.

MT reports grants from Janssen Inc, personal fees from Medtronic Inc, personal fees from Abbott, grants from Boehringer lngelheim, grants and personal fees from Cardiva Medical, personal fees from iRhythm, grants from Bristol Myers Squibb, grants from American Heart Association, grants from SentreHeart, personal fees from Novartis, personal fees from Biotronik, personal fees from Sanofi, personal fees from Pfizer, grants from Apple, grants from Bayer, personal fees from Myokardia, personal fees from Johnson & Johnson, personal fees from Milestone Pharmaceuticals, outside the submitted work; and Dr. Turakhia is an editor for JAMA Cardiology.

LS reports personal fees from J&J, outside the submitted work.

MG receives research grant support from Janssen.

AMN receives consulting fees from Janssen for serving on the steering committee of the Heartline study.  In addition, Dr. Navar receives funding for research to her institution from Amgen and Janssen, and honoraria and consulting fees from Amarin, Amgen, Astra Zeneca, BI, Esperion, Lilly, Sanofi, Regeneron, NovoNordisk, Novartis, The Medicines Company, New Amsterdam, Cerner, 89Bio, and Pfizer, outside the scope of this work."

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This piece reviews the demographic and attitudinal correlates of intentions to refuse vaccination for COVID-19, among older US adults (aged 65+) . The authors find that Black respondents and women are significantly more likely to intend to refuse vaccination than the 65+ population more generally. The authors also find that concerns about vaccine safety and efficacy were correlated with vaccination intentions, that most individuals who intend to refuse vaccination would nevertheless be willing to consult with a medical professional prior to doing so, and that intentions to vaccinate significantly increased following news of Pfizer's positive Stage 3 clinical results.

In general, I found this paper to be well written, substantively important, and methodologically rigorous. However, while I find the piece suitable for publication in a journal like this one, there are several areas where I believe the authors can offer additional conceptual and analytical detail. I have listed each of those concerns in the order in which I encountered each one in the manuscript. I also suggest several actions the authors might take to remedy those concerns, when revising this manuscript.

1. p. 4 (final paragraph): The authors do an excellent job motivating why it is important to study vaccination intentions among older Americans -- i.e., because they are particularly at risk of experiencing severe complications from contracting COVID-19.

However, I think the authors could do much more to put their research into dialogue with other work on the demographic correlates of COVID-19 vaccine refusal, as well as individuals' reasons for planning to refuse to vaccinate. For example, Callaghan et al (2021) report a nearly identical pattern of demographic correlates of vaccine hesitancy (i.e., that women and Black respondents are less likely to intend to vaccinate) as well as individuals' reasons for intending to not vaccinate (e.g., concerns about safety and efficacy).

Why might we expect to observe a similar (or, different?) pattern of results among older Americans? What do we learn about either the nature of COVID-19 vaccine hesitancy, or its potential health policy implications, from studying this group? I encourage the authors to consider expanding on this point when discussing the importance and novelty of their research.

2. p. 7, (penultimate paragraph) and pp. 15-16 (final Results sub-section): While I found the results of the pre/post Pfizer Clinical Trial news to be both interesting and informative, I have two major conceptual and methodological concerns about this analysis.

A. First, the study's enrollment period (11/6-20, 2020) included not only the release of Pfizer's clinical trial data (11/9), but Moderna's as well (11/16). It was unclear to me why the authors analyze the former, but not the latter, when estimating the effects of attention to positive vaccine-related news. (see: https://www.nytimes.com/interactive/2020/science/coronavirus-vaccine-tracker.html)

If the authors have the ability to replicate this analysis for the Moderna news, I encourage them to do so. If not (e.g., due to statistical power concerns), I encourage them to instead state what those concerns are.

B. I am also somewhat concerned about the estimation of news "treatment" effects in the pre vs. post Pfizer announcement groups. Because this study is correlational, it could be the case that the group of respondents interviewed later on in the survey administration process was systematically different from the pre-treatment group on demographic on other factors associated with vaccine uptake. For example, if men were more likely to take the survey later on in the administration process than women, the appearance of a news treatment effect could actually be the result of methodological artifact: i.e., a gender effect.

Although the authors lack the ability to test these claims longitudinally (which would allow for the estimation of *within* subject treatment effects; potentially alleviating the concerns mentioned above), the authors could instead address this concern by showing that the pre and post treatment subgroups were balanced with respect to (1) demographic characteristics and (2) reasons provided for intending/not intending to vaccinate. This would help ensure that attention to the vaccine-related news, and not differences between the subsamples, is responsible for the effects documented in the piece.

3. p. 8 (Table 1, row 1): the authors observe much lower levels of vaccine hesitancy among older survey respondents than has been typically observed in existing research on COVID-19 vaccine hesitancy (e.g., see the Callaghan et al., piece referenced in #1). It is somewhat at odds with recent findings (e.g., Daoust 2020) suggesting that older and younger people tend to have similar attitudinal and behavioral responses to the pandemic.

If the authors had a priori expectations regarding why older individuals might be more likely to intend to vaccinate (e.g., increased risk of experiencing severe health complications), I encourage them to expand on that rationale. If not, I invite them to speculate, perhaps in the piece's discussion section, as to why this might be the case. Either way, I think the authors should make more of an effort to put these findings into conversation with types of research cited above; i.e., to note that older folks may be exceptional in their intentions to vaccinate, and that the effect of age on vaccination intentions may differ from the effects of age in other attitudinally and behaviorally relevant domains.

4. p. 19: the authors do an excellent job recognizing the limitations of collecting responses from an opt-in sample of mobile health app users. In addition to the concerns listed already in the document, I encourage the authors to discuss potential limits of the mobile app platform *itself* as a recruitment mechanism for this study. In addition to demographic differences in mobile phone access and representation in clinical trial enrollment, individuals who choose to download and regularly consult phone applications related to their health may be healthier and/or more health-conscious than individuals who do not.

Consequently, while observing higher-than-typical rates of vaccine compliance among older individuals could reflect an age cohort effect (see: point #3 above), it could alternatively reflect differences in who chooses to use the platform (vs. the remainder of the 65+ population). I.e., it could be the case individuals enrolled in this study and who regularly consult the app are more concerned about their health, and therefore more likely to vaccinate; or that they are more attuned to the possibility of experiencing adverse vaccination effects?

I encourage the authors to expand on this point in the piece's discussion. If possible, comparing the demographic characteristics of the 65+ sample to known subpopulation demographic benchmarks (e.g., from the US Census) would help further underscore the representativeness of this sample.

Minor points:

- The authors refer to study participants as "Adults" in the piece's abstract. I think it would be more informative to list the study's target population here; i.e., US Adults aged 65 or older.

- On p.5, the authors note that the study was approved by "an IRB." Which IRB, specifically, approved this research? I encourage the authors to provide additional information here.

References

Callaghan, Timothy, Ali Moghtaderi, Jennifer A. Lueck, Peter J. Hotez, Ulrich Strych, Avi Dor, Erika Franklin Fowler, and Matt Motta. "Correlates and disparities of COVID-19 vaccine hesitancy." Available at SSRN 3667971 (2020).

Daoust, J. F. (2020). Elderly people and responses to COVID-19 in 27 Countries. PloS one, 15(7), e0235590.

Reviewer #2: Thank you for the opportunity to review "Factors Indicating Intention to Vaccinate with a COVID-19 Vaccine among Older U.S. Adults" for possible publication in PLOS One. This article relies on an original survey sample of adults enrolled in the Heartline clinical study to examine vaccination intentions among U.S. adults over the age of 65. The authors find several factors influence the decision to vaccinate, include race, gender, and beliefs about the safety and efficacy of COVID-19. While this paper is written on a timely and vital topic, a few issues need to be resolved before publication.

My first concern with the manuscript in its current form is the generalizability of the findings. I am concerned the findings shown here may not be representative of the U.S. population. For example, examining Table 1 shows that 92% of the sample identifies as White and over 50% of the survey has income over $75,000. While I believe it is important to study the vaccine intentions of this vulnerable population, I don't believe the sample the authors use are representative of the older population in the U.S. The authors do make reference to the limitations of their study in the discussion but I believe the authors need to be more cautious about the inferences readers should take from the article.

My second concern with the manuscript is I would have liked to see more literature on vaccine hesitancy, specifically related to COVID-19, given all the recent literature done on the issue. I would encourage the authors to look at articles from Callaghan et al.; Motta et al. (just to name a few). I also would have liked to see more framing up front about the importance of studying this vulnerable population.

Thank you again for the opportunity to review this research letter. With these key changes, I believe this manuscript will make a valuable addition to our understanding of the factors influencing individual decisions to vaccinate against COVID-19.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Matt Motta

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

8 Apr 2021

Reviewer #1: This piece reviews the demographic and attitudinal correlates of intentions to refuse vaccination for COVID-19, among older US adults (aged 65+) . The authors find that Black respondents and women are significantly more likely to intend to refuse vaccination than the 65+ population more generally. The authors also find that concerns about vaccine safety and efficacy were correlated with vaccination intentions, that most individuals who intend to refuse vaccination would nevertheless be willing to consult with a medical professional prior to doing so, and that intentions to vaccinate significantly increased following news of Pfizer's positive Stage 3 clinical results.

In general, I found this paper to be well written, substantively important, and methodologically rigorous. However, while I find the piece suitable for publication in a journal like this one, there are several areas where I believe the authors can offer additional conceptual and analytical detail. I have listed each of those concerns in the order in which I encountered each one in the manuscript. I also suggest several actions the authors might take to remedy those concerns, when revising this manuscript.

1. p. 4 (final paragraph): The authors do an excellent job motivating why it is important to study vaccination intentions among older Americans -- i.e., because they are particularly at risk of experiencing severe complications from contracting COVID-19.

However, I think the authors could do much more to put their research into dialogue with other work on the demographic correlates of COVID-19 vaccine refusal, as well as individuals' reasons for planning to refuse to vaccinate. For example, Callaghan et al (2021) report a nearly identical pattern of demographic correlates of vaccine hesitancy (i.e., that women and Black respondents are less likely to intend to vaccinate) as well as individuals' reasons for intending to not vaccinate (e.g., concerns about safety and efficacy).

We greatly appreciate the specific reference and agree with the reviewer. Callaghan et al (2021) indeed supports our findings nicely and has now been added in several places in our manuscript. Additional references have also been added to further support our discussion regarding demographic correlates of vaccine hesitancy.

p.18: “Vaccine safety and efficacy are among the most important factors reported influencing likeliness to vaccinate among Americans (Callaghan, Palm) and Americans 65 and over (Malani).”

Why might we expect to observe a similar (or, different?) pattern of results among older Americans? What do we learn about either the nature of COVID-19 vaccine hesitancy, or its potential health policy implications, from studying this group? I encourage the authors to consider expanding on this point when discussing the importance and novelty of their research.

While we weren’t surprised to see lower levels of vaccine hesitancy in older Americans as others have reported similar findings, one of our main findings was that this older population trusts healthcare providers, which is also consistent with prior work (https://www.pewresearch.org/science/2019/08/02/findings-at-a-glance-medical-doctors/). We found that providers are the predominant source of information about vaccines for this population (vs. the internet or social media). The primary health policy implication is that providers are a key leverage point and need to be appropriately resourced to reach and educate this older, at risk population to address vaccine hesitancy. In addition, policies to start vaccinations with older populations first could result in faster adoption and make it more acceptable to those who prefer to ‘wait and see how it goes for others first’ (Talev Axios-Ipsos poll).

We have added further discussion regarding the nature of vaccine hesitancy and health policy implications in the discussion.

p.18: “Vaccine safety and efficacy are among the most important factors reported influencing likeliness to vaccinate among Americans (Callaghan, Palm) and Americans 65 and over (Malani).”

p.21: “Potential health care policy implications from these findings include using providers as a key leverage point and ensure appropriate resources for them to reach and educate this older, at risk population to address vaccine hesitancy. In addition, policies to start vaccinations with older populations first could result in faster adoption and make vaccination more acceptable to those who prefer to ‘wait and see’ [Talev].”

2. p. 7, (penultimate paragraph) and pp. 15-16 (final Results sub-section): While I found the results of the pre/post Pfizer Clinical Trial news to be both interesting and informative, I have two major conceptual and methodological concerns about this analysis.

A. First, the study's enrollment period (11/6-20, 2020) included not only the release of Pfizer's clinical trial data (11/9), but Moderna's as well (11/16). It was unclear to me why the authors analyze the former, but not the latter, when estimating the effects of attention to positive vaccine-related news. (see: https://www.nytimes.com/interactive/2020/science/coronavirus-vaccine-tracker.html)

If the authors have the ability to replicate this analysis for the Moderna news, I encourage them to do so. If not (e.g., due to statistical power concerns), I encourage them to instead state what those concerns are.

The reviewer is correct: the reason this was not in our initial analysis was due to lack of statistical power. Moderna released their first preliminary data on November 16th, during the tail end of our survey. By then, we already had 95% of participants complete the survey, with only 337 completions at and after the 16th. To prevent contamination of that second positive news story on the inference regarding the Pfizer timing, we have revised our analysis to exclude the small number of surveys that were completed after the Moderna news was released on November 16th.

We now compare surveys completed before the Pfizer announcement (November 6-8) to those completed after that announcement (November 10-15), excluding any that came in after the Moderna announcement on the 16th. This analysis maintained a significant odds ratio of 1.41 (1.21-1.65). The original analysis gave an odds ratio of 1.46 (1.28-1.67). This has been updated in the Results on p.16.

While underpowered, we did perform an exploratory analysis regarding the Moderna timing. To test if a similar effect of the Moderna news was present, surveys completed after the Pfizer announcement (November 9) and before the Moderna announcement (November 16) were compared to those completed after the Moderna announcement (November 17-20). A similar effect was observed (odds ratio of 1.27, 95%CI 0.91 – 1.80), though not statistically significant due to small sample sizes.

We have added the below comment to the manuscript on p.7-8 and have updated Table S4 with the new analysis.

p.7-8: “Shortly after the November survey was offered, Pfizer announced a first interim analysis reporting >90% efficacy of their vaccine candidate on November 9, 2020 [16]. We investigated if that news had an impact on willingness to vaccinate in our population and compared those who answered the survey before the 9th to those who answered after (excluding those who answered on the 9th itself, and those who answered on or after November 16th, the Moderna announcement) using an ordered logistic regression model with four-level willingness to be vaccinated as the outcome and an indicator variable for the time period (before and after November 9th) and gender as predictors.”

B. I am also somewhat concerned about the estimation of news "treatment" effects in the pre vs. post Pfizer announcement groups. Because this study is correlational, it could be the case that the group of respondents interviewed later on in the survey administration process was systematically different from the pre-treatment group on demographic on other factors associated with vaccine uptake. For example, if men were more likely to take the survey later on in the administration process than women, the appearance of a news treatment effect could actually be the result of methodological artifact: i.e., a gender effect.

Although the authors lack the ability to test these claims longitudinally (which would allow for the estimation of *within* subject treatment effects; potentially alleviating the concerns mentioned above), the authors could instead address this concern by showing that the pre and post treatment subgroups were balanced with respect to (1) demographic characteristics and (2) reasons provided for intending/not intending to vaccinate. This would help ensure that attention to the vaccine-related news, and not differences between the subsamples, is responsible for the effects documented in the piece.

The reviewer makes good points and we investigated further. We compared gender, age, BMI, education, income and race in the populations who completed the survey prior to Pfizer’s announcement to those completing after (but prior to Moderna’s announcement). We found no meaningful difference in any of those characteristics between the two populations.

We did not evaluate reasons for intention to vaccinate or trust in providers before in after the news as we feel these belief variables may be impacted by the positive news itself.

The text below was added to the Results on p.16.

p.16: “No meaningful difference was found in gender, age, BMI, education, income or race between the population who completed the survey before the Pfizer announcement on November 9th and those completing the survey after, between November 10th and 15th. Notably, Moderna also released positive preliminary data regarding their vaccine on November 16th[ref]. By this date 95% (7065 out of the 7,402) of the respondents had already completed the survey. To test if a similar effect of the Moderna news was present, surveys completed after the Pfizer announcement (November 9) and before the Moderna announcement (November 16) were compared to those completed after that announcement (November 17-20). A similar effect was observed (odds ratio of 1.27, 95%CI 0.91 – 1.80), though not statistically significant due to small sample sizes.”

We have also recognized this as a limitation:

p.20-21: “We also cannot rule out that differences in vaccine willingness may be due to differences in the populations who completed the surveys at different time periods, though we are reassured that the demographic characteristics of the population did not appear to differ temporally.”

3. p. 8 (Table 1, row 1): the authors observe much lower levels of vaccine hesitancy among older survey respondents than has been typically observed in existing research on COVID-19 vaccine hesitancy (e.g., see the Callaghan et al., piece referenced in #1). It is somewhat at odds with recent findings (e.g., Daoust 2020) suggesting that older and younger people tend to have similar attitudinal and behavioral responses to the pandemic.

If the authors had a priori expectations regarding why older individuals might be more likely to intend to vaccinate (e.g., increased risk of experiencing severe health complications), I encourage them to expand on that rationale. If not, I invite them to speculate, perhaps in the piece's discussion section, as to why this might be the case. Either way, I think the authors should make more of an effort to put these findings into conversation with types of research cited above; i.e., to note that older folks may be exceptional in their intentions to vaccinate, and that the effect of age on vaccination intentions may differ from the effects of age in other attitudinally and behaviorally relevant domains.

As we note in the discussion, our high rate of vaccine willingness was similar to what has been seen in older adults in other studies. We have added a comment to note that this is higher than the general population.

p.19: “The high willingness among participants in our population to be vaccinated is similar to that reported in other assessments in older adults, though higher than the general population.”

We appreciate the Daoust 2020 reference which reports that when it comes to attitudes and compliance to different COVID-19 preventative measures, like willingness to isolate, advanced age (over 50 or 60) does not appear to be a significant factor, and indeed older and younger people tend to have similar attitudinal and behavioral responses to the pandemic. This is interesting given that age is a significant factor in COVID-19 mortality and certain behaviors known to be effective against COVID-19 spread, like mask wearing, were actually negatively correlated with age in that publication (through age 80 years). This has been added to the manuscript in the Discussion, p.19.

p.19: “In contrast, increasing age (over 50 or 60) does not appear to be a significant factor to predict willingness to engage in other preventative measures for COVID-19 including willingness to isolate (Daoust). In fact, certain behaviors known to be effective against COVID-19 spread, like mask wearing, were actually negatively correlated with age in that study (through age 80 years) (Daoust). In addition to the reasons mentioned above, other possible explanations for higher willingness to vaccinate in our study could include a high level of trust of healthcare providers, who are generally strongly pro-vaccination, and a selection bias from those willing and eligible to participate in the HeartlineTM Study towards more openness to vaccination.”

4. p. 19: the authors do an excellent job recognizing the limitations of collecting responses from an opt-in sample of mobile health app users. In addition to the concerns listed already in the document, I encourage the authors to discuss potential limits of the mobile app platform *itself* as a recruitment mechanism for this study. In addition to demographic differences in mobile phone access and representation in clinical trial enrollment, individuals who choose to download and regularly consult phone applications related to their health may be healthier and/or more health-conscious than individuals who do not.

Consequently, while observing higher-than-typical rates of vaccine compliance among older individuals could reflect an age cohort effect (see: point #3 above), it could alternatively reflect differences in who chooses to use the platform (vs. the remainder of the 65+ population). I.e., it could be the case individuals enrolled in this study and who regularly consult the app are more concerned about their health, and therefore more likely to vaccinate; or that they are more attuned to the possibility of experiencing adverse vaccination effects?

I encourage the authors to expand on this point in the piece's discussion. If possible, comparing the demographic characteristics of the 65+ sample to known subpopulation demographic benchmarks (e.g., from the US Census) would help further underscore the representativeness of this sample.

We agree with the reviewer’s comments and have added more detail to the limitations discussed on p. 20. We agree, as in any recruitment that is channel-specific – in our case, via mobile technologies and apps – bias can be introduced, such as selection of those with access to iPhones, which are premium mobile hardware; further selection bias of those familiar with app-based technologies; and selection of those willing to share extensive amounts of their health information digitally. Indeed, our recruited population may be healthier and/or more health conscious and more likely to vaccinate.

p.20: “These findings should be interpreted in the context of several potential limitations and the implications of this study’s findings should be understood in the context of the particular population recruited into the HeartlineTM Study. There is an inherent bias that comes with studying populations who have chosen to enroll into a clinical study and where recruitment is done through a mobile App platform. Bias could have been introduced in HeartlineTM, such as selection of those with access to iPhones, which are premium mobile hardware, those familiar with app-based technologies, and those who were willing to share extensive amounts of their health information digitally. In addition, those with interest in mobile applications related to their health may be healthier and/or more health-conscious, and therefore may be more likely to vaccinate.”

We have also drawn attention to the representativeness of our population related to Race, by comparing to the general U.S. 65+ population, now noting on p.20 , “…Black or African Americans (2.5% in this study vs 9% of 65 and older U.S. population) and Asians (3.2% in this study vs 4% of 65 and older U.S population)…”

Minor points:

- The authors refer to study participants as "Adults" in the piece's abstract. I think it would be more informative to list the study's target population here; i.e., US Adults aged 65 or older.

We agree and have made that change. “U.S. adults aged 65 and older enrolled in the HeartlineTM clinical study…”

- On p.5, the authors note that the study was approved by "an IRB." Which IRB, specifically, approved this research? I encourage the authors to provide additional information here.

We agree and have added that detail to the Methods, p.5: “The study protocol was approved by Western Institutional Review Board and all participants submit a signed informed consent to analyze de-identified data through the HeartlineTM app.”

References

- Callaghan, Timothy, Ali Moghtaderi, Jennifer A. Lueck, Peter J. Hotez, Ulrich Strych, Avi Dor, Erika Franklin Fowler, and Matt Motta. "Correlates and disparities of COVID-19 vaccine hesitancy." Available at SSRN 3667971 (2020).

- Daoust, J. F. (2020). Elderly people and responses to COVID-19 in 27 Countries. PloS one, 15(7), e0235590.

Reviewer #2: Thank you for the opportunity to review "Factors Indicating Intention to Vaccinate with a COVID-19 Vaccine among Older U.S. Adults" for possible publication in PLOS One. This article relies on an original survey sample of adults enrolled in the Heartline clinical study to examine vaccination intentions among U.S. adults over the age of 65. The authors find several factors influence the decision to vaccinate, include race, gender, and beliefs about the safety and efficacy of COVID-19. While this paper is written on a timely and vital topic, a few issues need to be resolved before publication.

My first concern with the manuscript in its current form is the generalizability of the findings. I am concerned the findings shown here may not be representative of the U.S. population. For example, examining Table 1 shows that 92% of the sample identifies as White and over 50% of the survey has income over $75,000. While I believe it is important to study the vaccine intentions of this vulnerable population, I don't believe the sample the authors use are representative of the older population in the U.S. The authors do make reference to the limitations of their study in the discussion but I believe the authors need to be more cautious about the inferences readers should take from the article.

We agree with the reviewer’s comments and have added language to qualify our findings better, now noting on p.20, “These findings should be interpreted in the context of several potential limitations and the implications of this study’s findings should be understood in the context of the particular population recruited into the HeartlineTM Study.”

We added more detail to the limitations discussed on p.20. We agree, as in any recruitment that is channel-specific – in our case, via mobile technologies and apps – bias can be introduced, such as selection of those with access to iPhones, which are premium mobile hardware; further selection bias of those familiar with app-based technologies; and selection of those willing to share extensive amounts of their health information digitally. Indeed, our recruited population may be healthier and/or more health conscious and more likely to vaccinate.

p.20: “These findings should be interpreted in the context of several potential limitations and the implications of this study’s findings should be understood in the context of the particular population recruited into the HeartlineTM Study. There is an inherent bias that comes with studying populations who have chosen to enroll into a clinical study and where recruitment is done through a mobile App platform. Bias could have been introduced in HeartlineTM, such as selection of those with access to iPhones, which are premium mobile hardware, those familiar with app-based technologies, and those who were willing to share extensive amounts of their health information digitally. In addition, those with interest in mobile applications related to their health may be healthier and/or more health-conscious, and therefore may be more likely to vaccinate.”

We have also drawn attention to the representativeness of our population related to Race, by comparing to the general U.S. 65+ population, now noting on p.19 , “…Black or African Americans (2.5% in this study vs 9% of 65 and older U.S. population) and Asians (3.2% in this study vs 4% of 65 and older U.S population)…”

My second concern with the manuscript is I would have liked to see more literature on vaccine hesitancy, specifically related to COVID-19, given all the recent literature done on the issue. I would encourage the authors to look at articles from Callaghan et al.; Motta et al. (just to name a few). I also would have liked to see more framing up front about the importance of studying this vulnerable population.

We agree and have updated our literature search and added several new references to the manuscript, including those mentioned.

p.19: “In contrast, increasing age (over 50 or 60) does not appear to be a significant factor to predict willingness to engage in other preventative measures for COVID-19 including willingness to isolate (Daoust). In fact, certain behaviors known to be effective against COVID-19 spread, like mask wearing, were actually negatively correlated with age in that study (through age 80 years) (Daoust). In addition to the reasons mentioned above, other possible explanations for higher willingness to vaccinate in our study could include a high level of trust of healthcare providers, who are generally strongly pro-vaccination, and a selection bias from those willing and eligible to participate in the HeartlineTM Study towards more openness to vaccination.”

p.17: “Vaccine safety and efficacy are among the most important factors reported influencing likeliness to vaccinate among Americans (Callaghan, Palm) and Americans 65 and over (Malani).”

p.18: “These data are supported by a survey experiment demonstrating those who were shown hypothetical messaging about the safety/efficacy of a vaccine were more likely, compared to a control group, to report they would take the vaccine (Palm).”

• Malani PN, Solway E, Kullgren JT. Older Adults’ Perspectives on a COVID-19 Vaccine. JAMA Health Forum. Published online December 23, 2020. doi:10.1001/jamahealthforum.2020.1539

• Daoust J-F (2020) Elderly people and responses to COVID-19 in 27 Countries. PLoS ONE 15(7): e0235590. https://doi.org/10.1371/journal.pone.0235590

• Palm R, Bolsen T, Kingsland JT. The effect of frames on COVID-19 vaccine hesitancy. January 6, 2021. https://doi.org/10.1101/2021.01.04.21249241

• Callaghan, Timothy, Ali Moghtaderi, Jennifer A. Lueck, Peter J. Hotez, Ulrich Strych, Avi Dor, Erika Franklin Fowler, and Matt Motta. "Correlates and disparities of COVID-19 vaccine hesitancy." Available at SSRN 3667971 (2020).

We also added more discussion in the Introduction about the importance of studying this vulnerable population.

p.4: “Vaccine uptake in older adults is of particular importance as increasing age is the leading risk factor for mortality and complications from COVID-19 infections [13,19]. Guidelines have been developed that identify priority populations for vaccination (CDC.gov), with recommendations including those older than 65 years.“

p.4: “We hypothesized that older adults in our sample population would have a high willingness to vaccinate and sought to understand factors associated with those less willing.”

Thank you again for the opportunity to review this research letter. With these key changes, I believe this manuscript will make a valuable addition to our understanding of the factors influencing individual decisions to vaccinate against COVID-19.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

7 May 2021

Factors indicating intention to vaccinate with a COVID-19 vaccine among older U.S. Adults

PONE-D-21-00518R1

Dear Dr. Navar,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Jean-François Daoust

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Congratulations!

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I thank the authors for addressing both my conceptual and methodological concerns about this manuscript. I especially appreciated the inclusion of revised/additional analyses regarding the Moderna vaccine news, and the assessment of demographic balance across conditions. I found the updated manuscript to be significantly improved, and I now recommend publication.

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Matt Motta

Reviewer #2: No

11 May 2021

PONE-D-21-00518R1

Factors indicating intention to vaccinate with a COVID-19 vaccine among older U.S. Adults

Dear Dr. Navar:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jean-François Daoust

Academic Editor

PLOS ONE