Prospective associations of regional social media messages with attitudes and actual vaccination: A big data and survey study of the influenza vaccine in the United States.

OBJECTIVE: Using longitudinal methods to assess regional associations between social media posts about vaccines and attitudes and actual vaccination against influenza in the US.
METHODS: Geolocated tweets from U.S. counties (N = 115,330) were analyzed using MALLET LDA (Latent Dirichlet allocation) topic modeling techniques to correlate with prospective individual survey data (N = 3005) about vaccine attitudes, actual vaccination, and real-life discussions about vaccines with family and friends during the 2018-2019 influenza season.
RESULTS: Ten topics were common across U.S. counties during the 2018-2019 influenza season. In the overall analyses, two of these topics (i.e., Vaccine Science Matters and Big Pharma) were associated with attitudes and behaviors. The topic concerning vaccine science in November-February was positively correlated with attitudes in February-March, r = 0.09, BF10 = 3. Moreover, among respondents who did not discuss the influenza vaccine with family and friends, the topic about vaccine fraud and children in November-February was negatively correlated with attitudes in February-March and with vaccination in February-March, and April-May (rs = -0.18 to -0.25, BF10 = 4-146). However, this was absent when participants had discussions about the influenza vaccine with family and friends. DISCUSSION: Regional vaccine content correlated with prospective measures of vaccine attitudes and actual vaccination.
CONCLUSIONS: Social media have demonstrated strong associations with vaccination patterns. When the associations are negative, discussions with family and friends appear to eliminate them. Programs to promote vaccination should encourage real-life conversations about vaccines.

Introduction

Although the influenza vaccine is one of the greatest achievements of public health and each year prevents millions of illnesses and thousands of deaths [1], vaccine messages expressing skepticism about vaccine safety and describing fake patient stories abound on social media [2], [3]. Important research has advanced our understanding of social media contents, including how Russian linked-accounts have discussed vaccines in an attempt to influence the American public [4]. However, the ultimate associations between these contents and vaccination behavior remains an underexplored problem that we investigated in this study. To address this question, we analyzed regional variability in social media messages in the US in combination with longitudinal survey measures from a representative sample of the U.S. adult population.

Regional associations between social media messages and attitudes and health behavior

As an example, even though more than a hundred thousand people had died of SAR-CoV-2 by the time this paper was written, and a vaccine may not be available until 2021, tweets opposing an eventual vaccine against the SARS-CoV-2 virus already populate the cyberworld. An example of these appears below.

“To each their own. I believe in medical freedom, not medical tyranny. If you want to get the flu shot, go ahead. But it shouldn’t be mandatory. Same with any medicine or vaccine. I personally would never take a Coronavirus vaccine. I know these sneaky snakes want it mandatory!” [5].

Other tweets, of course, support vaccination, such as the following example:

“The problem is when you make that choice [vaccine vs. chip] for yourself, you’re also potentially making it for other people who are unable to get vaccinated. Polio, measles, diphtheria, etc are not things anyone should suffer through. Especially not when we have means of preventing them.” [6]

A growing number of important studies have analyzed vaccine content on Facebook, Twitter, and YouTube. Although some studies have involved manual coding of posts [7], [8], [9], others have used computational techniques such as co-occurrence network analysis [10], and yet others have combined manual coding with computational methods, such as a support vector machine [11]. A recent study of anti- and pro-vaccine clips on YouTube identified words such as “chemical,” “mercury,” and “toxic” in anti-vaccine content and “hospital,” “chronic,” and “unvaccinated” in pro-vaccine material [9]. Similarly, an analysis of tweets showed factors containing words such as “cdcwhistleblower” and “coverup” posted by anti-vaccine users and topics containing “risk,” “health,” “chronic,” and “science” posted by pro-vaccine users [11]. These studies have demonstrated that linguistic analyses can identify alternative types of vaccine content on social media. Yet, most of these studies covered only content obtained by means of pre-determined keywords. More exploratory methods of unsupervised learning (e.g., topic modeling) may provide further insights into vaccine social media contents [12].

The psychological theory has shed light on the nature of the relations between health messages and attitudes, as well as correspondence between attitudes and behaviors themselves [13], [14], [15], [16], [17], [18], [19], [20], [21], [22]. Specifically, health contents can include information for people to act on, and include both persuasive posts by health officials as well as the personal opinions of lay people [18], [23], [24], [25]. Health messages can then influence attitudes, which in turn guide people’s decisions about what to do, including whether or not to vaccinate against the disease [14], [25], [26]. That is, people holding favorable attitudes toward the influenza vaccine are more likely to get vaccinated than those with unfavorable attitudes [27]. Recently, the relations between social media posts and vaccine attitudes and actual vaccination have also begun to be explored.

Prior studies have obtained cross-sectional evidence of relations between the use of or exposure to social media and vaccination status [28], [29]. According to research on individuals, people who use either Twitter or Facebook, or both, as sources of health information are more likely to have received the influenza vaccine than are people who do not [28]. Moreover, across the US states, how common HPV vaccine topics are on Twitter accounts for 68% of the variance in the proportion of females receiving at least one dose of the HPV vaccine [29]. Although each of these studies offers useful evidence, both were cross-sectional, and the regional study [29] lacked individual-level data, which limits its conclusions and introduces the possibility of the ecological fallacy [30], [31]. Thus, longitudinal data and a combination of individual- and regional-level data are necessary to understand the relations between social media messages posted within a region and individual attitudes and behaviors in that region.

The role of discussions with others as another influence source

Social media posts questioning the need for vaccines, for instance, may lead people to form negative attitudes toward vaccination and not vaccinate [32], [33], [34]. However, social media do not exist in a vacuum. Tweets posted in a region can be highly relevant to the local context [35], and their impact may also depend on discussions of the influenza vaccine with family and friends in real life (for similar work in other areas, see [36]). In this regard, although the CDC encourages people to talk to friends and family about the influenza vaccine [37], the effectiveness of discussing vaccines with them is unclear. On the one hand, discussions with family and friends may further vaccine misinformation and hesitancy [2], [38]. For example, people seek and circulate health information among family and friends [39], and this creates an information bubble (i.e., reading news or information that supports what people already believe in) that can pose challenges for public health announcements and physician recommendations to get through [40].

On the other hand, discussing vaccines with family and friends may correct inaccurate information found on social media. For example, discussing vaccination with family and friends predicts college students’ positive dispositions toward vaccination [41], and norms can have positive influences on actual vaccination as well [42]. Moreover, discussions within one’s network have been shown to reduce bias [43]. For example, Guilbeault and Centola [43] examined the idea of “networked collective intelligence” by assessing the impact of tobacco-prevention messages exchanged within the network. Participants who did and did not smoke were randomly assigned to either an experimental condition that allowed participants to have discussions with their networks or a control condition in which participants evaluated the health message by themselves. The results showed that smokers who discussed the message with others evaluated the health message more favorably than did smokers who evaluated the message alone. In short, introducing more diverse contents through discussions with family friends can be useful in correcting misinformation [44], [45].

The present study

This study attempted to contribute to (a) provide a longitudinal assessment of the regional effects of social media messages on vaccine attitudes and vaccination, (b) incorporate individual-level data to minimize the threat of the ecological fallacy, and (c) examine the role of discussions with others as a factor that may strengthen or weaken the influences of social media. The guiding research questions included: What are the regional associations between social media messages and vaccine attitudes and vaccination? And how does discussing vaccines with family and friends influence the relation of social media content and vaccination? We used Twitter and the survey data corresponding to the 2018–2019 influenza season to examine these questions.

Methods

Source of data

The proposed study combined individual-level survey data with county-level tweets about vaccines, which were geolocated. The archival survey data involved a probability-based, nationally representative sample of American adults and assessed attitudes concerning the influenza vaccine, actual vaccination, and real-life discussions of the influenza vaccine in a five-wave panel (between September 21, 2018 and October 6, 2018; between October 7, 2018 and November 28, 2018; between November 29, 2018 and February 5, 2019; between February 6, 2019 and March 18, 2019; and between April 30, 2019 and May 5, 2019). In that flu season, most influenza vaccination occurred by December in 2018 although some were taken as late as early May in 2019. We collected both topically relevant tweets from the US and their metadata between September 2017 and March 2019 using the Crimson Hexagon API. This social media analysis platform offers full access to all public tweets (i.e., Twitter Firehose). The survey data and the Twitter data are described next.

Survey data

Design and methods

We used five waves of a large panel survey assessing attitudes, knowledge, and health behaviors concerning infectious diseases and vaccines. The archival data included a probability-based, nationally representative sample of American adults who were randomly selected from AmeriSpeak panels of NORC at the University of Chicago in September-October and followed in October-November, November-February, February-March, and April-May (i.e., the 2018–2019 influenza season). This sampling frame covered 97% of U.S. households via a two-stage stratified sampling [46]. Respondents primarily completed the survey online, although a small percentage answered the survey over the phone (landlines and cell phones) to avoid a bias favoring people who use the Internet. This panel is the only probability panel that is available for hybrid online/phone administration in the US. After the data collection in September-October, weighted distributions of age, sex, race, and income showed only trivial differences compared to unweighted distributions. However, there was a discrepancy in education, with more of the survey sample having a college degree than the 2018 U.S. Census estimates. To mitigate this problem, respondents with a lower level of educational attainment were successfully targeted for retention in following waves with more incentives and reminders.

Measures

We analyzed responses to questions about vaccine attitudes, vaccination, and real-life discussions of the influenza vaccine. Respondents answered the questions about receiving the influenza vaccine in all five waves, whereas the questions about attitudes and real-life discussions were asked only in the first four waves of the study.

Vaccination

We used answers to the question, “Have you gotten the flu vaccine this season or not?” to indicate respondents’ influenza vaccination status. Answers were provided as “Yes” or “No.”

Vaccine attitudes

The survey included four attitude questions with 4-point scales: (a) “Based on what you know, how positive or negative do you feel about the flu vaccine?” (1: Very positive, 2: somewhat positive, 3: somewhat negative, 4: very negative); (b) “Just your best guess, how risky, if at all, do you think the flu vaccine is?” (1: Very risky, 2: somewhat risky, 3: not too risky, 4: not risky at all) (c) “Just your best guess, please indicate how effective, if at all, you think the flu vaccine will be at preventing the flu among those who get the vaccine this season?” (1: Very effective, 2: somewhat effective, 3: not too effective, 4: not effective at all), and (d) “How important, if at all, do you think it is that most people in your community get the flu vaccine?” (1: Very important, 2: somewhat important, 3: not too important, 4: not important at all). We reverse-scored all items except item (b), so a higher score indicates more favorable vaccine attitudes. The Cronbach’s alphas of the responses for all four waves were about 0.81–0.85, suggesting good internal consistencies. Therefore, we calculated a mean of these questions to represent respondents’ attitudes regarding the influenza vaccine in each wave.

Discussing the influenza vaccine with family and friends

Using a 4-point scale, participants also answered the question: “In the past month, how frequently have you discussed the flu vaccine with friends and family? (1: None, 2: rarely, 3: sometimes, 4: often). We rescored the responses of this question into a binary variable (i.e., 1: rarely, sometimes, or often as with discussions, and 0: none as without discussions of the influenza vaccine with family and friends) for the moderator analyses.

Tweets

Twitter continues to be one of the most popular social media platforms [47] and, unlike Facebook, provides open data access and has more precise and frequent location information. In this study, we collected Twitter data using Crimson Hexagon API. We adopted a top-down approach to collect relevant tweets by searching keywords related to vaccines posted between September 2017 and August 2018 to be the training data. We used additional keywords concerning influenza and the influenza vaccine with the keywords used for the training data to collect tweets between September 2018 and March 2019 to be the test data. In total, we collected 628,597 tweets and geotagged about 18% of the collected tweets for the analyses (see Table S1). This percentage is similar to the geotagging rates reported in previous studies [48], [49].

We developed lists of keywords about influenza and flu by reviewing relevant literature [50], [51], [52] and consulted a team of health communication researchers to identify all keywords and hashtags relevant to the study period. Keywords used to collect the training data: (a) “vaccineswork” or “getvaccinated” or “gettheshotnottheflu” or “whyivax” or “vaccinessavelives” or “vaxyourkids” or “vaxyourself” or “fightflu” or “stoptheoutbreaks” or “vaxwithme”) and (b) ((“vaccine” or “vax”) and (“cdcwhistleblower” or “hearthiswell” or “bigpharma” or “b1less” or “nomandates” or “antivax” or “vaccineinjury” or “vaccineskill” or “vaccinesharm” or “thimerosal”)). Additional keywords used to prepare the test data: (“flu” or “influenza”) and (“vaccine” or “vaccination” or “vaccinate” or “shot” or “nasal” or “spray” or “headache” or “cough” or “sick” or “fever” or “nose” or “throat” or “ill” or “outbreak” or “pandemic” or “jab” or “infected” or “infection” or “sneezing” or “temperature” or “thermometer” or “lung” or “pneumonia” or “recovered” or “cured” or “flufact” or “flufighter” or “fluseason” or “flushot” or “nivw” or “vaxxed” or “vaccineawareness” or “fluvax”).

Analytical procedure

We began with the geotagging procedure of tweets to US counties and applied topic modeling techniques to identify vaccine-related words that co-occurred (i.e., “topics”). We followed Maier et al.’s [53] recommended approach to select a final topic model and then computed each topic’s probabilities for U.S. counties during the 2018–2019 influenza season (see Fig. 1 ). Next, we combined topic probabilities with the survey data and performed correlations to examine the regional association of social media messages with vaccine attitudes and actual vaccination. Finally, we conducted the moderator analyses to assess if real-life discussions with friends and family affected the associations, and if so, what that influence was. Technical details about each step along with the topic modeling implementation follow.

Fig. 1

Processing of Twitter Messages and Calculation of Topic Probabilities in Each Wave.

Geotagging

Tweets, including retweets, are informative about popular topics and conversations within a community. Prior studies have shown that county-level tweets can provide signals for predicting HIV/STI infections [49], [54] and are associated with attitudes, knowledge, and behaviors of infectious diseases such as Zika across U.S. counties [55]. Therefore, we decided to use county as the unit of analysis. We used the timestamp metadata to exclude tweets that do not originate from U.S. time zones. For each tweet, the user profile location and each tweet’s precise latitude and longitude coordinates were used to map tweets to U.S. counties using Geographic Information Systems (GIS) database operations (see [49] for details of the geotagging process). Tweets from the same county were organized into a single document, which constituted the unit of analysis. Thus, the tweets could not be used to re-identify individual users, and no personal information was included in the analyses. Details about the training and test data appear in Table S1.

Topic modeling

We adapted Maier et al.’s [53] recommended approach to select a valid and reliable topic model, including (a) appropriate pre-processing of the text collection, (b) adequate selection of model parameters (i.e., the optimal number of topics of the topic model [K], alpha, and beta), and (c) evaluation of the model’s reliability. Before analysis, we removed unnecessary words and characters from the text corpus to retain only meaningful words and formed bi-grams (i.e., two-word phrases, see supplementary materials for details of the pre-processing step). Messages in each county were analyzed as a single document, and the words from all tweets from each county were combined into one single word-by-frequency matrix (see Fig. 1).

We used the Python package gensim to convert a collection of documents (i.e., each county’s tweets) into a frequency matrix of token counts. The matrix of token counts was then analyzed using a well-established algorithm in computer science, namely Latent Dirichlet Allocation [56]. LDA is a Bayesian mixture model that groups words that often appear together to create topics (see Fig. 1). Then, we used a Python wrapper for LDA by implementing MALLET [57], and calculated coherence and log perplexity scores to determine the optimal number of topics (i.e., K) and hyperparameters (i.e., alpha and beta, see supplementary materials for details). This process led to a topic model with ten topics (i.e., K = 10) and another with twenty topics (i.e., K = 20).

Next, we used the codes provided by Maier et al. (2018) to evaluate the reliability of two LDA topic models by conducting 33 experiments. Fig. 3 shows the reliability and coherence scores of the experiments, and a higher score indicates higher reliability and semantic coherence of the identified topics. As shown in Fig. 2 , the topic model with a K of 10 had significantly higher reliability and semantic coherence than the topic model with a K of 20. Therefore, the selected topic model had ten topics, random initialization, and 1000 iterations.

Fig. 3

Reliability and Semantic Coherence Results of Two Topic Models (Ks = 10 and 20). Note. Results were based on different initializations techniques (random = default random initialization; seed = fixed seed initialization; and cluster = semantic co-occurrence network initialization) and varying number of inference iterations (0–2000); K = number of topics (10 and 20).

Fig. 2

Word Clouds of Topics.

Finally, we computed the probability of each topic for each county in each wave, p(topic, county) = ∑ p(topic | word) × p(word | county), where p(word | county) represents the normalized proportion of words in a county document, and p(topic | word) represents the probability of the topic given that word, using tweets that correspond to the survey period. We also calculated the standardized topic probabilities per county (i.e., the sum of all topic probabilities per county is equal to 1) by replacing unpresentable topic probabilities with “0.00001”. The topic probabilities in each wave were then matched with the individual-level survey data by county and analyzed with Bayesian correlation analyses.

Statistical procedure

Intra-class correlation

The survey data involve individual participants who live in U.S. counties, and the Twitter topics belong to counties. Such a structure requires researchers to first determine the optimal way of analyzing the data, particularly whether participants should be nested within counties or whether the variance in survey responses can be assumed to vary as a function of participants. Hence, we calculated intra-class correlations (ICCs) to assess the overall variation in vaccination and vaccine attitudes explained by U.S. counties. Values of ICC can range from 0 to 1, and a value close to zero indicates that the observations within clusters are no more similar than observations from different clusters [58]. Values of ICC > 0.20 is used as evidence of the need to use multi-level modeling, which in our case would imply modeling participants and counties as separate levels. Values above 0.20 would imply that participants must be modeled as nested within counties. We used the performance package [59] in R to calculate ICCs with vaccination and vaccine attitudes as the outcomes for all waves (see Table S3).

Bayesian correlation

We next used an R package BayesMed to calculate the Bayes factors for correlations between topics and attitudes or vaccination using a default uniform prior (i.e., 1) in which the possible correlation between −1 and +1 is equally plausible [60]. A Bayes factor (i.e., BF10) between 3.2 and 10 provides substantial evidence that the data support the alternative hypothesis of a relation between two variables at least 3.2 to 10 times more than the null hypothesis of the absence of association [61].

Results

Identification of topics

Our selected topic model appears in Table 1 . Two of the authors named these ten topics: Vaccines Protect Children, Immunization Contributes to Global Health, Doctors Vaccinating Children, Fighting the Flu during the Flu Season, Vaccine Science Matters, Negative Vaccine Consequences, Vaccine Fraud and Children, Big Pharma, Anti-Vaccine Story, and Prevention Disease. Disagreements about names were resolved by means of discussion. Table 1 shows the top ten words of each topic, and Fig. 2 shows word clouds with the top 30 words of each topic. These topics were then used in correlation analyses to determine associations with our survey measures of attitudes and vaccination for each wave.

Table 1

The final topic model for flu-related tweets in the U.S.

Topic number	Label	Top-10 word
World Challenges
1	Vaccines Protect Children	measle, vaccination, disease, child, protect, immunization, health, year, world, kid
2	Immunization Contributes to Global Health	make, health, datum, global, great, immunization, learn, work, globalhealth, read
Vaccine Administration
5	Doctors Vaccinating Children	child, vaccinate, measle, doctor, science, kid, make, hpv, parent, give
9	Fighting the Flu during the Flu Season	fightflu*, season, shoot, protect, learn, getvaccinate*, today, vaccination, people, late
Vaccine Information
8	Vaccine Science Matters	vaxwithme*, ivax*, protect, anti, high, cancer, claim, evidence, sciencematter*, movement
Vaccine Concerns
0	Negative Vaccine Consequences	vaxwithme*, vaxxed*, vaxxe, injury, ppl, shot, autism, safe, kill, explain
3	Vaccine Fraud and Children	vaccinate, child, kid, worldwide, call, late, commit, improving_access, validate, share
6	Big Pharma	health, child, bigpharma*, vaccinate, flushot, damage, autism, murder, share, alarm
7	Anti-Vaccine Story	autism, family, antivaccine, story, infection, battle, bad, fund, movement, st_place
Infections and Diseases Prevention
4	Prevention Disease	whyivax*, share, adult, info, preventshingle*, patient, loved_one, change, follow, virus

Note. Asterisks indicate hashtags, “_” indicate bi-grams.

Associations of attitudes and vaccination with topics

The main goal of the study was to combine the topics in Table 1 with the survey data. This required assembling a data frame in which participants are nested within counties in each wave. Then, this data frame was combined with the topics, which are designated with probabilities for each county. We first describe the waves and demographics of the survey. We then describe the hierarchical analyses of our data frame, and last, we describe the correlational and moderator analyses.

Description of the survey waves and demographics

The sample sizes of the multiple-wave survey ranged from 1591 to 3005 (N in September-October = 3005, N in October-November = 2470, N in November-February = 2091, N in February-March = 1803, and N in April-May = 1591). The descriptive characteristics of the sample in September-October were as follows: About 63% Non-Hispanic White, 13% Black, 6% Asian, 14% Non-White Hispanic, 1% Other, and 3% Mixed. The mean age was 48.4 years old (SD = 18), with a median education level of an associate degree and a median annual household income level at $50,000–$59,999.

Modeling decisions and results

As explained before, when data are hierarchical, it is first necessary to determine if all levels contribute to the variability observed at the individual level. This decision is important to know if, for example, differences across counties are due to differences in the individuals who live there or to aspects of the counties, such as policies, driving attitudes, or behaviors. We computed ICCs in all waves and found very low levels of variability attributable to the county in which participants lived (ICCs < 0.05; see Table S3). This suggested that the associations of attitudes and vaccination with the topics could be assessed at the individual level. Therefore, we used single-level Bayesian correlation analyses in the following analyses.

Our first research question concerned the relations between the topics in Fig. 2 and vaccine attitudes and actual vaccination longitudinally (see Table 2 ). We conducted Bayesian correlations between these topics and attitudes and vaccination, and the results showed moderate evidence of a prospective correlation with vaccine attitudes. Specifically, Vaccine Science Matters in November-February was positively correlated with attitudes in February-March, r = 0.09, BF10 = 3.22. As for vaccination status, the results showed no evidence of the presence of associations with topics.

Table 2

Bayesian correlation results (r, Bayes Factors10) between Twitter topics and vaccine attitudes and vaccination in all waves.

Topic probability	September–October		October–November		November–February		February–March		April–May
	N = 1591–1662		N = 1591–1662		N = 1591–1662		N = 1591–1662		N = 881–1091
	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccination
September-October
Negative Vaccine Consequences	0.03, 0	−0.03, 0	0.02, 0	0, 0	0.01, 0	−0.01, 0	0.01, 0	0, 0	−0.01, 0
Vaccines Protect Children	−0.01, 0	0.01, 0	−0.04, 0	−0.04, 0	−0.01, 0	0, 0	0.01, 0	0.02, 0	0.03, 0
Immunization Contributes to Global Health	0.01, 0	0.06, 1	0, 0	−0.01, 0	−0.02, 0	−0.02, 0	−0.03, 0	−0.02, 0	−0.04, 0
Vaccine Fraud and Children	0.01, 0	−0.02, 0	0.02, 0	0.02, 0	0, 0	0.03, 0	0.03, 0	0.04, 0	0.03, 0
Prevention Disease	0.05, 0	−0.04, 0	0.04, 0	0.04, 0	0.03, 0	0.02, 0	0.01, 0	0, 0	−0.01, 0
Doctors Vaccinating Children	−0.05, 0	0, 0	−0.02, 0	−0.05, 0	−0.05, 0	−0.07, 1	−0.03, 0	−0.06, 0	−0.05, 0
Big Pharma	−0.01, 0	0.05, 0	0, 0	0, 0	0, 0	0.03, 0	−0.03, 0	0, 0	0.01, 0
Anti-Vaccine Story	0, 0	−0.03, 0	−0.03, 0	0.03, 0	−0.01, 0	0.02, 0	−0.02, 0	0.01, 0	0.01, 0
Vaccine Science Matters	0, 0	0.02, 0	0.01, 0	0.01, 0	0.01, 0	0.01, 0	0, 0	0.01, 0	0.01, 0
Fighting the Flu during the Flu Season	0.02, 0	0.01, 0	0.04, 0	0.07, 1	0.07, 1	0.07, 1	0.08, 1	0.08, 1	0.08, 1
October-November
Negative Vaccine Consequences			0.03, 0	0.02, 0	0.04, 0	0.06, 0	0.02, 0	0.05, 0	0.05, 0
Vaccines Protect Children			0, 0	−0.04, 0	0.01, 0	0, 0	0, 0	0, 0	−0.01, 0
Immunization Contributes to Global Health			−0.02, 0	−0.01, 0	−0.02, 0	−0.03, 0	−0.04, 0	−0.04, 0	−0.06, 0
Vaccine Fraud and Children			0.05, 0	0.05, 0	0.07, 0	0.06, 0	0.08, 0	0.08, 0	0.08, 0
Prevention Disease			−0.01, 0	0.01, 0	0, 0	−0.02, 0	0, 0	−0.01, 0	−0.02, 0
Doctors Vaccinating Children			0.03, 0	0.06, 0	0.05, 0	0.08, 0	0.08, 0	0.06, 0	0.07, 0
Big Pharma			0.02, 0	−0.02, 0	0.02, 0	−0.02, 0	0.03, 0	0.02, 0	0.02, 0
Anti-Vaccine Story			−0.02, 0	0.01, 0	−0.02, 0	−0.03, 0	−0.02, 0	−0.04, 0	−0.05, 0
Vaccine Science Matters			0, 0	−0.03, 0	−0.01, 0	−0.05, 0	−0.04, 0	−0.06, 0	−0.04, 0
Fighting the Flu during the Flu Season			−0.04, 0	−0.03, 0	−0.07, 0	−0.05, 0	−0.09, 0	−0.06, 0	−0.05, 0
November–February
Negative Vaccine Consequences					−0.06, 0	−0.05, 0	−0.05, 0	−0.06, 0	−0.06, 0
Vaccines Protect Children					0.04, 0	0, 0	0.05, 0	0, 0	0, 0
Immunization Contributes to Global Health					0.03, 0	0.04, 0	0.01, 0	0.04, 0	0.05, 0
Vaccine Fraud and Children					−0.06, 1	−0.06, 1	−0.05, 0	−0.07, 1	−0.06, 0
Prevention Disease					0.01, 0	0.01, 0	0.02, 0	0.01, 0	0.01, 0
Doctors Vaccinating Children					−0.01, 0	0.01, 0	−0.04, 0	−0.01, 0	−0.02, 0
Big Pharma					0.03, 0	−0.01, 0	0.05, 0	0, 0	0.01, 0
Anti-Vaccine Story					−0.03, 0	−0.02, 0	−0.04, 0	−0.03, 0	−0.04, 0
Vaccine Science Matters					0.06, 1	0.01, 0	0.09, 3*	0.02, 0	0.01, 0
Fighting the Flu during the Flu Season					−0.01, 0	0.03, 0	−0.01, 0	0.05, 0	0.05, 0
February–March
Negative Vaccine Consequences							−0.02, 0	−0.03, 0	−0.01, 0
Vaccines Protect Children							0.05, 0	0.05, 0	0.05, 0
Immunization Contributes to Global Health							0.05, 0	0.05, 0	0.05, 0
Vaccine Fraud and Children							0.01, 0	−0.01, 0	0.01, 0
Prevention Disease							−0.07, 0	−0.04, 0	−0.01, 0
Doctors Vaccinating Children							0.01, 0	−0.02, 0	−0.02, 0
Big Pharma							−0.1, 6*	−0.06, 0	−0.05, 0
Anti-Vaccine Story							−0.02, 0	−0.06, 0	−0.06, 0
Vaccine Science Matters							−0.01, 0	0.02, 0	0.01, 0
Fighting the Flu during the Flu Season							0.01, 0	0.04, 0	0.02, 0

* BF10 > 3.2, ** BF10 > 10, *** BF10 > 32, **** BF10 > 100.

Moderator analyses of discussing the influenza vaccine with friends and family

Next, we conducted moderator analyses to examine whether discussions with friends and family affected the correlation paths, and if so, what that influence was. We repeated the correlation analyses for respondents reporting real-life discussions of the influenza vaccine and those reporting no real-life discussions of the influenza vaccine, respectively (see Table 3 ). For respondents with no discussions with family and friends, we observed negative associations between Vaccine Fraud and Children in November-February and attitudes in February-March, r = −0.18, BF10 = 4. Vaccine Fraud and Children in November-February was also inversely associated with vaccination in February-March and April-May, rs = −0.24 to −0.25, BF10 = 49–146. Additionally, the results showed no evidence of any relations in respondents having real-life influenza vaccine discussions.

Table 3

Bayesian correlation results (r, Bayes Factors10) between Twitter topics and vaccine attitudes and vaccination for two subgroups.

Topic probability	September–October		October–November		November–February		February–March		April–May
	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccine attitude	Vaccination	Vaccination
Subgroup analyses: Having no discussions of the influenza vaccine with family and friends
September–October
Negative Vaccine Consequences	0.02, 0	−0.06, 0	0, 0	−0.03, 0	0, 0	−0.04, 0	−0.04, 0	−0.03, 0	−0.03, 0
Vaccines Protect Children	−0.01, 0	−0.02, 0	−0.04, 0	−0.04, 0	0, 0	0.02, 0	−0.03, 0	0.01, 0	0.01, 0
Immunization Contributes to Global Health	0.01, 0	0.1, 1	−0.03, 0	0, 0	0.01, 0	−0.01, 0	−0.02, 0	−0.01, 0	−0.03, 0
Vaccine Fraud and Children	−0.02, 0	−0.01, 0	0.01, 0	−0.01, 0	−0.01, 0	0.01, 0	0.13, 0	0.06, 0	0.05, 0
Prevention Disease	0.02, 0	−0.05, 0	0, 0	−0.03, 0	−0.04, 0	−0.05, 0	−0.07, 0	−0.08, 0	−0.08, 0
Doctors Vaccinating Children	0.01, 0	0.08, 0	0.06, 0	0.04, 0	0.02, 0	0.01, 0	0.05, 0	0.02, 0	0.02, 0
Big Pharma	−0.06, 0	−0.01, 0	−0.01, 0	0.01, 0	−0.02, 0	−0.01, 0	0, 0	0, 0	0, 0
Anti-Vaccine Story	0.06, 0	0, 0	−0.04, 0	0.08, 0	0, 0	0.08, 0	0.03, 0	0.07, 0	0.08, 0
Vaccine Science Matters	0, 0	−0.02, 0	0.06, 0	0.02, 0	0.06, 0	0.04, 0	0.08, 0	0.07, 0	0.05, 0
Fighting the Flu during the Flu Season	−0.03, 0	−0.01, 0	−0.03, 0	0, 0	0.03, 0	0.03, 0	0.05, 0	0.03, 0	0.05, 0
October–November
Negative Vaccine Consequences			0.05, 0	0, 0	0.02, 0	−0.03, 0	−0.05, 0	−0.02, 0	−0.02, 0
Vaccines Protect Children			−0.1, 0	−0.05, 0	−0.01, 0	0.03, 0	−0.03, 0	0.03, 0	0, 0
Immunization Contributes to Global Health			−0.03, 0	0.01, 0	−0.02, 0	−0.03, 0	−0.06, 0	−0.06, 0	−0.08, 0
Vaccine Fraud and Children			0.08, 0	0.06, 0	0.01, 0	0, 0	0.11, 0	0.1, 0	0.1, 0
Prevention Disease			−0.12, 0	−0.11, 0	−0.13, 0	−0.15, 0	−0.09, 0	−0.14, 0	−0.15, 0
Doctors Vaccinating Children			−0.01, 0	−0.03, 0	0, 0	0.06, 0	0.01, 0	0.07, 0	0.09, 0
Big Pharma			0.02, 0	−0.09, 0	0, 0	−0.1, 0	0.02, 0	−0.09, 0	−0.1, 0
Anti-Vaccine Story			0.11, 0	0.07, 0	0.09, 0	−0.06, 0	0, 0	−0.07, 0	−0.1, 0
Vaccine Science Matters			0, 0	−0.04, 0	−0.03, 0	−0.05, 0	0.02, 0	−0.02, 0	0, 0
Fighting the Flu during the Flu Season			0.06, 0	0.13, 1	0.02, 0	0.06, 0	0.02, 0	0.02, 0	0.05, 0
November–February
Negative Vaccine Consequences					−0.07, 0	−0.09, 0	0.01, 0	−0.07, 0	−0.08, 0
Vaccines Protect Children					−0.05, 0	0.02, 0	−0.04, 0	0.03, 0	0.03, 0
Immunization Contributes to Global Health					0.05, 0	0.08, 0	0.03, 0	0.07, 0	0.07, 0
Vaccine Fraud and Children					−0.2, 36***	−0.2, 32***	−0.18, 4*	−0.25, 164****	−0.24, 49***
Prevention Disease					0.03, 0	0.05, 0	0, 0	0.06, 0	0.06, 0
Doctors Vaccinating Children					−0.06, 0	−0.05, 0	−0.04, 0	−0.07, 0	−0.09, 0
Big Pharma					0.03, 0	0.03, 0	0.11, 0	0.05, 0	0.05, 0
Anti-Vaccine Story					0.02, 0	−0.04, 0	−0.07, 0	−0.05, 0	−0.07, 0
Vaccine Science Matters					0.05, 0	−0.03, 0	0.07, 0	0, 0	−0.03, 0
Fighting the Flu during the Flu Season					0.13, 1	0.11, 0	0.08, 0	0.11, 0	0.13, 1
February–March
Negative Vaccine Consequences							−0.02, 0	−0.02, 0	0, 0
Vaccines Protect Children							0.1, 0	0.05, 0	0.05, 0
Immunization Contributes to Global Health							0.09, 0	0.06, 0	0.06, 0
Vaccine Fraud and Children							−0.02, 0	−0.09, 0	−0.07, 0
Prevention Disease							−0.15, 0	−0.12, 0	−0.1, 0
Doctors Vaccinating Children							0.02, 0	0.02, 0	0.03, 0
Big Pharma							−0.1, 0	−0.13, 1	−0.15, 1
Anti-Vaccine Story							−0.13, 1	−0.1, 0	−0.11, 0
Vaccine Science Matters							−0.07, 0	−0.03, 0	−0.04, 0
Fighting the Flu during the Flu Season							0.04, 0	0.08, 0	0.09, 0
Subgroup analyses: Having discussions of the influenza vaccine with family and friends
September–October
Negative Vaccine Consequences	0.06, 0	0, 0	0.05, 0	0.04, 0	0.04, 0	0.03, 0	0.05, 0	0.03, 0	0.02, 0
Vaccines Protect Children	−0.01, 0	0.02, 0	−0.03, 0	−0.04, 0	−0.02, 0	−0.01, 0	0.03, 0	0.02, 0	0.04, 0
Immunization Contributes to Global Health	0.01, 0	0.04, 0	0.01, 0	−0.03, 0	−0.04, 0	−0.04, 0	−0.04, 0	−0.03, 0	−0.04, 0
Vaccine Fraud and Children	−0.01, 0	−0.02, 0	−0.04, 0	0, 0	−0.05, 0	0.01, 0	−0.06, 0	0, 0	−0.01, 0
Prevention Disease	0.08, 1	−0.02, 0	0.07, 0	0.08, 1	0.08, 1	0.07, 0	0.06, 0	0.06, 0	0.03, 0
Doctors Vaccinating Children	−0.08, 2	−0.03, 0	−0.06, 0	−0.09, 2	−0.08, 1	−0.10, 3	−0.07, 0	−0.09, 1	−0.09, 1
Big Pharma	0, 0	0.06, 0	−0.01, 0	−0.02, 0	−0.01, 0	0.03, 0	−0.06, 0	−0.02, 0	−0.01, 0
Anti-Vaccine Story	−0.03, 0	−0.04, 0	−0.02, 0	0.02, 0	−0.01, 0	0, 0	−0.03, 0	−0.01, 0	−0.02, 0
Vaccine Science Matters	0, 0	0.03, 0	−0.02, 0	−0.01, 0	−0.03, 0	−0.02, 0	−0.05, 0	−0.03, 0	−0.02, 0
Fighting the Flu during the Flu Season	0.04, 0	0.02, 0	0.06, 0	0.09, 2	0.09, 1	0.09, 1	0.09, 1	0.1, 1	0.09, 1
October–November
Negative Vaccine Consequences			0.03, 0	0.04, 0	0.04, 0	0.08, 0	0.04, 0	0.07, 0	0.07, 0
Vaccines Protect Children			0.04, 0	−0.02, 0	0.03, 0	0.01, 0	0.01, 0	0, 0	0, 0
Immunization Contributes to Global Health			−0.01, 0	0, 0	−0.02, 0	−0.02, 0	−0.02, 0	−0.03, 0	−0.04, 0
Vaccine Fraud and Children			0.04, 0	0.04, 0	0.08, 0	0.06, 0	0.06, 0	0.06, 0	0.07, 0
Prevention Disease			0.02, 0	0.03, 0	0.03, 0	0.01, 0	0.02, 0	0.01, 0	0, 0
Doctors Vaccinating Children			0.04, 0	0.07, 0	0.05, 0	0.07, 0	0.09, 0	0.06, 0	0.06, 0
Big Pharma			0.02, 0	0, 0	0.02, 0	0, 0	0.03, 0	0.05, 0	0.05, 0
Anti-Vaccine Story			−0.06, 0	0, 0	−0.05, 0	−0.02, 0	−0.02, 0	−0.03, 0	−0.03, 0
Vaccine Science Matters			0, 0	−0.02, 0	0, 0	−0.04, 0	−0.05, 0	−0.06, 0	−0.04, 0
Fighting the Flu during the Flu Season			−0.08, 0	−0.07, 0	−0.1, 0	−0.08, 0	−0.12, 0	−0.08, 0	−0.08, 0
November–February
Negative Vaccine Consequences					−0.05, 0	−0.03, 0	−0.06, 0	−0.05, 0	−0.05, 0
Vaccines Protect Children					0.07, 1	0, 0	0.07, 1	−0.01, 0	−0.01, 0
Immunization Contributes to Global Health					0.03, 0	0.03, 0	0.01, 0	0.03, 0	0.04, 0
Vaccine Fraud and Children					−0.02, 0	−0.02, 0	−0.01, 0	−0.02, 0	−0.03, 0
Prevention Disease					0.01, 0	0.01, 0	0.03, 0	0.01, 0	0.01, 0
Doctors Vaccinating Children					0, 0	0.02, 0	−0.05, 0	0, 0	0, 0
Big Pharma					0.03, 0	−0.02, 0	0.03, 0	−0.01, 0	0, 0
Anti-Vaccine Story					−0.04, 0	−0.01, 0	−0.04, 0	−0.02, 0	−0.04, 0
Vaccine Science Matters					0.06, 0	0.02, 0	0.09, 2	0.02, 0	0.01, 0
Fighting the Flu during the Flu Season					−0.06, 0	0.01, 0	−0.03, 0	0.03, 0	0.02, 0
February–March
Negative Vaccine Consequences							−0.03, 0	−0.05, 0	−0.04, 0
Vaccines Protect Children							0.03, 0	0.04, 0	0.05, 0
Immunization Contributes to Global Health							0.04, 0	0.05, 0	0.04, 0
Vaccine Fraud and Children							0.01, 0	0, 0	0.02, 0
Prevention Disease							−0.04, 0	−0.02, 0	0.01, 0
Doctors Vaccinating Children							0.01, 0	−0.03, 0	−0.04, 0
Big Pharma							−0.09, 1	−0.02, 0	0, 0
Anti-Vaccine Story							0.01, 0	−0.06, 0	−0.06, 0
Vaccine Science Matters							0, 0	0.02, 0	0.01, 0
Fighting the Flu during the Flu Season							0.01, 0	0.03, 0	0.01, 0

* BF10 > 3.2, ** BF10 > 10, *** BF10 > 32, **** BF10 > 100.

Discussion

Combating an “infodemic” [62] on social media is important in preventing the spread of misinformation in the community. Prior research on social media and vaccines has focused mainly on either cross-sectional correlations between the use of or exposure to social media content and vaccination status, or cross-sectional correlations between the volume of tweets and state-level vaccination rates. Thus, the prior findings have not shed light on the paths between social media and vaccination, which require individual and longitudinal data. Filling this void, our study identified whether social media messages across U.S. counties are associated with vaccine attitudes and vaccination, and how vaccine discussions moderate the relations of social media messages with vaccine attitudes and with actual vaccination. We also found three prevailing vaccine topics: Vaccine Science Matters, Big Pharma, and Vaccine Fraud and Children. Furthermore, the probabilities of Vaccine Science Matters were prospectively associated with vaccine attitudes, and the probabilities of Vaccine Fraud and Children were prospectively associated with vaccine attitudes and actual vaccination. Importantly, in line with the idea of networked collective intelligence [43], discussing the influenza vaccine with others in real-life appeared to mitigate the negative regional relations between Vaccine Fraud and Children and vaccine attitudes and vaccination.

Even though our results showed strong to very strong evidence of associations between topics (i.e., Vaccine Science Matters, Big Pharma, and Vaccine Fraud and Children) and vaccine attitudes and behavior, these associations do not necessarily imply causation and await experimental results. However, the findings can provide important insights into social media effects on health attitudes and behaviors. For example, given the prospective associations, it is possible that posts about scientific evidence of vaccines promote the spread of factual information of vaccines, which contribute to more favorable attitudes towards the flu vaccine. Future studies may assess whether the presence of an unobserved or third variable, such as the change of regional patients’ minimum age for convenient immunization at pharmacies [63], increases discussions on Twitter while also increasing vaccine attitudes and actual vaccination. Another direction of research may be to study the spread of conspiracy theories (e.g., Big Pharma) on social media, which may raise concerns about vaccine safety. People who are high in needle sensitivity may be more prone to the influences of these posts, and a downward spiral may develop, which may result in more negative attitudes toward vaccines [64].

This study has a number of noteworthy limitations. Twitter users may not be representative of social media messaging in general or of communication in the non social media world. First, although Twitter continues to be a popular social media platform [47], other social media should be analyzed in future research. Second, those on Twitter use that platform to connect to friends and acquaintances who live in their area, cross-regional influences also may be important and deserve future consideration [65]. Third, the present study identified a topic model based on a limited albeit large set of tweets (i.e., tweets with location metadata that allow geotagging to U.S. counties). Fourth, in addition to social media, other factors such as health insurance coverage are critical determinants of vaccination, and their influence in interaction with social media should be investigated in the future. Despite these limitations, our findings echo the notion that Twitter can have an influence on the local context [35], [66].

CRediT authorship contribution statement

Man-pui Sally Chan: Conceptualization, Methodology, Software, Formal analysis, Data curation, Writing - original draft, Visualization, Project administration, Funding acquisition. Kathleen Hall Jamieson: Resources, Writing - review & editing. Dolores Albarracin: Conceptualization, Writing - review & editing, Supervision, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.