The 'Fauci Effect': Reducing COVID-19 misconceptions and vaccine hesitancy using an authentic multimodal intervention.

Social media environments enable users to proliferate misinformation surrounding COVID-19. Expert sources, such as Dr. Anthony Fauci have leveraged social media to present corrective multimedia messages. However, little is known about the efficacy of these messages in revising common misconceptions about COVID-19 and influencing behavior. In this study, we examined the efficacy of a multimodal intervention using authentic social media messages that directly addressed common COVID-19 misconceptions. Going further, we identified individual differences that influenced the effectiveness of the intervention, as well as whether those factors predicted individuals' willingness to receive a COVID-19 vaccine. The results showed that the intervention was successful in increasing knowledge when compared to a baseline control. Those who were older and reported less vaccine hesitancy showed greater learning from the intervention. Factors that significantly predicted intention to vaccinate included receiving the intervention, vaccine confidence, vaccine hesitancy, prior flu vaccination history, age, and fear of COVID-19. These findings indicate that multimodal messages can be effectively leveraged in social media to both fight misinformation and increase intention to be vaccinated - however, these interventions may not be as effective for all individuals.

Introduction

Since the onset of the COVID-19 pandemic, social scientists have been called to support response efforts because the crisis requires belief and behavior change (Van Bavel et al., 2020). To illustrate, even though vaccines have been developed to reduce infection rates, vaccine uptake in the United States is relatively low and has plateaued (Hamel et al., 2021), which is troubling given that subsequent booster vaccines may also be critical for prolonged protection (Mahase, 2021). The effectiveness of vaccines depends on the public’s uptake (Fine et al., 2011). One barrier to vaccine uptake is the proliferation and persistence of inaccurate information related to COVID-19 and COVID-19 vaccines. Exposure to inaccurate information may instill, reactivate, and reinforce misconceptions about COVID-19 vaccinations, which in turn contribute to vaccine hesitancy (Reiter et al., 2020, Taylor et al., 2020). Therefore, revising misconceived knowledge and increasing endorsement of accurate information is critical to encourage uptake of COVID-19 vaccines among vaccine-hesitant individuals, who may be especially likely to endorse misconceptions.

A key challenge in revising misconceived knowledge is that individuals routinely encounter information in social media contexts that may reactivate and reinforce their misconceptions. Although social media contexts are the main platforms that facilitate the spread of inaccurate information (Lazer et al., 2018), these platforms may also be leveraged to spread accurate information to correct and revise misconceptions (Vraga & Bode, 2020). To do so, corrective messages must fulfill several conditions and follow best practices (Kendeou et al., 2013, Kendeou et al., 2014, Kendeou et al., 2019). Namely, presenting multiple accurate messages from high-credibility sources in conjunction with causal explanations that support the correct idea may be effective at fostering revision of COVID-related misconceptions (Lewandowsky, Cook, et al., 2020). Although these conditions and best practices are intended to maximize the overall effectiveness of corrective messages, the extent to which individuals revise their misconceptions may depend on a host of individual differences (e.g., attitudes toward vaccines, emotions, political conservatism). Thus, it is critical to identify whether and for whom corrective messages on social media are effective at revising misconceptions about COVID-19 and increasing the likelihood of receiving a vaccine.

In the present study, we have three primary aims. First, we test the efficacy of an authentic multimodal intervention in the context of social media to correct common COVID-19 misconceptions and improve accurate knowledge about COVID-19. Second, we explore the extent to which individual differences influence the effectiveness of the intervention in improving accurate knowledge about COVID-19. Third, we examine the extent to which engagement with the intervention and individual differences predict individuals’ intentions to receive a COVID-19 vaccine. We situate this investigation at the intersection of the extant literatures on knowledge revision, social media, and vaccine hesitancy.

COVID-19 misconceptions and knowledge revision

Individuals encounter inaccurate information on social media, which may reactivate and reinforce misconceptions about COVID-19. Because misconceptions create interference when learners encounter accurate information (Butterfuss and Kendeou, 2020, Potvin et al., 2015), revising these misconceptions is critical to effective learning of accurate information. To foster knowledge revision, the information presented to learners should ideally fulfill certain conditions. Namely, corrective messages or texts must clearly state the targeted misconception, explicitly refute it, and then present a plausible (Lombardi et al., 2016), causal explanation to support the correct idea (i.e., refutation texts; Kendeou et al., 2013, Kendeou et al., 2019, Sinatra and Broughton, 2011). In addition to these conditions, existing research has identified several best practices (see The Debunking Handbook, Lewandowsky, Cook et al., 2020) that may further optimize knowledge revision. For example, presenting accurate information that individuals can easily understand, ideally from high-credibility sources, may increase the extent to which individuals revise their misconceptions. Additionally, refutation texts that place greater emphasis on the accurate information as opposed to the targeted misconception may foster better revision and minimize the likelihood that readers will reject the accurate information and reinforce their initial misconception (Chan et al., 2017, Cook and Lewandowsky, 2011, Wood and Porter, 2019).

According to the Knowledge Revision Components (KReC: Kendeou & O’Brien, 2014) framework, refutation texts are an effective tool for knowledge revision because they induce co-activation of the reader’s misconception and the correct idea, and facilitate their subsequent integration into the same network in the learner’s memory representation. Because refutation texts include causal explanations that support the correct idea, the learner generates a rich, elaborated network of accurate information that competes with the misconception for reactivation. Knowledge revision is successful when the activation of the newly encoded correct information overcomes the activation of the misconception.

The processes and conditions that underlie knowledge revision are complex, even in controlled laboratory settings. We understand even less about how revision unfolds in authentic contexts such as social media, in which individuals may encounter multiple presentations of information that challenge their prior misconceptions. Such an understanding is critical, as multiple presentations may offer opportunities to address learners’ inaccurate understanding of complex issues such as COVID-19 and vaccination uptake (Butterfuss & Kendeou, 2021). Namely, multiple corrective refutation messages may effectively address networks of misconceptions in domains such as COVID-19.

The Knowledge Revision Components Framework - Multiple Documents (KReC-MD; Butterfuss & Kendeou, 2021) was proposed as an extension to the KReC Framework to account for how revision of multiple misconceptions can be achieved when learners engage with multiple corrective messages that are interrelated. KReC-MD attempts to integrate KReC with the core representational and processing aspects from the multiple document comprehension literature (e.g., Perfetti et al., 1999). The framework proposes that source credibility and intertextual integration are two key factors that influence the success of knowledge revision in the context of multiple presentations. Specifically, information from high-credibility sources is more readily integrated and may attain greater activation than information from lower-credibility sources, when credibility is attended to (Van Boekel et al., 2017). Therefore, if multiple refutations and explanations come from high-credibility sources, then the learner may be more likely to integrate and draw inferences among the information that supports the correct ideas. In turn, this integrated, enriched network of accurate information is likely to draw high levels of activation and should therefore outcompete readers’ reactivated misconceptions.

KReC-MD also proposes that the integrated network of accurate information may allow for revision of flawed mental models. For example, an individual could endorse a network of related individual misconceptions that (1) vaccines damage the immune system, (2) vaccines contain toxins, and (3) vaccines can cause the illness they intend to prevent. This network of individual misconceptions may be embedded within a flawed mental model (Chi, 2013) that vaccines are harmful. If a learner who has a flawed mental model about vaccines were to engage with a series of corrective messages, then that reader may sequentially revise the individual misconceptions. As the reader revises each misconception and integrates the accurate information, their accumulated network of accurate information becomes richer, particularly if connections are made across the corrective messages. Rich, highly integrated networks draw more activation, so if the activation of this network overcomes the activation of the flawed mental model, then global revision of the flawed mental model may be successful. However, entrenched beliefs within flawed mental models can be difficult to revise; thus, the revision of mental models depends, in part, on the nature, strength, and coherence of false beliefs that comprise it (Braasch et al., 2013).

In the context of the current study, we focus on addressing the popular but flawed mental model that COVID-19 vaccines are harmful because individuals who endorse such inaccurate information are more hesitant to get vaccinated (Roozenbeek et al., 2020). We aim to facilitate revision using multiple authentic refutation messages from a credible source in the United States (Dr. Fauci) in social media. Therefore, it is also important to understand how the affordances of social media may be leveraged to foster revision in this context.

Affordances of social media

Social media platforms, including YouTube, Instagram, Snapchat, Facebook, Twitter, and TikTok are prominent players in the information landscape. According to a Pew Research Center survey, social media use in the United States is extremely high: 97% of respondents report using social media, 95% report owning a smartphone, and 45% report they are online on a near-constant basis. The most popular social media platform was YouTube, with 85% reporting regular use, followed by Instagram (72%), Snapchat (69%), Facebook (51%), and Twitter (32%) (Anderson & Jiang, 2018). In the context of COVID-19, one analysis of the most viewed YouTube videos showed that over 25% of videos contained misleading information (Li et al., 2020). In the UK, a poll found that 46% of social media users reported that they were exposed to fake news about COVID-19 (Ofcom, 2020). Among viewers knowingly exposed to misinformation, two-thirds reported seeing misinformation on a daily basis, which is problematic given that frequent exposure to inaccurate information from multiple sources can strengthen misconceptions (Pennycook et al., 2018).

Although social media exacerbates the effects of misinformation, it may also be used as a tool to correct misinformation and communicate accurate information. Features of social media that appeal to purveyors of misinformation (e.g., the ability to spread information far, wide, and fast), make social media an ideal vehicle for spreading accurate information as well. In fact, existing work has shown that corrections of misinformation on social media from experts can reduce misconceptions (Vraga and Bode, 2017, Vraga et al., 2020). Thus, spreading corrective messages on social media in partnership with experts may also help to reduce misconceptions about COVID-19 and therefore reduce a key barrier to vaccination uptake. For example, Dr. Anthony Fauci participated in four COVID-19 focused interviews with popular YouTube creators, one of which was the Canadian Comedian Lilly Singh (Singh, 2020). With nearly 15 million subscribers and over 3.5 billion video views, Singh has a powerful platform to reach those who may not encounter Dr. Anthony Fauci on conventional news sources. The efficacy of these approaches, though, has not been systematically examined.

Multimodal texts that involve a combination of visuals, sounds, and text are becoming a dominant form of communication in social media, and so they may be particularly useful in the fight against inaccurate information about COVID-19 (Kiili et al., 2021, Kress, 2010). Videos, in particular, may provide an effective and engaging means of addressing misinformation and presenting accurate information in a highly accessible way because they use a variety of modes (i.e., dynamic visuals, sound, text, gestures) to aid viewers in constructing a coherent mental representation from complex information in ways that simple text unimodal presentations cannot (Cohn, 2019, Kiili et al., 2021). A multimodal approach to administering corrective messages has the potential to result in revision of individuals’ misconceptions and challenge flawed mental models about COVID-19, which may in turn have a positive effect by reducing hesitance to receive a COVID-19 vaccine.

This being said, a multimodal approach to correcting inaccurate information about COVID-19 may not lead to better outcomes than a traditional text-based approach. Research in the extant literature comparing videos and texts has shown that videos may be harder to integrate than texts, and texts can lead to the use of more effective strategies and higher comprehension (Lee and List, 2019, List & Ballenger, 2019). Other research has found text and video to not significantly differ in terms of comprehension and attention (Delgado et al., 2021) and that while texts may improve individuals’ integration of concepts, videos may have a stronger effect on individuals’ beliefs (Salmerón et al., 2020). Thus, the potential effects of videos on correcting misconceptions depend on a multitude of factors that go well beyond the affordances of the medium itself.

Beyond misconceptions: additional barriers to revision and vaccine uptake

Besides learners’ inaccurate prior knowledge of COVID-19 and vaccines, there are a host of individual differences that may influence the extent to which learners endorse and integrate accurate information, as well their likelihood of receiving vaccines. These variables include vaccine confidence, vaccine hesitancy, fear of COVID-19, political conservatism, age, and previous vaccine record (Lewandowsky, Jetter et al., 2020). An understanding of how these variables influence vaccine uptake is crucial to inform the development of targeted corrective messages that promote accurate information and encourage vaccination for specific audiences.

Individuals who have less confidence in vaccines tend to have higher vaccine hesitancy and are therefore less likely to receive a COVID-19 vaccine (Badur et al., 2020, Machingaidze and Wiysonge, 2021). Vaccine-hesitant individuals may also be less likely to integrate accurate information when they encounter corrective messages, as vaccine hesitancy has been associated with rejection of scientific information about vaccines (Hornsey et al., 2020). Additionally, vaccine-hesitant individuals tend to report less fear related to COVID-19 than vaccine-accepting individuals (Killgore et al., 2021). The extent to which individuals fear COVID-19 may also influence their likelihood of vaccine uptake, as fear is associated with increased compliance with other preventive health behaviors (e.g., masking, social distancing; Yıldırım et al., 2021). At the same time, negative activating emotions, such as fear and anxiety, have been shown to interfere with learning (Pekrun, 2014, Trevors et al., 2021) and motivation to learn (Pekrun, 2006). For example, individuals’ negative emotions (such as confusion, anxiety, and frustration) can mediate the relationship between their self-concept and knowledge revision (Trevors et al., 2016) and in fact corrections attacking people’s worldviews and leading to an identity threat can lead to negative emotions that motivate strategies to disconfirm the corrections (Ecker et al., 2022).

There are also several relevant sociodemographic variables that may influence vaccine uptake. Namely, participants who were higher in political conservatism have been shown to take COVID-19 less seriously (Calvillo et al., 2020), engage in fewer protective health behaviors (Utych, 2021), and be the most hesitant to receive a COVID-19 vaccine (Colvin & Slodysko, 2021). Decreased willingness to accept a COVID-19 vaccine has also been associated with younger age as a function of decreased risk for serious symptoms compared to older adults (Freeman et al., 2021, Roozenbeek et al., 2020). Finally, receiving the flu vaccine has been positively associated with uptake of the COVID-19 vaccine, which reflects a general acceptance of vaccines for some individuals (Caserotti et al., 2021).

The current study

The present study is situated at the intersection of knowledge revision, social media, and vaccine hesitancy around COVID-19 in the United States context and had three aims. First, we evaluated the efficacy of a multimodal intervention that leveraged authentic messages delivered by a credible source, Dr. Fauci, in the context of a YouTube video. We examined the extent to which messages that addressed COVID-19 misinformation and misconceptions were effective at increasing accurate knowledge about COVID-19 compared to a baseline control group that did not receive the intervention. Second, we identified factors that could influence the efficacy of the multimodal intervention. Specifically, we assessed the extent to which individual differences--including vaccine confidence, vaccine hesitancy, and fear about COVID-19--influenced the effectiveness of the intervention. Third, we identified the factors that predicted individuals’ willingness to receive a COVID-19 vaccine. Specifically, we assessed how variables including intervention condition (intervention vs. control), vaccine confidence, vaccine hesitancy, prior flu shot uptake, political conservatism, age, and fear about COVID-19 predicted individuals’ intention to receive a COVID-19 vaccine.

Method

Participants

Nine hundred sixty-eight participants were recruited across the United States between April 30th and May 1st, 2020 using Amazon’s Mechanical Turk (MTurk). Participants were randomly assigned to either the intervention condition or the control condition. Of the 493 participants in the intervention condition, 126 were removed for not watching at least five of the videos, leaving a final intervention sample of 367. The final sample (N = 842) had an average age of 38.56 years (SD = 12.75); 38% reported their biological sex as female (0.7% declined to respond); 67% reported their ethnicity as White, 15% as African American, 7% as Hispanic/Latino, 7% as Asian/Pacific Islander, 2% as Native American/American Indian, 0.4% as Middle Eastern, 1% as Other/Prefer not to respond; 55% reported having received a Bachelor’s degree; 54% reported a household annual income under $60,000; 70% reported having at least one full-time job (9% reported being unemployed); 67% were home owners, with an average household membership of 3.27 (SD = 1.45). Regarding political ideology, 43% identified as Democrat and 39% identified as Republican. With respect to social and economic views on a seven-point Likert scale (1 - “very liberal,” 4 - “moderate,” 7 - “very conservative”), the sample average for social views was 4.27 (SD = 1.96) and the sample average for economic views was 4.43 (SD = 1.86). Every state was represented except for Maine, Montana, North Dakota, and South Dakota.

Materials

PANAS-X: Fear

A ten-item scale measuring general negative affect from the PANAS-X scale (Watson & Clark, 1994) was used. The scale consists of words that describe different emotions (e.g. afraid, nervous, upset). Items are rated on a five-point Likert scale (“Very slightly or not at all” to “Extremely”) where participants indicate the extent to which they have felt the emotion described during the last week due to the COVID-19 pandemic. Items from the general negative affect score that were indicative of fear were then summed to create a general fear score. Items used included afraid, scared, nervous, and jittery (α = 0.89).

Previous Vaccine Behavior: Flu Shot

A single item asked participants to indicate whether or not they had received a flu shot within the past year. Participants responded with either “Yes” or “No.”

Vaccine Attitudes: Hesitancy and Confidence

A scale was developed to measure participants’ attitudes about vaccines. Items were drawn from previously developed vaccine measures (Larson et al., 2015a, Larson et al., 2015b, Lewandowsky et al., 2020). The scale included 16 statements about vaccines (e.g. “Vaccines are an effective way to stop the spread of preventable diseases.”), with respondents rating each statement on a five-point Likert scale (“Strongly Disagree” to “Strongly Agree”). An exploratory factor analysis revealed two main factors, Chi square X (136, N = 842) = 6669.44, p <.01; KMO = 0.92, which explained 58.63% of variance. Nine items reflected vaccine confidence (α = 0.88) and five items reflected vaccine hesitancy (α = 0.84). Two items did not load adequately on either factor and were removed (“I have the right to refuse required vaccines for any reason” and “I know where to go to get a vaccine”). Sum scores for the vaccine confidence items and vaccine hesitancy items were used in the analysis. See Table A.1 in the Appendix for the items used. In this analysis, vaccine hesitancy scores were then reverse coded so that higher scores indicated higher vaccine hesitancy.

Table A1

Vaccine Hesitancy and Confidence Measure Items.

	Statement	Factor
1	Vaccines are important for children to have.	Confidence
2	Childhood vaccines are effective in preventing diseases.	Confidence
3	Getting vaccines is a good way to protect children from diseases.	Confidence
4*	Vaccines have negative side effects that outweigh the benefits of vaccination for children.	Hesitancy
5*	Childhood vaccines have adverse effects.	Hesitancy
6	Children need vaccines for diseases that are not common anymore.	Confidence
7	Vaccines are a significant contribution to public health.	Confidence
8	Vaccines are a safe way to stop the spread of preventable diseases.	Confidence
9	Vaccines are an effective way to stop the spread of preventable diseases.	Confidence
10	I follow what my doctor recommends about vaccines.	Confidence
11	I have the right to refuse required vaccines for any reason.	---
12	Getting vaccinated is compatible with my religious beliefs.	Confidence
13*	I am concerned about serious adverse effects of vaccines.	Hesitancy
14*	Getting vaccines is costly for me.	Hesitancy
15*	I don't have time to get vaccinated.	Hesitancy
16	I know where to go to get a vaccine.	–

Note. * Indicates a reverse-coded item.

Source Familiarity and Trust

Participants assigned in the intervention condition were presented a labeled photo of Dr. Anthony Fauci and were asked to rate the extent to which they found Dr. Fauci trustworthy on a three-point scale (“Not At All Trustworthy,” “Somewhat Trustworthy,” “Very Trustworthy”) with an option to report having no knowledge of Dr. Fauci. 3.5% of participants did not know who Dr. Fauci is, 3.5% of participants rated Dr. Fauci as “Not Trustworthy,” 32.4% as “Somewhat Trustworthy,” and 60.5% as “Very Trustworthy.”

COVID-19 Knowledge Measure: Accuracy and Confidence

All participants completed a knowledge measure that evaluated their understanding of 15 statements about COVID-19, including 6 misconceptions and 9 facts. These were the same topics that were addressed in the intervention. Participants responded True/False and then rated their confidence on a five-point scale (1 “Not at all confident” – 5 “Completely confident”). Participants were awarded a score of 1 for correct responses and −1 for incorrect responses. This scoring system was necessary because participants’ scores were calculated by multiplying each True/False response by its confidence score and then summing all values, producing a scale that ranged from −75 to 75. Combining accuracy and confidence was used so that being incorrect but less sure would be less consequential for overall scores in comparison to having a misconception and being confident of it, as guesses are less impactful on the overall score. Reliability for the knowledge measure scores was good (α = 0.73).

Future Vaccine Behavior: COVID-19 vaccine

A single item asked participants to indicate whether they would receive a COVID-19 vaccine in the future when one becomes available. Participants responded with either “Yes,” “No,” or “Maybe.”

COVID-19 misinformation multimodal refutation intervention

The video-based intervention was designed to reactivate 15 plausible beliefs about COVID-19 and followed each one with video clip from Dr. Fauci. First, each belief was presented as a single sentence, and participants were asked to respond with True/False for each item, and indicate their confidence in their response using a five-point rating scale (1 “Not at all confident” – 5 “Completely confident”). After responding to each statement, a short video clip (less than 60 s) was presented, in which Dr. Anthony Fauci either confirmed or refuted the information from the prior item and provided a causal explanation. Specifically, if the item was framed as a correct idea, Dr. Fauci confirmed and explained. If the item was framed as a misconception, then Dr. Fauci refuted and explained. In total, fourteen video clips (one video clip contained two facts about COVID-19) were spliced from an informational interview of Dr. Fauci by YouTuber Lilly Singh, originally posted on YouTube in March 2020 (Singh, 2020; see Fig. 1 for an example video image). The average video length was 31 s. See Table A.2 in the Appendix for a list of the items used (facts and misconceptions) as well as a link to the original YouTube interview of Dr. Fauci.

Fig. 1

Example video image of refutation. Retrieved from https://www.youtube.com/watch?v=F2YKKba6ps0.

Table A2

COVID-19 Knowledge Measure.

Item No.	Statement
1*	The coronavirus has the same mortality rate as the seasonal flu virus.
2	The coronavirus and other respiratory viruses spread through respiratory droplets in the air (i.e. coughing and sneezing).
3	The coronavirus can remain suspended in the air for a few minutes.
4	The coronavirus can live on surfaces for a few hours.
5	The coronavirus survives longer on hard surfaces (i.e. plastic, stainless steel).
6*	We know that the coronavirus will likely die in warm, humid weather.
7	Face masks are not 100% effective at blocking coronavirus transmission.
8	Individuals living in the U.S. were encouraged not to buy face masks to preserve the supply for healthcare workers.
9*	The majority of people infected with the coronavirus will never show symptoms.
10	There is currently no drug that has proven effective for treating COVID-19.
11	A vaccine for the coronavirus will not be available for at least another year.
12*	Vitamins C and D will help protect you from the coronavirus.
13*	We understand why men tend to have worse COVID-19 symptoms than women.
14*	Anti-inflammatory medicine (i.e. ibuprofen/Advil) will worsen the symptoms of COVID-19.
15	We won’t see a drop in coronavirus cases in the U.S. for several more weeks.

Note. * Indicates a misconception item.

Original Video Link: https://www.youtube.com/watch?v=F2YKKba6ps0.

Procedure

After providing consent, participants were randomly assigned to either the intervention or baseline control condition. All participants then completed the Previous Vaccine Behavior and Vaccine Attitudes measures. Intervention participants completed the Source Familiarity and Trust measure, and then engaged in the multimodal intervention. Next, all participants completed the COVID-19 Knowledge Measure and indicated their intention to receive the COVID-19 vaccine. All participants provided demographic information and were debriefed. Finally, all participants were given additional COVID-19 resources and were compensated for their participation in the study. Data was collected in late April of 2020, only a few weeks after lockdowns began in the United States and many months before a COVID-19 vaccine was developed and ready for distribution.

Results

Group equivalence

Of the 842 participants, 374 (44.4%) reported that they received a flu shot in the past year and 468 (55.6%) reported that they did not receive a flu shot. Moreover, 542 (64.4%) participants indicated that they would get the COVID-19 vaccine; 60 (7.1%) participants indicated they would not get the vaccine; and 240 participants (28.5%) indicated that they might get the COVID-19 vaccine. Further descriptive statistics and group equivalence are reported in Table 1 . As can be seen, the random assignment to condition resulted in intervention and baseline control groups that did not significantly differ in any of the core variables examined.

Table 1

Descriptive Statistics and Group Equivalence.

	Control (n = 475) Intervention (n = 367)							t-value p-value
	M	SD		M	SD	Min	Max
COVID-19 Fear	10.98	4.43		10.70	4.75	4	20	0.897	0.370
Vaccine Hesitancy	14.37	5.57		14.40	5.71	5	25	-0.073	0.942
Vaccine Confidence	37.26	6.23		37.67	6.26	9	45	-0.950	0.343
Social Political Conservatism	4.32	1.96		4.21	1.97	1	7	0.787	0.432
Economic Political Conservatism	4.44	1.83		4.43	1.90	1	7	0.094	0.925
Age	38.80	12.93		38.25	12.52	19	92	0.626	0.513
Flu Shot (proportion received)	0.46	0.49		0.43	0.50	0	1

Efficacy of the intervention

A Welch two-sample t-test was conducted to examine the extent to which the intervention and baseline control groups differed on post-test knowledge scores. The analysis showed that the intervention group (M = 30.16, SD = 23.56) scored significantly higher than the baseline control group (M = 21.73, SD = 20.52), t (727.7) = 5.44, p <.001, d = 0.30.

We also examined and compared frequency plots to understand deeper the effects of the intervention on knowledge scores. As seen in Fig. 2a , the baseline control group had moderate amounts of correct knowledge and moderate confidence in that knowledge (scores 0 to 50). Few participants had moderate amounts of incorrect knowledge and moderate confidence (scores −50 to 0). There were no participants with high incorrect knowledge and strong confidence or what would be indicative of strongly held misconceptions. Given that the two groups were randomly assigned to condition and were equivalent in all core variables of interest, these characteristics of knowledge base may be indicative of those in the general population we recruited from. The effects of the intervention manifest as higher correct knowledge and confidence (scores 50 to 75) in relation to the baseline control group, as can be seen in Fig. 2b .

Fig. 2a

Baseline Control Post-Test Scores.

Fig. 2b

Intervention Post-Test Scores.

Individual differences influencing intervention efficacy

Next, we identified individual difference variables that influenced the effectiveness of the intervention. We conducted a multiple regression analysis to test the extent to which individual difference variables--including (1) vaccine hesitancy, (2) vaccine confidence, (3) social and economic political conservatism, (4) age, (5) fear about COVID-19, and (6) previous flu shot -- interacted with the intervention condition to influence participants’ knowledge scores. See Table 2 for correlations among the predictors included in the model.

Table 2

Correlations between core variables.

Variable	1	2	3	4	5	6	7
1. COVID-19 Fear	–
2. Vaccine Hesitancy	0.47**	–
3. Vaccine Confidence	−0.02	−0.40**	–
4. COVID-19 Knowledge	−0.37**	−0.66**	0.34**	–
5. Social Political Conservatism	0.29**	0.51**	−0.21**	−0.46**	–
6. Economic Political Conservatism	0.21**	0.42**	−0.16**	−0.38**	0.82**	–
7. Age	−0.16**	−0.14**	0.11**	0.18**	−0.02	−0.06	–

Note. **p <.01.

The regression model accounted for a significant proportion of the overall variance in knowledge scores, R 2 = 0.52, F (15, 826) = 58.76, p <.001. The results showed that the intervention worked differentially based on age and vaccine hesitancy. Specifically, knowledge scores significantly increased for participants in the intervention condition as age increased and vaccine hesitancy decreased (see Table 3 for further details). Results also showed main effects of COVID-19 fear, vaccine confidence, vaccine hesitancy, and social political conservatism. Knowledge scores significantly increased as vaccine confidence increased, whereas knowledge scores decreased as COVID-19 fear and vaccine hesitancy increased. Knowledge scores also increased as social political conservatism decreased. Economic political conservativism and prior vaccine behavior were not associated with knowledge scores.

Table 3

Multiple Regression Analysis.

	Estimate	SE	t-value	p-value
Intercept	46.45	6.32	7.34	<0.001***
Condition	0.18	9.48	0.02	0.98
COVID-19 Fear	−0.39	0.19	−2.05	0.04*
Vaccine Hesitancy	−1.65	0.18	−9.19	<0.001***
Vaccine Confidence	0.27	0.13	2.03	0.04*
Social Political Conservatism	−1.46	0.65	−2.25	0.02*
Economic Political Conservatism	−0.65	0.66	−0.98	0.33
Age	0.08	0.06	1.31	0.19
Flu Shot	−0.91	1.50	−0.60	0.55
COVID-19 Fear*Condition	0.10	0.28	0.35	0.73
Vaccine Hesitancy*Condition	−0.69	0.27	−2.61	0.009**
Vaccine Confidence*Condition	0.17	0.20	0.86	0.39
Social Political Conservatism*Condition	0.30	1.05	0.29	0.77
Economic Political Conservatism*Condition	0.30	1.05	0.29	0.77
Age*Condition	0.21	0.09	2.34	0.02*
Flu Shot*Condition	−0.19	2.29	−0.08	0.93

Note. *p <.05; **p <.01; ***p <.001.

Intention to receive a COVID-19 vaccine

A multinomial logistic regression model using backward elimination was conducted to identify variables that predicted participants’ intention to receive a COVID-19 vaccine (“yes” vs. “maybe” vs. “no”). For the multinomial model, we designated those who provided a “maybe” response to the vaccine intention question as the reference group. This reference group allowed us to identify predictors that may be important to determining the likelihood that someone who is ‘persuadable’ (and responded “maybe” to the COVID-19 vaccine) would be willing to receive (“Yes”) or decline (“No”) the vaccine. Predictors included intervention condition (treatment vs. baseline control), vaccine confidence, vaccine hesitancy, social and economic political ideologies, age, flu shot uptake, knowledge of COVID-19, and fear emotions associated with COVID-19. The final model fit was significant, X 2 (12) = 252.05, R 2 = 0.28, p <.001. As shown in Table 4 , intervention condition, vaccine confidence, vaccine hesitancy, age, flu shot uptake, and fear associated with COVID-19 were significant unique predictors of COVID-19 vaccination intention. Social and economic political ideologies and knowledge of COVID-19 were not significant predictors of vaccine intentions and were therefore eliminated from the final model.

Table 4

Parameter Estimates for Significant Predictors in the Multinomial Logistic Regression (N = 842).

Response	Reference	Predictor	Exp(B)	Exp(B) 95% CI
“No”	“Maybe”	Intervention Condition	1.067	[0.44, 1.70]
		Vaccine Confidence	0.889***	[0.84, 0.94]
		Vaccine Hesitancy	1.060	[0.97, 1.15]
		Age	1.015	[0.99, 1.04]
		Flu Shot Uptake	0.765	[0.04, 1.50]
		COVID-19 Fear	0.938	[0.86, 1.02]
“Yes”	“Maybe”	Intervention Condition	1.408**	[1.07, 1.75]
		Vaccine Confidence	1.120***	[1.09, 1.15]
		Vaccine Hesitancy	0.921***	[0.88, 0.96]
		Age	0.983**	[0.97, 1.00]
		Flu Shot Uptake	2.136***	[1.79, 2.48]
		COVID-19 Fear	1.041∼	[1.00, 1.08]

Note. Exp(B) = odds ratio (OR). ∼ p <.10, *p <.05, **p <.01, ***p <.001.

Overall, the intervention seems to improve vaccination intention. For participants in the intervention condition, the odds of responding “yes” to receiving a vaccine rather than “maybe” was 1.408 times the odds of responding “yes” rather than “maybe” in the non-intervention condition. Furthermore, these results also identify a constellation of variables that includes attitudes (i.e., vaccine confidence, vaccine hesitancy), demographics (i.e., age), behaviors (i.e., flu shot uptake), and emotions (i.e., fear of COVID-19) that predict vaccine intentions1 .

The results also show that increases in vaccine confidence are associated with increases in vaccination intention. For every one-unit increase in vaccine confidence, participants were less likely to respond “no” than “maybe” (OR = 0.889) and more likely to say “yes” than “maybe” (OR = 1.120). On the other hand, vaccine hesitancy was associated with decreases in vaccination intention. For every one-unit increase in vaccine hesitancy, participants were less likely to respond “yes” than “maybe” (OR = 0.921), but vaccine hesitancy did not influence the odds of saying “no” compared to “maybe.” Next, age influenced vaccination intention. For every one-unit increase in age, participants were less likely to respond “yes” to the vaccine compared to “maybe” (OR = 0.983), but age did not influence the odds of saying “no” compared to “maybe.” Moreover, receiving a prior flu shot seemed to improve vaccination intention. For participants who reported taking the flu vaccine, the odds of choosing “yes” rather than “maybe” was 2.136 times higher the odds of choosing “yes” rather than “maybe” for participants who reported not taking the flu vaccine. Finally, fear about COVID-19 was associated with increased vaccination intention. That is, for every one-unit increase in fear associated with COVID-19, participants were more likely to respond “yes” compared to “maybe” (OR = 1.041).

Discussion

The primary goal of the current study was to evaluate the efficacy of a multimodal intervention from an authentic social media context for correcting misconceptions related to COVID-19, and thus increasing accurate knowledge and confidence in this topic. We also examined the extent to which the intervention, along with several individual differences (i.e., attitudes toward vaccines, prior flu vaccination history, political ideology, age, and COVID-19-related fear) influenced the extent to which participants learned accurate information from the intervention, as well as participants’ intention to receive a COVID-19 vaccine. Understanding factors that influence uptake of accurate information about COVID-19 and COVID-19 vaccines is critical because vaccine rates are relatively low and have plateaued in the United States (Hamel et al., 2021); at the same time, booster vaccines appear to be necessary to continue to maintain protection. The factors identified in this study may inform more targeted messaging approaches to increase accurate knowledge about COVID-19 and encourage vaccination among vaccine-hesitant individuals.

Overall, the results showed that the intervention was successful in increasing accurate knowledge and confidence about COVID-19 when compared to a baseline no-treatment control. This finding is consistent with previous work showing that authentic social media messages can reduce misconceptions (Vraga and Bode, 2017, Vraga et al., 2020) and extends these findings in the context of YouTube, one of the most popular social media platforms. Although our results show that the intervention increased participants’ accurate knowledge overall, it was also critical to examine for whom such interventions may be more or less effective. Thus, we examined how several individual difference variables influenced the extent to which participants learned accurate information.

With respect to the influence of individual differences, the results showed that participants who were less hesitant about vaccines and older tended to hold more accurate knowledge about COVID-19 after engaging with the intervention. In general, regardless of whether participants engaged with the intervention or not, participants who were more confident in vaccines, less hesitant about vaccines, less afraid of COVID-19, and more socially liberal held more accurate knowledge about COVID-19. Looking to the finding that less vaccine hesitancy is associated with more accurate knowledge, this is consistent with existing evidence that vaccine-hesitant individuals are more likely to reject scientific information about vaccines than vaccine-accepting individuals (Hornsey et al., 2020), as well as research that suggests individuals are more likely to reject information in social media contexts that conflicts with their beliefs compared to belief-consistent information (Butterfuss et al., 2021). Thus, participants who were vaccine-hesitant may have been more likely to reject the accurate, scientific information provided in the intervention, thereby resulting in less COVID-19 knowledge.

With respect to predicting vaccine intentions, the factors that significantly predicted intention to vaccinate against COVID-19 included intervention condition, vaccine confidence, vaccine hesitancy, prior flu vaccination history, age, and fear of COVID-19. Interestingly, receiving the intervention positively predicted intention to vaccinate (compared to the control group), but knowledge about COVID-19 did not. These results suggest that simply holding more correct information about COVID-19 may not be sufficient to influence vaccine uptake; rather, it may be the case that there is a dissociation between knowledge and beliefs or attitudes in this context (Sinatra & Mason, 2013), with attitudes playing a more important role in behavior change than knowledge (Thacker et al., 2020).

Taken together, the results of the present study advance our understanding for whom and under what conditions corrective messages delivered in social media via credible sources may be effective. The results showed that participants who received the multimodal intervention had higher correct knowledge and were more confident about that knowledge than participants in a baseline control condition. This finding suggests that knowledge revision processes theorized and documented in controlled lab settings likely also unfold in authentic multimedia contexts. It is reasonable to assume that the same conditions that foster knowledge revision in the context of reading refutation texts (i.e., activation, integration, and competing activation; Butterfuss and Kendeou, 2021, Kendeou and O’Brien, 2014) will also be at play in more authentic settings, such as the multimodal refutation messages delivered by Dr. Fauci. In the current intervention, these conditions may have been facilitated as participants first read and rated the accuracy of knowledge statements (i.e., activation) and next watched Dr. Fauci explaining this information (i.e., integration; competing activation). Even though it would be tempting to conclude that these processes unfolded as theorized, such a conclusion cannot be drawn from the current study. Future research needs to address this issue by examining directly the actual processes (e.g., by using think-alouds or other online methodology) during multimodal refutation messages like the ones used in this study.

A related question that remains is whether such multimodal approaches lead to better outcomes than a traditional text-based approach. On the one hand, videos may be harder to integrate than texts, and texts can lead to the use of more effective strategies and higher comprehension (Lee and List, 2019, List & Ballenger, 2019). On the other hand, videos may have a stronger effect on individuals’ beliefs (Salmerón et al., 2020). Thus, in future research it would be important to directly compare similar multimedia interventions to traditional refutation texts addressing socio-scientific misconceptions. Future work should also examine the extent to which multimedia affordances influence readers’ attention to and evaluation of source credibility compared to refutation texts, building on recent work examining the effects of source on multiple document comprehension across modalities (Salmerón et al., 2020). It may be the case that the ability to see and hear the information source could influence credibility evaluations and, in turn, the extent to which accurate information is endorsed and integrated.

Even though the current work did not include a pre-test to avoid the testing effect as a potential explanation of the findings (Karpicke and Blunt, 2011, Roediger and Butler, 2011), we cannot rule out testing and its effects completely. This is due, in part, to the use of T/F judgements on items similar to those in the post-test before watching each refutation video. This was an important component to ensure that we followed best practices in debunking (Ecker et al., 2022, Lewandowsky et al., 2020a), namely reactivating the misconception in memory before refuting and explaining it. As Zengilowski et al (2021) notes, the testing effect is rarely mitigated in these studies, and this study is no exception. In the few studies though where the testing effect was accounted for, the findings showed that the refutation effects on learning were above and beyond mere testing (Kendeou et al., 2016). It would be important, though, to adopt different designs in future work to address this issue (e.g., the Solomon Four-Groups design as suggested by Zengilowski et al., 2021).

The findings also showed that social media interventions like the one we examined in this study, although effective 'on average', is not be effective for all individuals. On the one hand, we found that receiving the intervention was associated with an increased likelihood of saying “yes” to vaccines compared to “maybe,” which suggests that individuals who are “on the fence” about receiving a vaccine may respond to knowledge-based interventions and may therefore shift to greater vaccine acceptance. On the other hand, we found that the intervention was not associated with a greater likelihood of saying “maybe” compared to “no,” which suggests that individuals who outright reject vaccines are unlikely to respond to knowledge-based interventions. This may indicate that participants need to be willing and open to revising their misconceptions about COVID-19 and gaining new information in order for the intervention to be successful. This echoes findings in other scientific domains that individuals need to be open to change in order to revise their beliefs (Södervik et al., 2015, Stathopoulou and Vosniadou, 2007).

These findings also highlight the critical role that vaccine hesitancy plays, and future research must identify approaches to reducing vaccine hesitance. As a requisite step, we must first understand the complexities that underlie vaccine hesitancy in general, and in the context of COVID-19, in particular. Knowledge alone is not sufficient to increase vaccination intentions and should instead be leveraged in conjunction with other factors such as attitudes, beliefs, and affect (Mason et al., 2008, Sinatra et al., 2014). While we acknowledge that we cannot make a causal claim in this study that perceptions changed and as a result there was a change in intention, we point to this as a potential mechanism due to previous work on the mediating role of perceptions and beliefs (e.g., Thacker et al., 2020). Therefore, greater emphasis and interventions must be focused towards not only reducing vaccination misconceptions, but also increasing the public’s confidence in the efficacy of vaccines. One potentially beneficial approach could be to investigate how different messaging frames influence perceptions of vaccines for different audiences (e.g., vaccine-hesitant individuals, political conservatives, different age groups). Different message frames could differentially appeal to individuals’ emotions, knowledge, and values. For example, how would liberals and conservatives differ in their responses to messages that were framed in terms of the economic benefits of vaccination (e.g., fewer business closures)? How would older individuals respond to messages framed in terms of the COVID-19 vaccines reducing fear of severe illness?

Going further, data collection for this study occurred at a very specific point in time - around one month after pandemic lockdowns began in the United States and using MTurk. Thus, the findings need to be interpreted within this time and context. With respect to time, views surrounding COVID-19 have changed over time; for instance, there has been a decline in believing misinformation since the early days of COVID-19 vaccination efforts (although reports suggest that many still hold vaccine misperceptions and are uncertain about the truth of vaccine misconceptions), and the socioeconomic status among people most likely to hold vaccine misperceptions has shifted over time (Ognyanova et al., 2022). Views of figures such as Dr. Fauci have also changed - Dr. Fauci’s credibility dropped 10% overall and 29% among conservatives between April and September of 2020 alone (KFF, 2020). The world has been changing, and as the perceived credibility of sources changes, this may influence beliefs and knowledge (Johnson et al., 2021), indicating the importance of measuring these concepts as well as the need for future research.

Furthermore, and consistent with an increasing trend in social sciences, turning to crowdsourcing platforms, such as MTurk, in efforts to recruit larger and more diverse samples (Paolacci & Chandler, 2014), also comes with certain limitations and constraints. While participants from MTurk are considered more representative of the United States population than the traditional samples from our college campuses, they also have higher information literacy and technical expertise (Yaqub et al., 2020). In the context of this study, a close look at the sample characteristics shows that it is generally representative of the US population with respect to most demographic characteristics. A stark difference is level of education, showing that the MTurk participants had higher education than the level reported by U.S. census (55% vs. 39% with at least a Bachelor’s degree, respectively). This difference could have impacted the effectiveness of a multimodal intervention in a positive way. Despite these limitations and constraints, the findings of the present study have enriched our understanding as to the potential of using social media to enact multimodal refutation interventions and the impact exposure to those interventions could have.

In sum, the medical sciences are doing their part in developing, testing, and distributing vaccines and implementing public health measures to slow the spread of COVID-19 and save lives. The social sciences must also keep striving to do their part--namely, understanding and improving the public’s willingness to do what is necessary to endure the pandemic. However, improving individuals’ knowledge of COVID-19 and vaccinations is only one small piece of a very complex puzzle. The pandemic has imposed several economic, sociological, educational, technological, and psychological problems that the social sciences must examine. Doing so is critical to inform how individuals negotiate the key challenges associated with the pandemic, including the threat of recurring outbreaks, rapid changes to our information ecosystems, and updates to public health regulations and recommendations.

Author Note

The research reported in this paper was supported by the College of Education and Human Development Graduate Student Fellowship at the University of Minnesota and the Guy Bond Chair in Reading to P. Kendeou from the University of Minnesota. We thank Daheen Choi and Kelsey Will for their assistance in this research.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.