Effects of politicized media coverage: Experimental evidence from the HPV vaccine and COVID-19.

Although concerns about politicization of health and science are not new, the COVID-19 pandemic has amplified attention to how political disagreement over scientific guidelines and recommendations might influence attitudes and behaviors about the health topics in question and might even spill or carry over to affect other attitudes important to public health. The literature employs differing definitions of politicization-at times referring to controversy in the public sphere, at others referring to the exploitation of the uncertainty inherent in science, and at still others referring to whether the issue enters political discourse-all of which are viewed as distinct dimensions by the public. What is not known is how these different aspects of politicization influence public attitudes about the health topics and or broader attitudes about scientific guidelines, and-assuming adverse effects-what strategies might be effective at mitigating the consequences. This paper draws on a survey experiment of 3012U.S. respondents fielded in summer 2020 that was designed as a pilot study to assess the effects of different dimensions of politicization. Findings do not suggest that one type of politicization is necessarily more pernicious than the others. In fact, all types of politicization increased negative emotional responses and confusion, both with respect to the health topic in question (HPV vaccine and COVID-19) but also on other domains, although opinions about policy were unaffected. The findings also suggest that inoculation may have potential as a messaging strategy for blunting the adverse effects of exposure to politicization.

Although scholarly concerns about politicization of health and science are not new,1, 2, 3, 4, 5, 6 the COVID-19 pandemic has increased attention to how political disagreement over scientific guidelines and recommendations might influence the public's attitudes and behaviors, both about the health topics in question7, 8, 9, 10, 11, 12, 13, 14, 15 as well as other attitudes important to public health. As prior work has acknowledged, the literature employs differing definitions of politicization—at times referring to controversy in the public sphere, at others referring to the exploitation of the uncertainty inherent in science, and at still others referring to the extent to which the issue enters political discourse. In previous work, we have found that the public perceives these distinct dimensions and that these dimensions have different associations with policy attitudes.19, 20 What is not known is the causal impact of these different aspects of politicization on public attitudes about the topics in question or more general attitudes about scientific guidelines and recommendations. And should there be adverse effects of different dimensions of politicized messages about health topics, there is insufficient research on what strategies might be most effective at mitigating each type.

In this paper, we draw on a survey experiment that was designed as a pilot study to assess the effects of different dimensions of politicization (i.e., emphasizing controversy, scientific uncertainty, and prevalent partisan political cues). We randomly assigned participants to a topic (HPV vaccine or COVID-19) and then to receive either general information about updated scientific recommendations on those topics or to information about those recommendations that includes differing forms of politicization about the topic. We also tested whether one specific type of messaging strategy—an inoculation message tailored to each dimension that warns of people seeking to politicize health and science issues, delivered in advance of the politicization message treatments—can blunt the effects of politicization. The findings do not suggest that one type of politicization is necessarily more pernicious than the others. In fact, all types of politicization increased negative emotional responses and confusion, both with respect to the health topic in question but also on related domains, although policy support remained unchanged. The results also suggest that inoculation may sometimes blunt the adverse effects of exposure to politicization, although testing additional strategies is needed, given the increasing frequency of politically-charged and controversial health messaging in the media ecosystem.

The science of guidelines and updating health recommendations

The scientific enterprise depends upon the accumulation of knowledge, which is acquired through testing and retesting different hypotheses and systematic syntheses of existing studies. As new technologies and methods are developed, the evidence base guiding procedures and recommendations for whom, when and how new innovations should be implemented (or de-implemented21, 22) and used grows. Of course, who interprets the evidence and how recommendations and guidelines are established is a social process that can be shaped by politics. In the context of health care, although the United States has been comparatively slow to implement evidence-based medicine recommendations into public policy in large part due to politics and a lack of public understanding,24, 25, 26, 27 guidelines for medical care recommendations have been long been established by professional organizations and sometimes by panels of independent medical experts who are convened for specific reasons. For example, the Advisory Committee on Immunization Practices (ACIP) was established in 1964 by the Surgeon General to provide independent, outside advice to the Centers for Disease Control, and the U.S. Preventive Services Task Force was created in 1984 to examine evidence and develop recommendations for primary and preventive care.

As research accumulates and evolves, the recommendations and guidelines experts follow should be expected to change in accordance with the body of evidence, and yet the way in which guidelines are updated and described in public discourse can influence acceptance of the recommendations. More specifically, the public is less trusting of expert panels described to be government sponsored and less likely to support politicians who challenge medical experts on the appropriateness of particular treatments even if the evidence supports the politician's argument. While some recommendations like the push to move colorectal cancer screening earlier (i.e., from 50 to 45) have largely been accepted without public pushback or scrutiny, others have been sparked with controversy and concerns over politicization.

Concerns over politicization, dimensions of the concept, and strategies for mitigation

Politicization concerns and consequences

Although some scholars argue that science is inherently political and uncertain and therefore politicization will always be a feature of public communication about these issues—especially in the realm of public health topics3, 29—others argue that there are significant concerns related to the politicization of health and science issues, especially when it is a prominent feature of public discourse surrounding scientific recommendations. In particular, prior research has articulated concerns and/or documented evidence demonstrating politicization leads to a host of adverse responses including: a lack of consensus over proper policy action and support for science, decreased trust in the medical profession and doctors and in science overall, along with jeopardizing funding for basic scientific research. These warnings led to a growing body of work attempting to mitigate the adverse consequences of politicization,30, 31 even before the COVID-19 pandemic served to highlight just how deadly the consequences of politicized science and polarized partisan reactions really are.7, 14, 32, 33

Several earlier studies about the adverse influence of politicization have generated evidence that it increases negative emotional reactions, heightens doubt about scientific claims, and decreases the uptake of beneficial technologies or policies in the public interest. For example, prior work has shown that politicization messaging increases anxiety and perceptions of threat. Studies have shown that news coverage providing equal coverage to opposing views of scientific claims, whether on climate science or other health issues, results in increased uncertainty about the veracity of scientific claims. In other cases, politicized framing of news can result in reductions of support for nuclear power, hydraulic fracturing and carbon nanotubes, and for requiring the HPV vaccine. The politicized nature of the discussion surrounding the Affordable Care Act has even contributed to differential uptake of health insurance based on enrollees' party identification, suggesting that partisan reactions can have real impacts on health.37, 38, 39

Concerns about potential carryover effects of politicization

Exposure to politicized messages might produce effects not only on emotional responses to and public attitudes about the health topics in question, but also on more general attitudes about scientific guidelines and recommendations. Such spillover or “carryover” effects have been observed in response to exposure to health-related conflict and controversy. For example, exposure to conflicting information about one health topic (e.g., mammography) has been shown to influence cognitive and behavioral responses about other health topics (e.g., ambivalence not only to mammography but also other types of cancer screening); such exposure also can affect cognitions that are not topic-specific (e.g., attitudes toward health research in general).41, 42 There is even some evidence that these cognitive responses to conflict can decrease receptivity to subsequent unrelated health messages, including those about behaviors for which there is broad scientific consensus (e.g., fruit and vegetable consumption, physical activity). This body of research suggests that politicized health content—especially when characterized by controversy—could carry over to other health topics, thus jeopardizing a much broader set of preventive medicine and public health recommendations whose effects might be felt long after the COVID-19 pandemic.

Dimensions of politicization

Although numerous pieces of scholarship have examined the causes and consequences of politicization of health and science,1, 3, 6, 7, 10, 13, 18, 35, 43 how politicization is defined and operationalized (if at all) differs from study to study. In particular, prior work19, 20 has identified at least three dimensions to how politicization in media messaging is referenced in the literature: as disagreement and controversial discussion of science in the public sphere; as amplification and exploitation of the inherent uncertainty of science in order to cast doubt on guidance; and as the inclusion and incorporation of partisan, political conflict in discussions of health and science topics.18, 36 What is commonly held between all of these articulations is the notion that politicization occurs in public discussion over health and science recommendations, and that the communication surrounding these issues is contested. Although disagreement and controversy is one specific dimension or operationalization of politicization in media, both of the others (amplifying scientific uncertainty and incorporating partisan conflict) add additional layers of messaging on top of the emphasized feature of controversy. In other words, scientific uncertainty and partisan dimensions of politicization might be thought of as additional features layered atop controversial depictions of issues.

Does the type of politicization matter for the effects we see on emotions, public attitudes and behaviors? Because most of the literature utilizes one of the dimensions (if it does define one at all), we know little about how the different types of dimensions might differently influence public reactions. Therefore, this study attempts to test the variety of different effects that politicization has been shown to have on affective, cognitive and policy responses and to assess whether any one type is particularly problematic compared to the others and to a control condition that describes the topic in question without politicized dimensions. Because there is no prior work comparing the effects of different types of politicization, we view our work here as primarily exploratory, but to the extent that both of the uncertainty and political dimensions build on controversy, one might expect that the effects could be stronger for either of those types as compared to controversy alone.

Messaging strategies to mitigate politicization

Given the growing literature on the adverse effects of politicized messaging in the public sphere, there has also been burgeoning work on strategies to counteract it.6, 30, 31, 44 A range of possible tactics to mitigate politicization have been explored and include warnings in advance or “inoculation” messages,6, 45, 46 corrections after prior exposure,6, 47 visual imagery, self-affirmation, and the provision of specific information on causes of scientific findings. Although many of these strategies may merit examination with respect to individual dimensions of politicization, we focus here on the first: inoculation messages or warnings in advance of politicized messages. Inoculation has been conceptualized as a persuasion strategy that includes a weakened or refuted version of a message (that is to say, a weaker form of a later message, to apply the vaccination metaphor) that helps individuals to develop psychological resistance to a subsequent communication.48, 49 In a previous study related to public health messaging (i.e., soda taxes), this was conceptualized as a warning about the motives of a communicator and preemptive refutation of that messenger's argument. In work specifically on politicization (defined as strategic emphasis of scientific uncertainty), it has been conceptualized as a warning that scientific consensus exists and future politicization should be disregarded. The key feature of an inoculation-style message is that it is delivered in advance of another message, in order to increase the likelihood that people resist being persuaded by (or adversely affected by, in the case of the current study) a subsequent communication. To our knowledge, such a messaging strategy has not been studied in the context of comparing its effectiveness in light of differing types of politicized communication, but we expect inoculation to help blunt the effects of politicization.

Study objectives

While the dire nature of the consequences of politicization generally may have been clarified with the onset of the pandemic and the concomitant of polarized responses among the public,7, 14 what is not yet clear is whether the type of politicization matters. As described earlier, prior work differs in the definition and operationalization of politicization, using at least three distinct dimensions of the concept. In particular, it is not clear whether disagreement and controversy itself is the most damaging or whether exploiting the uncertainty of science to create controversy vs the intrusion of partisan cues in controversy over science are worse. As such, we designed a pilot study in the context of HPV vaccine recommendations to determine whether we could uncover differential effects from each of the three distinct types of politicization across a range of outcomes, including affective responses, cognitive responses, and support for policies. In addition, because we conducted the study during the early stages of the pandemic, which was fundamentally characterized by extensive politicization in the United States, we also decided to examine the extent to which exposure to additional politicization of mask-wearing recommendations surrounding COVID-19—operationalized as the presence of all three dimensions, as by July 2020 all three were already indistinguishable in COVID-19 discourse—would influence the same types of outcomes. Thus our study examines the consequences of politicization messaging across two distinct cases, the HPV vaccine and COVID-19, and we examine outcomes that encompass both domain-specific attitudes (i.e., emotional arousal, confusion, backlash, and policy support related to the assigned topic), as well as potential carryover attitudes about vaccination more generally (i.e., confusion on other issues as well as policy support for other issues).

Following the prior literature, we expect that all types of politicization conditions will increase negative emotions, decrease policy support and increase confusion and backlash regarding the specific domain, and that these effects might carry over to related domains (i.e., increase carryover confusion and decrease support for policies beyond the specific domain). Is one type of politicization in messaging worse than another? As mentioned above, we do not have strong expectations about which dimension of politicization may be the most harmful. To the extent that the uncertainty and partisan domains add on additional layers to controversy, it might be reasonable to suggest that both could amplify the adverse effects compared to controversy, but our goal is simply to explore the effects of each type. Therefore, we frame the comparison of different dimensions as a research question: Are the dimensions of politicized messaging different from one another in their influence?

Finally, as noted above, there has been little research that examines strategies to help the public combat or mitigate any adverse effects of different types of politicized messaging. Thus, we also included an initial test of an inoculation strategy to preemptively help people consider that politicization might be a strategic construction of health issues, and that they do not have to be persuaded by it. As mentioned above, we hypothesize that inoculation will help to mitigate potential adverse effects—that is, relative to exposure to politicized messaging alone, it will lower negative emotional reactions, increase policy support and decrease confusion on the domain in question and will increase policy support and decrease confusion on potential carryover topics—of politicized communication. One important caveat to these expectations is that pretreatment, or citizens' prior exposure to information similar to any treatment language, can affect subsequent responses to experimental conditions. Because politicization was already baked into much of the communication surrounding COVID-19 in particular, there is reason to believe that people may not be as responsive to an additional dose of politicization, as they may have already adjusted their attitudes and beliefs in response to earlier messaging.

We discuss each of the features of the study design below.

Data and methods

Study design

In this analysis, we draw on a survey experiment of 3012 respondents,a which was fielded online between June 30 and July 6, 2020, by Dynata on a national sample of U.S. adults. In particular, the sample was designed to be representative of U.S. adults 18 years or older with recruitment designed to achieve U.S. Census estimates on two characteristics: age and gender. Participants were randomly assigned to a topic first (either HPV vaccine recommendations or mask-wearing recommendations surrounding the spread of COVID-19) and then to either receiving a news article with information about updated scientific recommendations on that topic (hereafter called base level information or base information) or to a news article with the same base information about recommendations that also included content containing differing forms of politicization (see Table 1 for headlines and inoculations by topic and see the Appendix for the complete treatment language).

Table 1

Headlines and inoculation treatments for each of the experimental study conditions.

	HPV vaccine headlines	COVID-19 headlines	Inoculation
Base	Panel to Expand HPV Vaccine Recommendations	CDC Updates Recommendations on Mask Wearing
+Controversy	Controversy Erupts Over New HPV Vaccine Recommendations		…people…try to make health recommendations seem more controversial than they really are…. They create drama to draw attention away from the large amount of agreement…
+Controversy +Uncertainty	Advocates Cast Doubt on Scientific Evidence Behind New HPV Vaccine Recommendations as Controversy Erupts		…people…try to make health recommendations seem more controversial than they really are, saying that the science is more uncertain than it really is…. They create drama and exaggerate uncertainty to draw attention away from the large amount of scientific agreement…
+Controversy +Political	Political Pushback to New HPV Vaccine Recommendations as Controversy Erupts		…politicians…try to make health recommendations seem more controversial than they really are…. They create drama for political gain to draw attention away from the large amount of bipartisan agreement…
All Politicized Elements		Controversy and Political Pushback Over CDC Mask Wearing Recommendations in Light of Doubts About Scientific Evidence	…politicians or other people…try to make health recommendations seem more controversial than they really are, saying that the science is more uncertain than it really is or that there is more division between Republicans and Democrats than there really is…. They create drama and exaggerate uncertainty to draw attention away from the large amount of scientific and bipartisan agreement…

Note: News article treatment language varied between the two topics. Each of the politicized conditions added additional language (in addition to the headline changes listed in the table) to the base information to help emphasize the particular treatment which included discussion of heated debate and disagreement and a quote.

The base information for both the HPV vaccine and for the COVID-19 condition contained news about updated recommendations in light of evidence (or the accumulation of additional data as would be expected in the normal progression of science). More specifically, the HPV vaccine condition (labeled as an August 2019 news article) stated that the Advisory Committee on Immunization Practices (ACIP) recommended that men and women up to age 26 receive a “catch-up” vaccine if they missed the shots in pre-adolescence, whereas the COVID-19 condition (labeled as a June 2020 news article) provided information about the updated recommendations from the CDC advising that people wear masks in public. In all politicization treatments, we modified the headline of the news article to emphasize the specific aspect of politicization and added both a paragraph describing heated debate along with a quote highlighting the politicization to emphasize the disagreement.

Specific details of the treatments varied by topic. In the case of the HPV vaccine, we tested three separate politicization conditions: (1) one that exposes participants to disagreement and controversy surrounding the updated recommendations, as well as a “local advocate” summing up the concerns (the Base + Controversy condition, hereafter controversy for short), (2) one that adds explicit discussion of the uncertainty of the science underpinning the recommendations, as well as a “local advocate” questioning the science (the Base + Controversy + Uncertainty condition, hereafter uncertainty for short), and (3) one that adds political discourse, specifically a “longtime politician” weighing in, questioning, and highlighting the controversy over the recommendations (the Base + Controversy + Political condition, hereafter political for short). In the case of COVID-19, we were fielding the study during the early stage of the pandemic when attention to news about coronavirus was especially high and politicization of health recommendations had already begun. Therefore, we decided that we could not reliably assess the different dimensions of politicization given the interference of pretreatment messaging, which likely varied in ways we could not adequately control posthoc. Therefore, we only tested one combined politicization condition that contained all three elements (controversy, uncertainty, and political discourse, each part of the broad public discourse surrounding masking as a prevention strategy) as a way of understanding how the politicization emphasis in news coverage affected responses. In the COVID-19 politicization treatment, similar to the HPV vaccine, we added a paragraph on heated debate along with a quote that further emphasized it along with the headline. The headlines for each of the six conditions are listed in the middle two columns of Table 1 (and for the full language, see the Appendix).

Finally, we randomly assigned half of respondents who were not assigned to the base condition to view an inoculation treatment in advance of receiving the news media article. The initial prompt for inoculation treatments said, “On the next page, you will see a statement followed by a recent news article published in a local paper. Please take a moment to read it and you will be asked a few questions about it afterward.” As shown in the far right column of Table 1, the inoculation language warned of actors emphasizing controversy, controversy and uncertainty, or controversy and political influence (depending on the assigned politicization dimension for HPV vaccine; or all of the above elements for COVID-19); the message strategy here was to help respondents refute or dismiss the impact of these politicized communication dimensions when they see them in the news article. Finally, Table 2 provides an overview of the sample size for each of the experimental design conditions.

Table 2

Experimental conditions and sample size by topic.

	HPV vaccine		COVID-19
	No Inoc.	+ Inoc.	No Inoc.	+ Inoc.
Base	296		303
+Controversy	305	308
+Controversy +Uncertainty	286	303
+Controversy +Political	300	296
All Politicized Elements			302	313
Total N = 3012	2094		918

Measures

As described above, based on previous research examining the effects of exposure to politicized messages and conflicting information (a related, but not identical, concern as politicization), we conceptualized five sets of outcome measures, adapted from previous studies6, 16, 20, 36, 40, 51: negative emotional responses; support for the policy action in the specific domain; confusion or perceived ambiguity about the specific topic, backlash or negative beliefs toward the specific topic, and a set of potential carryover effects—in this case, general confusion about broader topics than those featured in the experimental treatment, as well as vaccine-related attitudes distinct from the health topic in question.

Negative emotional responses

We measured emotional reactions by asking respondents to indicate how they feel after having read the news article with respect to five different emotions: interested, frustrated, surprised, annoyed and distressed. Given our a priori interests in negative emotional responses and consistent with past research,16, 40 we focused on the three negatively-valenced items (frustrated, annoyed, and distressed). Response options ranged from “very slightly or not at all” (1) to “extremely” (5). The distressed item has mean of 2.05 and a standard deviation of 1.19. As the frustrated and annoyed items scaled reliably (reliability coefficient = 0.82), we combined them into an irritation index (mean = 2.16; SD = 1.14).

Support for domain specific policy action

With respect to domain-specific policy support, we utilized a question from prior literature in the HPV vaccine context: “How much do you support or oppose a state law requiring all children in middle school to get the HPV vaccine with opt-outs allowed as determined by your state?” Response options ranged from (1) strongly oppose to (7) strongly support (mean = 4.44; SD = 1.82). In the case of COVID-19, we asked two questions about how much the participant supports or opposes “local governments requiring people to wear masks in public” and “businesses (e.g., stores, restaurants) requiring people to wear masks in their establishments.” Both measures ranged from (1) strongly oppose to (7) strongly support. While we had originally conceived that the public may differ in their attitudes toward government mandates versus mandates in the private sector, because the alpha scale reliability coefficient is 0.91, we created one index of the two, which also ranges from 1 to 7 (mean 5.91; SD = 1.72).

Confusion and backlash

To assess HPV vaccine-related confusion, we adapted two items from past research16, 40: “I am not sure whether getting an HPV vaccine is good or bad for a young person's health” and “I find information about getting an HPV vaccine to be confusing.” We combined them into one index, which had a reliability coefficient of 0.77. Response options ranged from (1) strongly disagree to (7) strongly disagree (mean = 3.66; SD = 1.60). Similarly, we adapted two items to assess COVID-19-related mask-wearing confusion: “I am not sure whether mask wearing is an effective or ineffective strategy for preventing COVID-19 (coronavirus) spread” and “I find information about mask wearing effectiveness to be confusing.” The scale reliability for the index was 0.91 with response options from (1) strongly disagree to (7) strongly agree (mean = 3.26; SD = 1.62).

To assess HPV vaccine-related backlash, we adapted two items from past work16, 40: “Scientists really don't know whether getting an HPV vaccine is good or bad for a young person's health” and “Recommendations about getting an HPV vaccine should be taken with a grain of salt” into one index (reliability coefficient 0.81). Response options ranged from (1) strongly disagree to (7) strongly agree (mean = 3.56; SD = 1.59). Two similar items were used to assess COVID-19-related mask-wearing backlash: “Scientists really don't know whether mask wearing is an effective or ineffective strategy for preventing COVID-19 (coronavirus) spread” and “Recommendations about wearing a mask should be taken with a grain of salt,” which we combined (reliability coefficient 0.81). Response options also ranged from (1) strongly disagree to (7) strongly agree (mean 3.21; SD = 1.81).

Potential carryover effects

We assessed potential carryover effects in two ways. First, given the context of the HPV vaccine topic, we assessed generalized confusion about childhood vaccinations: “I am not sure whether getting childhood vaccinations is good or bad for children's health.” We asked a similar item in the context of COVID-19 to assess broader confusion than just about masks as mitigation: “I am not sure whether there are effective strategies for preventing COVID-19 (coronavirus) spread” (both items had response options from (1) strongly disagree to (7) strongly disagree; HPV vaccine mean = 3.17; SD = 1.79, and COVID-19 mean = 3.52; SD = 1.92). Second, we examined whether there were potential carryover effects on support for immunization program requirements, adapting a set of items from past work. In the HPV vaccine context, this included a five item index of responses to: “Required childhood vaccinations important for protecting the American public from disease,” “Government should have authority to require vaccines,” “Vaccines protect children's health,” “Vaccines protect adults' health,” and “Vaccines are safe.” Three of these items were used to create an index to measure support for adult immunization programs in the COVID-19 context: “Government should have the authority to require vaccines,” “Vaccines protect adults' health,” and “Vaccines are safe.” Response options for each ranged from (1) strongly disagree to (7) strongly agree. The scale reliability for the general childhood immunization index was 0.92 (mean = 5.30; SD = 1.40) and it was 0.84 for the general adult immunization index (mean = 5.06; SD = 1.54).

Analytic approach

We used linear regression modeling to compare the politicization conditions to the base condition for each topic and difference of means t-tests to assess whether the inoculation conditions decrease adverse reactions compared to the politicization treatments without inoculation messaging. We report one-tailed tests for our directional expectations (adverse effects of exposure to politicization relative to the base conditions and protective effects of inoculation relative to politicized messaging alone) but two-tailed tests for the comparison of each politicized dimension compared to each other in the HPV vaccine case. All analyses were conducted with Stata SE, version 14.2.

Results

Is one particular type of politicization more damaging than the others? Or do they all operate similarly with respect to their effect on negative emotions, policy support, confusion and backlash, and carryover responses? And does inoculation provide protection against adverse effects of different dimensions of politicization?

Negative emotional reactions to politicization and inoculation

We begin with an assessment of negative affective responses, which are perhaps most proximal to the treatment. Prior work suggests that exposure to conflict and politicization (defined as uncertainty) arouses negative emotions.6, 16, 40 We find that all dimensions of politicization increase negative emotional reactions. As shown in the left panel of Fig. 1 , all three types of politicization in the HPV vaccine case increase respondent irritation compared to those only exposed to base information, although there is no meaningful difference between the three types. In addition, respondents exposed to all three types of politicization also report more distress than those exposed to only the base information (see the right panel of Fig. 1); however, only the controversy and political dimensions distress scores are statistically significantly different from the base treatment (P  = 0.026 and P  = 0.048, respectively). In comparing the three conditions to each other, however, we cannot distinguish between the effects of controversy, uncertainty and political, suggesting that none are particularly worse than the other.

Fig. 1

Effects of politicization treatments on negative emotional reactions (HPV vaccine).

Turning to emotional reactions to politicization in the case of COVID-19, we find strong evidence for both an increase in irritation and an increase in distress for respondents exposed to all three combined elements of politicization, as shown in Fig. 2 . There are two noteworthy observations regarding the difference between responses to the HPV vaccine and the COVID-19 cases. First, respondents reported greater negative affect (as reflected in both irritation and distress) after simply reading the updated guidelines regarding masks (the base treatment) than they did after reading politicized versions of the HPV vaccine treatments: The mean values are 2.2 and 2.2, respectively, for irritation and distress for the COVID-19 base treatment, compared to values that range from 2.1 to 2.2 and 2.0 to 2.1, respectively, for the HPV vaccine politicized treatments. Second, the negative emotional reaction to politicization (operationalized through all three dimensions) was much larger on both irritation and distress (mean values of 2.7 and 2.4) than it was for any one of the dimensions with respect to the HPV vaccine (all mean negative emotional responses were below 2.2).

Fig. 2

Effects of Politicization treatments on negative emotional reactions (COVID-19).

Can we successfully blunt the negative emotional effects of politicization with a warning prior to exposure? Table 3 provides mixed and unexpected evidence. More specifically, in comparing the respondents' emotional reactions in the treatment conditions (without inoculation) to those who received inoculation warnings prior to treatment, we find in every case except one (the controversy treatment's effect on distress) that exposure to a warning intended to inoculate respondents against the effect of politicization actually increased negative responses. It is important to acknowledge that most of the mean differences between those receiving inoculation and those who did not receive them are not statistically significant; however, in both COVID comparisons, we find a meaningful difference in irritation and in distress above the levels reported by respondents in the politicization condition alone. Furthermore, posthoc t-tests between the conditions (not shown in in Table 3) with and without inoculation actually suggest that three of the inoculation conditions were statistically higher in negative affect compared to respondents assigned to conditions without inoculation (the political with inoculation condition for the HPV vaccine treatment on distress and the combined politicization with inoculation COVID treatment conditions for both irritation and distress) and three more were marginally significant at the P  < 0.1 level (controversy and political inoculation conditions on irritation and uncertainty inoculation on distress, which were P  = 0.08 for all three).

Table 3

T-test comparisons of inoculation warnings on negative emotions compared to politicization conditions alone.

		Irritation			Distress
		Mean	SE	p	Mean	SE	p
HPV Vaccine Conditions	Controversy			0.92			0.21
	No Inoc.	2.14	0.065		2.08	0.070
	+ Inoc.	2.27	0.066		2.00	0.067
	+Uncertainty			0.78			0.92
	No Inoc.	2.13	0.065		1.98	0.068
	+ Inoc.	2.20	0.063		2.11	0.066
	+Political			0.92			0.97
	No Inoc.	2.18	0.070		2.05	0.070
	+ Inoc.	2.32	0.066		2.24	0.073
COVID-19	Politicized			0.99			0.99
	No Inoc.	2.70	0.074		2.39	0.074
	+ Inoc.	2.95	0.073		2.65	0.077

Note: One-tailed tests that inoculations are lower than regular treatment cells are reported.

Domain specific policy support responses to politicization and inoculation

Prior research has shown decreases in support for relevant policy actions (such as state vaccine requirements) as a consequence of exposure to politicization.20, 36 Therefore, we expected that exposure to politicization would decrease support for HPV vaccine requirements in the case of the HPV vaccine and support for requiring masks in public places and businesses with respect to COVID-19. We find neither, as Fig. 3, Fig. 4 show, respectively. Although all politicization treatments—across the three dimensions in the context of HPV and in combined form in the COVID-19 case—are negatively signed, there is no statistically significant difference between the base level conditions and the politicized ones at standard levels of significance in a one-tailed test. Although the controversy condition is just outside the standard cut-off (P  = 0.071) in a comparison to the base condition, similar to the investigation with respect to emotions, we cannot statistically distinguish between the politicized dimensions in levels of policy support.

Fig. 3

Effects of politicization treatments on HPV vaccine requirements.

Fig. 4

Effects of politicization treatments on COVID-19 mask requirements.

Although exposure to inoculation caused elevated negative emotional reactions among respondents, as described above, the pretreatment warnings do not seem to have significant effects on support for domain-specific policy action (see Table 4 ). Interestingly, across all four politicization comparisons, the inoculated conditions have higher mean support for policy action, which is the expected direction if inoculation were helping to blunt the negative effects of politicization; however, none are statistically significant by standard levels (the uncertainty condition P-value in the one-tailed test was 0.092, however).

Table 4

T-test comparisons of inoculation warnings on policy support compared to politicization conditions alone.

		Support for domain-specific mandates
		Mean	SE	p
HPV Vaccine	Controversy			0.40
	No Inoculation	4.30	0.104
	+ Inoculation	4.34	0.106
	+Uncertainty			0.092
	No Inoculation	4.41	0.106
	+ Inoculation	4.60	0.098
	+Political			0.21
	No Inoculation	4.38	0.109
	+ Inoculation	4.5	0.106
COVID-19	Politicized			0.19
	No Inoculation	5.61	0.100
	+ Inoculation	5.73	0.088

Note: One-tailed tests that inoculations are higher than regular treatment cells are reported.

Confusion and backlash responses to politicization and inoculation

Turning to confusion and backlash, we find strong evidence that exposure to any type of politicization increases confusion about the HPV vaccine (Fig. 5 , left panel). Although the political condition has the highest mean, it is not statistically significantly different from either of the other two politicized conditions, suggesting that all three elicit similar responses. We do not find evidence that politicization causes backlash with respect to science at standard significance levels, although all three of the politicization conditions are positively signed—the expected direction if politicization were increasing backlash (Fig. 5, right panel)—and the political condition is marginally higher (P  = 0.85 in the one-tailed directional test compared to the base condition).

Fig. 5

Effects of politicization treatments on confusion and backlash (HPV vaccine).

We find similar patterns as shown in Fig. 6 for both confusion and backlash (positive coefficients signaling an increase in both) in the context of the COVID-19 case, although the increase in backlash is just outside the standard cut-off for significance (P  = 0.055).

Fig. 6

Effects of politicization treatments on confusion and backlash (COVID-19).

We do not find much support for the notion that inoculation could blunt the increased confusion, as none of the t-tests comparing means between the politicized treatments with and without inoculation are significant at standard levels. As shown in Table 5 , we only find marginal support for a decrease in the political condition of the HPV vaccine at P  < 0.1 and in no other case. Therefore, we can mostly reject the hypothesis that inoculation can defend against increased confusion.

Table 5

T-test comparisons of inoculation warnings on confusion and backlash compared to politicization conditions alone.

		Confusion			Backlash
		Mean	SE	p	Mean	SE	p
HPV Vaccine	Controversy			0.66			0.81
	No Inoc.	3.72	0.088		3.59	0.087
	+ Inoc.	3.77	0.090		3.70	0.091
	+Uncertainty			0.30			0.25
	No Inoc.	3.70	0.094		3.53	0.089
	+ Inoc.	3.63	0.93		3.45	0.92
	+Political			0.095			0.18
	No Inoc.	3.79	0.096		3.65	0.097
	+ Inoc.	3.61	0.092		3.53	0.094
COVID-19	Politicized			0.18			0.44
	No Inoc.	3.92	0.096		3.29	0.102
	+ Inoc.	3.26	0.108		3.27	0.108

Note: One-tailed tests that inoculations are lower than regular treatment cells are reported.

Potential carryover effects of politicization and inoculation

Turning to potential carryover effects, we first examine the extent to which confusion over politicization in one domain (HPV vaccine or masks) spills or carries over to influence confusion more broadly. As shown in Fig. 7 , we find that the coefficients for exposure to all types of politicization (each dimension with respect to the HPV vaccine treatments as shown in the left panel and the combined dimension for COVID-19 on the right panel) are positive, indicating increased confusion over “whether getting childhood vaccinations is good or bad for children's health” in the case of the HPV treatment and “whether there are effective strategies for preventing COVID-19 (coronavirus) spread” in the case of the COVID-19 masks treatment. While the increase in confusion carryover is statistically significant in the COVID-19 case, only the political cues operationalization of politicization in the HPV treatments is significant at standard levels. When we compare the political condition directly to the other two politicized dimensions, however, the levels are not statistically different from each other, even at the P  < 0.1 threshold in a two-tailed test. Turning to the COVID-19 context, we see that the politicized condition is statistically higher than the base case in confusion carryover.

Fig. 7

Effects of politicization treatments on carryover confusion (HPV vaccine and COVID-19).

In comparing the conditions with and without the inoculation messaging for both the political dimension for HPV and all elements of politicization for COVID-19 alone, we find some evidence that inoculation may blunt the increase in confusion (Table 6 ). With inoculation, the HPV political condition mean is 3.1 compared to 3.4 without it, which is statistically significant in a one-tailed test for a decrease (P  = 0.03). These results suggest that although the political condition may increase confusion, it does appear to be the case that inoculation against political controversy can mitigate the effects. In the case of COVID-19, the mean for the politicization treatment with inoculation is also lower than the mean without inoculation (3.5 compared to 3.7) although the one-tailed test is outside of traditional standards for significance (P  = 0.08).

Table 6

T-test comparisons of inoculation warnings on carryover confusion compared to politicization conditions alone.

		Carryover confusion
		Mean	SE	p
HPV Vaccine	Controversy			0.87
	No Inoculation	3.16	0.095
	+ Inoculation	3.31	0.103
	+Uncertainty			0.36
	Treatment	3.16	0.105
	with Inoculation	3.11	0.101
	+Political			0.029
	No Inoculation	3.36	0.109
	+ Inoculation	3.07	0.104
COVID-19	Politicized			0.077
	No Inoculation	3.72	0.108
	+ Inoculation	3.50	0.112

Note: One-tailed tests that inoculations are lower than regular treatment cells are reported.

To examine another potential marker of carryover effects, we ask whether reading about politicization decreases support for immunization programs more broadly. As shown in Fig. 8 , exposure to politicization (whether one of the dimensions in the HPV vaccine context or to the all version in the COVID-19 context) consistently results in negative coefficients, suggesting that it lowers support for immunization programs. However, only the HPV vaccine politicization condition operationalized as controversy alone is significant at standard levels in a one-tailed test for a decrease (P  = 0.04). (Similar to earlier analyses, however, this condition is not statistically distinguishable from the uncertainty or political dimension results).

Fig. 8

Effects of politicization treatments on carryover support for immunization program requirements (HPV vaccine and COVID-19).

Turning to the effect of inoculation messaging, none of the inoculation comparisons are statistically different from their counterparts without inoculation, even though all conditions with inoculation have a mean of immunization program support that is greater than the conditions without inoculation, the expected direction for protective effects (Table 7 ).

Table 7

T-test comparisons of inoculation warnings on carryover immunization support compared to politicization conditions alone.

		Immunization support
		Mean	SE	p
HPV Vaccine	Controversy			0.25
	No Inoculation	5.21	0.079
	+ Inoculation	5.29	0.077
	+Uncertainty			0.23
	No Inoculation	5.23	0.086
	+ Inoculation	5.31	0.079
	+Political			0.30
	No Inoculation	5.29	0.084
	+ Inoculation	5.35	0.082
COVID-19	Politicized			0.11
	No Inoculation	4.97	0.091
	+ Inoculation	5.12	0.089

Note: One-tailed tests that inoculations are higher than regular treatment cells are reported.

Discussion

In this analysis, we find that exposure to media messages emphasizing politicization increases negative emotional reactions with respect to irritation and distress in particular. Reading about COVID-19 generally was particularly likely to increase negative emotional arousal; reading politicization messages on this high salience issue was even more likely to exacerbate these negative emotions. Although our study does not identify evidence of domain-specific decreases in policy support as a result of politicization messages, we do find support for the notion that politicization increases confusion both on the topical domain in question and in potential carryover domains (that is, confusion about broader health recommendations). For the latter, our findings suggest that politicization could have a deleterious influence on broader health-related attitudes. Although our results are consistent in sign with prior work finding that exposure to conflicting evidence about health produces heightened backlash, we do not find support for substantial increases in backlash as a result of any type of politicization in this study. Similarly, although results are in the expected direction, we find little support for potential carryover effects on broader policy attitudes (support for general immunization programs, in this case), which is a reassuring finding although certainly not definitive evidence on the matter.

One goal of this large pilot study was to assess whether, through an experimental design, the various conceptual ways scholars have operationalized politicization have any differential effects. Previous observational studies have shown that the public does distinguish among controversy, evidence uncertainty, and political discourse or cues, and that perceptions of evidence uncertainty are correlated with reduced policy support.19, 20 However, experimental evidence is needed to evaluate the causal effects of these dimensions. By and large this pilot study does not suggest that any one operationalization of politicization is more or less damaging than the others. Rather, the findings suggest that all types of politicization are equally concerning, particularly as judged by their effects on negative emotions and confusion. There are some instances in which the political operationalization (the reference to a politician) elicits the strongest reaction compared to the base information, albeit not statistically different from the other dimensions, and thus the presence of political officials in health media coverage is worthy of future exploration. Given that politicization of the COVID-19 pandemic was well underway at the time of fielding, it seems especially important to acknowledge that we still uncovered effects of additional exposure to the experimental politicization message, regardless of the extent to which some respondents may have been pretreated with exposure to politicization before taking our survey.18, 50

One important innovation of this study relative to prior work is that we tested a potential message strategy to mitigate any adverse effects of exposure to media messages about politicization that was tailored to the particular dimension of politicization. Can providing such a tailored warning prior to exposure help to reduce some of its negative effects? We find some evidence that inoculation can help blunt the adverse effects of exposure to politicization, especially with respect to carryover confusion. However, it is notable that we largely do not find that inoculation can help mitigate confusion on the topic-specific domain. In addition, it is important to acknowledge that respondents became more agitated when we provided a warning in advance of exposure to politicization, suggesting that this particular inoculation strategy is not a costless intervention, in that it heightened irritation and distress among participants.

Our analysis is not without limitations. First and foremost, our sample—although recruited nationally (rather than in just one geographic area) and diverse in demographic characteristics like age, gender and race—is not nationally representative. The sample (see Table A1 in Appendix) is somewhat more educated than average, and because the survey was fielded exclusively online, respondents all had Internet access. These characteristics may affect our ability to generalize the findings to the broader population. However, because we used a randomized experimental design, the sample composition does not affect our ability to detect causal effects of our politicization and inoculation treatments. Second, although we expect that several factors would moderate the effects of politicization (including education, attitudes toward science, and partisanship/ideology, to name a few), analyzing potential subgroup effects is beyond the scope of the current pilot investigation. We did not power this pilot study to ensure sufficient numbers within subgroups; we conceptualized this mainly as a pilot of the main effects of politicization dimension treatments and inoculation. Further, as noted above, given we fielded the experiment during the summer of 2020 in the heat of the early days of the COVID-19 pandemic, pretreatment exposure to politicized controversy, amplification of scientific uncertainty, and partisan intrusion in science was likely a broad influence on respondents. It also could have been unequally distributed among respondents in ways that could have affected our experiment (i.e., distributed according to media exposure, or by trust in science) since we cannot easily test for randomization conditioned on prior exposure. If anything, however, we think these factors may have made it harder to detect effects of an additional treatment in our study. Finally, our political cues treatments only included a reference to “politicians,” and we did not provide a specific partisan cue. However, we would anticipate that inclusion of partisan cues may be particularly potent influences on respondents based on past research.52, 53 (Although, evidence is uncertain on the power of partisan cues on COVID-19 attitudes in particular, perhaps because the partisan characteristics of COVID-19 were already so well understood in public discourse). Regardless, future experimental tests of the various dimensions of politicization should consider partisan cueing in particular, to test for uniform or asymmetric effects of partisan rhetoric.

Table A1

Demographics of the survey experiment sample

			Number of respondents	Percentage
Gender		Female	1616	53.7%
Age		18–24	355	11.8%
		25–34	465	15.5%
		35–44	547	18.2%
		45–54	522	17.3%
		55–64	468	15.6%
		65 +	653	21.7%
Race		Nonwhite	1073	35.6%
Education		HS or less	597	19.8%
		Some college	844	28.0%
		College and above	1571	52.2%

Overall, our results suggest that while different types of messages might convey a politicized context to the public in media coverage, our findings as they relate to the HPV vaccine context do not suggest that one type of politicization is necessarily more pernicious than the others. In fact, all types of politicization appear to foster negative responses among the public, particularly arousing negative emotions and confusion, on both the topics in question and on broader health recommendations. Finally, our findings offer suggestive evidence that one type of communication strategy, a warning about actors' strategic use of politicization, can sometimes and partially blunt the adverse effects of exposure to different types of politicization, although not without concerns. Future research on more types of strategies (including interventions in media, education, and other social domains) are needed given the increasing frequency of politically-charged and controversial messaging in health and science discourse.

{Base:}	{Controversy:}	{Uncertainty:}	{Political:}
Panel to Expand HPV Vaccine Recommendations	Controversy Erupts Over New HPV Vaccine Recommendations	Advocates Cast Doubt on Scientific Evidence Behind New HPV Vaccine Recommendations as Controversy Erupts	Political Pushback to New HPV Vaccine Recommendations as Controversy Erupts

{Controversy:}	{Uncertainty:}	{Political:}
Local advocate Jennifer Peters summed up the concerns: “Why are we requiring 10- and 11-year old kids to get a vaccine that they can get later as adults?”	In addition, local advocate Jennifer Peters cast doubt on the evidence provided for the recommended age to receive vaccination. “Researchers are always changing their minds about when to vaccinate, and the evidence is uncertain and can be used to support different positions. In this case, they know that the vaccine is fine for adults and yet they say that it has to be given to children to be effective.” Peters summed up the concerns: “Why are we requiring 10- and 11-year old kids to get a vaccine that they can get later as adults?”	Longtime politician and state senator Jennifer Peters sided with opponents of required vaccination, summing up the concerns: “Why are we requiring 10- and 11-year old kids to get a vaccine that they can get later as adults?”

{Base:}	{All Politicized Elements:}
CDC Updates Recommendations on Mask Wearing	Controversy and Political Pushback Over CDC Mask Wearing Recommendations in Light of Doubts About Scientific Evidence