The effect of misinformation and inoculation: Replication of an experiment on the effect of false experts in the context of climate change communication.

An important communication strategy of climate skeptics is the use of fake experts, who act as spokespersons, although they do not possess any expertise in the field. One promising approach to tackle the effect of misinformation is inoculation. Previous research focuses on the United States, and the comparably low effect sizes of previous research call for further examination and consolidation. This study aims to complement those findings with data for Germany and replicates and extends an experiment by Cook et al. with a 2 × 2 between-subjects design. Our study confirms the importance of pre-existing worldviews for climate-related attitudes. Regarding the effects of misinformation messages and most notably, the effects of inoculation messages we could not replicate the findings of Cook et al.: At least in our setting, the misinformation message and also inoculation preceding misinformation had hardly any effect on the climate-related attitudes under study.

Although a broad majority in Germany supports the fact that climate change exists, that it is mainly caused by human activity, and that it will have severe negative consequences (European Commission, 2017; IPCC, 2018; Poortinga et al., 2018), there is still a “skeptical” minority doubting either the occurrence, the anthropogenic contributions to, or the (negative) impacts of climate change and/or opposing political action to tackle this issue, be it for ideological or simply for economic reasons (Brunnengräber, 2013, 2018; Forchtner et al., 2018; Steentjes et al., 2017). The most prominent skeptical organization in Germany is the “European Institute for Climate and Energy” (EIKE). EIKE is not a scientific institute, but a society and lobby organization with scientists, politicians, and business representatives among its members (Haupt, 2020).

It has been shown that it is very difficult to correct such misbeliefs, especially when a person feels very certain about an issue and the attitude is intense and/or perceived as important (for an overview, see Johnson et al., 2005). Thus, alternative methods are needed that prevent people from forming misbeliefs, for example, by counteracting and weakening misinformation. One promising approach has been brought forward by inoculation theory, which has shown that people are less susceptible to misinformation when they are informed beforehand about possible misinformation techniques and thus inoculated against their persuasive effects (Banas and Rains, 2010).

Among others, Cook et al. (2017) have studied the effectiveness of inoculation and demonstrated the importance of free-market support as a significant intervening variable in the field of climate change communication. They found that misinformation about the scientific consensus regarding climate change had a polarizing effect on attitudes toward climate change. It decreased climate change acceptance among people who support unregulated markets and increased acceptance among people who do not support free markets. This effect could be neutralized with an inoculation message. However, the comparably low effect sizes call for further examination and consolidation. In addition, existing research focused on participants from the United States (Cook et al., 2017; van der Linden et al., 2017a). It is thus not clear (1) which effect misinformation has in contexts in which skepticism is comparably low (in Germany compared to the US setting), (2) if inoculation would also be successful there, and (3) what influence further intervening variables have (populist attitudes, political orientation).

Following these arguments, this study replicates one of the experiments conducted by Cook et al. (2017) for Germany and extends their study by including further intervening variables (populist attitudes, political orientation), for which other studies have found an influence on climate-related attitudes (for a summary, see Cook, 2016). The detailed research questions of this study are as follows: (1) What effect does misinformation have on the attitudes toward climate change? (2) Under which circumstances and for which groups can inoculation neutralize the influence of misinformation?

1. (Dis)belief in climate change and intervening variables

Several studies have shown that the perceived consensus among scientists concerning the occurrence and impact of climate change can be seen as a gateway belief that then influences a number of further attitudes toward climate change, for example, the support of binding policy regulations (e.g. Aklin and Urpelainen, 2014; Ding et al., 2011; van der Linden et al., 2015).

Therefore, one of the main aims of the skeptical countermovement is to spread misinformation about the scientific consensus, with the aim to reduce support for climate policy and finally, to prevent binding regulations (Cook, 2016). This misinformation possibly undermines the positive effect of the consensus message (Koehler, 2016; van der Linden et al., 2017a).

Misinformation can be understood as a broad term denoting any form of incorrect or counterfactual information, encompassing intentionally false information (also known as disinformation) as well as accidental forms of false information (e.g. canards) and is, therefore, related to the popular, yet politically charged term “Fake News” (c.f. Scheufele and Krause, 2019). Cook et al. (2017) took an audience-oriented perspective on the concept and defined it as “information that people accept as true despite it being false” (p. 1).

Common misinformation techniques which may also be used in the context of climate change include impersonating other accounts, producing provocative emotional content, group polarization, disseminating conspiracy theories, deflecting blame by discrediting opponents, and trolling (Basol et al., 2020; Roozenbeek and van der Linden, 2019). The use of fake experts, that is, “spokespeople who convey the impression of expertise without possessing any relevant scientific expertise” (Cook et al., 2017: 11), is a misinformation technique that is especially relevant in the context of science communication to create legitimacy. It capitalizes on the fact that a communicator’s credibility is determined by the recipients’ perception of her or his expertise and trustworthiness (Stiff, 1994). In other words, if you consider someone to be an expert, you also judge their statements to be more credible, which can play into the hands of the skeptics’ goal of sowing doubt. Due to the abstract and complex nature of most scientific problems, it is often impossible for audiences to evaluate whether a scientific expert is “real” and “reputable” in his or her field or whether this person is only being sold as an expert. Several studies have shown that the use of (fake) experts in communication about climate change by skeptics lowers the perceived consensus, the general acceptance of climate change, and trust in climate science (e.g. Cook et al., 2017; van der Linden et al., 2017a).

The perception and interpretation of consensus information (and also information about climate change, in general) are strongly influenced by existing cultural cognitions, worldviews, and values or norms (Cook, 2016; Lewandowsky et al., 2013). Thus, information is seldom processed in an objective manner. Instead, our beliefs and worldviews often guide how we interpret information. These motivated-reasoning processes may preserve already existing attitudes (Bolsen and Druckman, 2015). However, cases in which people are motivated to be accurate lead to deeper processing of information, and thus, lower the effect of possible cognitive biases (Kunda, 1990). With regard to this study, we can therefore expect that worldviews affect climate-related attitudes and the impact of misinformation. We can also expect inoculation to be most effective if the inoculation message motivates participants to engage with misinformation in a profound way.

Previous research has identified a number of factors that influence belief in or skepticism about climate change. For example, one commonly known predictor of climate skepticism is political ideology, namely conservatism (e.g. Häkkinen and Akrami, 2014; Jessani and Harris, 2018; McCright et al., 2013; Marlon et al., 2013; Tranter and Booth, 2015). This predictive power of conservatism can be explained by the aim of the conservative movement to protect the status quo (e.g. Jost et al., 2003) and conservative values, such as liberty and small government (Campbell and Kay, 2014). Therefore, conservatives are often against binding policy regulations designed to protect the environment or to mitigate climate change.

Two further factors related to political ideology that may influence climate change beliefs are the degree of support for free, unregulated markets (Heath and Gifford, 2006) and populist attitudes. Support for free markets goes along with a strong belief in market efficiency and beliefs that are linked to economic values and are often inconsistent with environmental protection (Heath and Gifford, 2006). Underlying these beliefs is the notion that the market and the economy as a whole will sort everything out, so that no political interventions are needed. Similar to the conservative worldview, the free-market ideology also opposes government regulation. “From this perspective it is more self-serving to believe that human actions are not the causes of global climate change, and therefore government regulation of industry is unnecessary” (Heath and Gifford, 2006: 52). Similarly, populist attitudes arise in opposition to the opinion of the majority and the political elites. Following the conceptualization of Mudde and Rovira Kaltwasser (2012), populism divides societies in two homogeneous groups: “the pure people” and the “corrupt elite” (p. 8). Therefore, the main elements of populist attitudes are anti-establishment, anti-pluralism, and the sovereignty of the people (see also Akkerman et al., 2013; Hawkins et al., 2012). Populist ideology in its right-wing version can be linked to climate skepticism in several ways (Lockwood, 2018; Reusswig et al., 2020). First, by framing mitigation policies as an ideologically driven elitist project which benefits the special interests of environmentalists and climate scientists at the expense of national interests and the regular guy on Main Street. Second, by framing them as further alienating political leaders from “the people” due to the issue’s complexity. Third, by framing them as threatening the status quo lifestyle. Right-wing populist ideology can also be linked to economic liberalism and then be hostile toward state interventions, in general (see, for example, Otjes et al., 2018 for European parties and Michelsen and Walter, 2014 for the early AfD in Germany).

2. Inoculation as a possible counter-measure against misinformation

As stated earlier, people’s beliefs and worldviews influence how they perceive and interpret new information or misinformation. Therefore, misinformation has an effect on climate attitudes precisely when it resonates with existing beliefs. In this context of influential beliefs, inoculation theory states that people can be protected against misinformation when they are informed beforehand about the misinformation techniques used, for example, by climate skeptics; thus, the effects of their pre-existing beliefs and attitudes can be neutralized.

Inoculation theory was first formulated by McGuire and colleagues. Their aim was to analyze how people can be protected against misinformation (e.g. McGuire and Papageorgis, 1961; for an overview, see Compton, 2013). The basic assumption of the theory is that people can be “inoculated” against misinformation when they understand beforehand the underlying mechanisms and logical fallacies that skeptical countermovements normally use in their argumentation. With these explanations in mind, it is easier for people to recognize false and misleading arguments. Empirical studies have shown that these measures are more effective at countering misinformation than the sole use of counterarguments without explaining the argumentative techniques used to sow the seeds of doubt (Banas and Rains, 2010; Bolsen and Druckman, 2015).

A prototypical inoculation message consists of two elements: The first is an explicit warning of an existing threat that should be prevented (e.g. a warning that there are arguments that cast doubt on the scientific consensus regarding climate change). The second is an explanation of the argumentative techniques used, which uncovers the logical fallacies (e.g. the use of fake experts to give skeptical arguments stronger legitimacy). The general aim of inoculation is to bring people out of their heuristic information processing (Kahneman, 2003) so that they can elaborate more on the arguments they perceive and process them more deeply.

The effects of inoculation have also been tested experimentally in the context of climate change communication (e.g. Cook et al., 2017; van der Linden et al., 2017a). van der Linden et al. (2017a) focused on the perceived scientific consensus, which they regarded to be a gateway belief, that is, a key determinant for attitudes toward climate change. In a first experiment, they identified the most convincing misinformation statement (“There is no consensus among scientists about climate change” (van der Linden et al., 2017b, p. 2)). In a second experiment, they showed for a US sample that an inoculation treatment can preserve the positive effects of a consensus message in the face of misinformation. However, they only included party affiliation as an intervening variable, which did not diminish the effectiveness of inoculation. Cook et al. (2017) conducted two experiments: one on the effects of inoculation against false media balance and one on its effects against a “fake experts” technique. In their first experiment, they found that interventions, especially consensus information, had a neutralizing effect on misinformation and especially increased the perceived consensus. Consensus information was also found to have a neutralizing effect on free-market supporters. With regard to the use of “fake experts,” they demonstrated that an inoculation treatment did not protect all respondents against misinformation but especially influenced those who support the free market and “inoculated” them against misinformation that resonated with their previously held beliefs. However, the effect sizes (in terms of explained variance) that they reported regarding the interaction between inoculation and free-market support were rather small. Interestingly, the inoculation especially protected the perceived consensus, the acceptance of anthropogenic global warming, the attribution of climate change to human influences, and the support of mitigation policies for those who support a free market. It had no effect on trust in climate scientists. Both described experiments were conducted in the US–American context.

Thus, the existing experiments and results demand a replication, an extension to other contexts, and the inclusion of further intervening variables. These desiderata are where our study ties in. We test the effect of misinformation and an inoculation intervention preceding exposure to misinformation on the perception of five aspects of climate change and include three central intervening variables based on previous studies. The following five dependent variables are in accordance with Cook et al. (2017): (1) the perceived scientific consensus, (2) the perceived human influence on global warming, (3) acceptance of anthropogenic global warming, (4) trust in climate scientists, and (5) support for mitigation policies. The intervening variables are the support for an unregulated market (replicating Cook et al., 2017), populist attitudes (based on Hawkins et al., 2012 and Akkerman et al., 2013), as well as political left-right orientation (based on Häkkinen and Akrami, 2014).

3. Methods and measurement

Procedure

To answer our research questions, we conducted an experiment with a 2 × 2 between-subjects design, which resulted in four different conditions: a control group, an inoculation group, a misinformation group, and an inoculation preceding misinformation group.

The participants were recruited through an online access panel for scientific studies called SoSci. A total of 647 respondents participated and were randomly assigned to one of the conditions.

Depending on the assigned condition, participants were shown no stimulus (control group), one stimulus (inoculation or misinformation for the inoculation or misinformation groups, respectively), or two stimuli (inoculation preceding misinformation). Both stimuli followed those used by Cook et al. (2017) for their second experiment and were translated into German to serve the purpose of our study. The inoculation stimulus outlined the usage of fake experts as one misinformation technique used to undermine perceived scientific consensus. The text drew mainly on the example of the debate surrounding tobacco consumption and its health implications, but it linked the technique to the fossil fuel industry in the context of climate change as well. The misinformation stimulus reported on the so-called “Oregon Petition,” an initiative of climate skeptics that seeks to give the impression that skeptical positions are shared by many scientific experts by using fake experts. To adapt the stimulus to the German context, the Oregon Petition was linked to its German equivalent, the “Leipzig Declaration on Global Climate Change” (Leipziger Erklärung). This declaration was first initiated on occasion of a skeptical conference held in 1995 in Leipzig, Germany, which was cosponsored by the prominent climate skeptical organization Science and Environmental Policy Project (Jacques et al., 2008). The declaration was signed by several “experts” who had, for example, ties to actors of the fossil fuel industry or were qualified in fields unrelated to climate science (Jensen, 1998). Since these signatories are not climate scientists, we consider them fake experts in this context.

The misinformation groups were later debriefed in a short text after completing the survey. The stimuli and study’s design, were, as described earlier, adapted from the study by Cook et al. (2017). In addition, we extensively pretested our survey and the materials used as part of a seminar for graduate students on the misinformation techniques of climate skeptics and counterstrategies for addressing them (the students themselves gave feedback on the materials and also conducted a pretest with 10 respondents, using the thinking-aloud-technique to detect possible problems). Based on these pretests the texts for the stimuli were slightly shortened. All the used materials are available both in German and English in the Supplemental material.

Sample

In total, 647 participants completed our survey. After reading the stimulus or stimuli, participants were asked to answer two simple stimulus-related questions to test their level of attention (e.g. What was the topic of the article? Which strategies do lobbyists use to promote skeptical arguments and to sow the seeds of doubt?). Participants who did not answer the attention filter question(s) correctly were not included in our dataset (n = 10). Participants with a very short response time for the entire survey (less than 5 minutes) were excluded as well (n = 8). Thus, in sum, 18 respondents were removed from the original dataset, which resulted in a final sample of 629 respondents across the control group (n = 176), the inoculation group (n = 156), the misinformation group (n = 151), and the inoculation preceding misinformation group (n = 146). The mean age of the sample participants was 45 years (SD = 15.61). 54% were female and 63% held a university degree (for more information regarding participants’ characteristics see Table A1 in the Supplemental material). Whereas the distribution of age (mean in Germany: 44.5 years) and gender (50.7% female in Germany) was quite close to that of the population, our respondents mainly had high formal education (17.7% with university degrees in Germany; Statistisches Bundesamt, 2019a, 2019b; Bundesinstitut für Bevölkerungsforschung, n.d.). In contrast to the sample used by Cook et al. (2017), which was representative for the US population regarding age, gender, and income, our sample is skewed toward the higher educated segments.

Dependent variable: Acceptance of global warming

We tested the influences of misinformation and inoculation on the participants by adopting five dependent variables from Cook et al. (2017; see also Arlt et al., 2010). First, to measure acceptance of anthropogenic global warming, we asked participants to rate eight items regarding their degree of agreement or disagreement on a five-point Likert-type scale (ranging from 1 = totally disagree to 5 = fully agree). The items presented participants with different statements about climate change as a threat, unproblematic, or a benefit; climate change as caused naturally or by human activity; and climate change as linked to extreme weather phenomena and to the existence and effects of global warming (e.g. “Climate change is one of the greatest threats to humanity.” Cronbach’s α = .82). These items were then aggregated to a mean index (M = 4.24, SD = .64). This high acceptance rate corresponds with representative findings (see above).

Dependent variable: Trust in climate science

Second, to measure trust in climate science, we proceeded in a similar way by asking for a rating of four items on a five-point Likert-type scale (ranging from 1 = totally disagree to 5 = fully agree), with statements about climate science’s credibility, the reliability of its results, its value, and fair deliberations within the field (e.g. “Climate scientists can be depended upon to help increase our understanding of what’s happening to our climate.” Cronbach’s α = .79). The four items were again compressed into a mean index (M = 3.92, SD = .73).

Dependent variable: Policy support

Third, we measured support for mitigation policies by the degree of agreement with nine items on a five-point Likert-type scale (ranging from 1 = totally disagree to 5 = fully agree), covering different political control mechanisms to reduce emissions in Germany, as well as Germany’s climate-related self-commitments in the international community (e.g. “Germany must adopt measures exceeding the objectives of the Paris Agreement.” Cronbach’s α = .81), again combined to a mean index (M = 3.5, SD = .76).

Dependent variables: Perceived scientific consensus and perceived human influence

Fourth, we measured the perceived consensus among climate scientists on anthropogenic global warming on a scale ranging from 0% to 100% (M = 77.68; SD = 17.9). In a similar manner, we measured as a fifth dependent variable the perceived influence of carbon dioxide emitted by human activity on extreme weather phenomena in Germany with four items referring to different weather phenomena (“The emergence of hurricanes in Germany”; “Increase in atmospheric temperature of 0.8 degrees Celsius since 1980”; “The 2013 flood of the century in Passau”; “Increase of global sea level of 20 cm since 1880”), also using a scale from 0% to 100% to indicate the perceived influence for each. These items were compressed into a mean index (Cronbach’s α = .84; M = 66.59; SD = 20.9).

Intervening variable: Free-market support

As a first intervening variable, based on Cook et al. (2017), we measured the participants’ support of a free (i.e. unregulated) market. For this purpose, participants rated five items regarding their degree of disapproval or approval on a five-point Likert-type scale (ranging from 1 = totally disagree to 5 = fully agree). These consisted of statements regarding the priority of the market over environmental issues, its efficacy, and the (charged) relationship between the market and social justice, as well as sustainable development and consumption (e.g. “An economic system based on free markets unrestrained by government interference automatically works best to meet human needs.” Cronbach’s α = .68). Similar to the measurement of our dependent variables, the items were consolidated into a mean index (M = 1.96, SD = .69).

Intervening variables: Populist attitudes and political orientation

As further intervening variables, we included agreement with populist attitudes and political orientation. To measure the first, nine statements were presented to the participants. They revolved around two dimensions of populism adapted to the climate change context. First, a perceived antagonism between “political elites” and “ordinary citizens,” that is the concept of anti-establishment (e.g. “The parties are only interested in the citizens’ votes, not in their views”). Second, support of anti-pluralism, understood as the sovereignty of the popular will (e.g. “Important decisions on climate protection should not be made by parliaments, but by the people.”). With our measurement, we relied on scales originally developed by Hawkins et al. (2012) and Akkerman et al. (2013). The respondents had to indicate their level of agreement with each statement using a five-point Likert-type scale (ranging from 1 = totally disagree to 5 = fully agree). Here, the value for internal consistency was good (Cronbach’s α = .86) and the items were averaged to a mean index (M = 2.85, SD = .84). To measure political orientation, we asked participants to classify themselves on a seven-point scale from left (1) to right (7) (M = 3.03, SD = 1.24).

4. Results

We performed the same analyses as Cook et al. (2017) and structured this section to closely resemble the original study’s results section. Therefore, tables and figures that provide additional information, for example, those on the effects of further intervening variables, are presented in the Supplemental material.

Descriptive effects of misinformation and inoculation

Our first research question concerned the effects of misinformation on attitudes toward climate change. A look at the descriptive results in Table 1 shows that, although the means between the experimental groups differ in the expected direction, the differences are only very small and statistically not significant. Thus, our study did not replicate the findings of Cook et al. (2017) in this respect.

Table 1.

Means (standard deviations) across interventions for all dependent variables.

Dependent variable	Control	Misinformation only	Inoculation only	Inoculation + misinformation
Perceived consensus	77.45 (17.98)	74.98 (17.11)	78.94 (17.45)	79.34 (18.86)
Human influence/attribution	65.95 (21.8)	63.94 (19.95)	67.82 (21.06)	68.80 (20.46)
Acceptance of anthropogenic global warming	4.28 (.61)	4.13 (.68)	4.24 (.66)	4.32 (.58)
Trust in climate scientists	3.93 (.80)	3.85 (.73)	3.94 (.66)	3.98 (.69)
Policy support	3.50 (.78)	3.45 (.79)	3.47 (.72)	3.56 (.78)

N = 629 (Control: N = 176; misinformation: N = 151; inoculation: N = 156; inoculation preceding misinformation: N = 146).

Comparing our results with the results of Cook et al. (2017), the negative effects of misinformation on the perceived consensus among climate scientists and the attribution of climate change to human activity were stronger in the United States than they are in Germany (−10 and −4.1 compared to −2.47 and −2.01). Irrespective of the experimental condition, the mean values show clear differences concerning the dominant popular opinion on climate change in the two countries. For example, our German sample perceives climate change to be less of a controversial issue among scientists (a mean 77.45% perceived consensus in the German control group compared to 54.5% in the United States control group). It also attributes climate change more strongly to emissions caused by humans (65.95% and 44.7% in the respective control groups) and denies it less commonly (4.28 and 3.39 in the respective control groups). Interestingly, these differences do not translate into a general support for mitigation policies, with a mean policy support of 3.60 in the US control group and a mean of 3.50 in the German control group.

Effects considering free-market support

Our second research question refers to the circumstances and groups (i.e. our intervening variables free-market support, political orientation, populist attitudes) for which inoculation can neutralize the influence of misinformation. The first circumstance under study is examining the effects of inoculation and misinformation interventions, while considering the support for a free market. To fully replicate the study by Cook et al. (2017), we performed five different Type-II ANOVAs for our dependent variables, with misinformation and inoculation stimuli as independent variables and the participants’ support for a free market as an intervening variable (see Table 2).

Table 2.

ANOVA results for the effects of interventions and free-market support (General linear model, type II; replication of Cook et al., 2017).

Dependent variable	Effects	Partial eta-squared	F	p
1. Perceived consensus	Inoculation	.013	6.032	.014**
	Misinformation	.001	0.245	.621
	Free-market support	.127	3.999	.000***
	Inoculation × misinformation	.000	0.020	.887
	Inoculation × free-market support	.015	0.556	.888
	Misinformation × free-market support	.049	1.832	.036*
	Inoculation × misinformation × free-market support	.02	0.876	.564
2. Acceptance of anthropogenic global warming	Inoculation	.002	1.072	.301
	Misinformation	.002	0.879	.349
	Free-market support	.201	6.720	.000***
	Inoculation × misinformation	.006	2.639	.105
	Inoculation × free-market support	.031	1.103	.354
	Misinformation × free-market support	.050	1.838	.035*
	Inoculation × misinformation × free-market support	.032	1.387	.175
3. Human influence/attribution	Inoculation	.008	3.768	.053*
	Misinformation	.000	0.000	1.000
	Free-market support	.118	3.580	.000***
	Inoculation × misinformation	.000	0.067	.795
	Inoculation × free-market support	.025	0.894	.560
	Misinformation × free-market support	.054	1.999	.019*
	Inoculation × misinformation × free-market support	.026	1.128	.337
4. Trust in climate scientists	Inoculation	.002	0.890	.346
	Misinformation	.002	1.014	.315
	Free-market support	.158	5.087	.000***
	Inoculation × misinformation	.002	1.074	.301
	Inoculation × free-market support	.022	0.815	.645
	Misinformation × free-market support	.029	1.066	.387
	Inoculation × misinformation × free-market support	.023	0.967	.476
5. Policy support	Inoculation	.003	1.510	.220
	Misinformation	.000	0.000	.996
	Free-market support	.240	8.394	.000***
	Inoculation × misinformation	.001	0.453	.501
	Inoculation × free-market support	.046	1.673	.064
	Misinformation × free-market support	.032	1.133	.329
	Inoculation × misinformation × free-market support	.031	1.305	.218

F(525/524) = 1.962, p < .001, R2 = .11.

F(514/513) = 3.161, p < .001, R2 = .20.

F(516/515) = 2.029, p < .001, R2 = .10.

F(521/520) = 2.195, p < .001, R2 = .12.

F(511/510) = 3.376, p < .001, R2 = .21.

p ⩽ .05; **p ⩽ .01; ***p < .001.

First of all, the results confirm the importance of free-market support as a predictor for attitudes toward climate change. More specifically, the analyses show that people who support a free-market estimate the scientific consensus concerning climate change to be lower, have a lower acceptance of anthropogenic global warming, rate the human influence lower, report a lower trust in scientists, and support policies to mitigate climate change to a lower degree.

In addition, misinformation and free-market support have a significant interaction effect on the perceived consensus, the acceptance of anthropogenic global warming as well as the perceived human influence. However, interaction effects of free-market support with message conditions were generally small and difficult to interpret. One pattern that emerged with the acceptance of anthropogenic global warming and, to an even smaller degree, with policy support may deserve attention in follow-up studies: The effect of free-market support on these outcome variables seemed to be softened if participants had been exposed to combined inoculation and misinformation (see Figure A1, in the Supplemental material).

In comparison to the results of Cook et al. (2017), free-market support seems to have a similar effect on climate-related attitudes in Germany and the United States alike. However, inoculation was more effective for the US sample, as the intervention has no significant effect for the German sample.

Effects considering populist attitudes and political orientation

As mentioned earlier, we included further intervening variables (political ideology and populist attitudes). With stepwise, linear regressions, we proved in a first step that these also have an independent main effect on the dependent variables. Based on these results, we calculated in a second step two further ANOVAs analogously to the procedure described earlier: with perceived consensus, the acceptance of anthropogenic global warming, human influence or attribution, trust in climate scientists, and policy support as dependent variables, with misinformation and inoculation stimuli as independent variables, and with the participants’ support for populist attitudes and their political orientation as intervening variables (see Tables A2 and A3 in Supplemental material). We find a significant main effect of populist attitudes on the perceived consensus among climate scientists, the acceptance of global warming, trust in climate scientists, and the support of mitigation policies. This effect means that people with populist attitudes rate the mentioned dependent variables lower and have more skeptical views concerning climate change. In contrast to the previously reported models misinformation has no reinforcing effect on climate skeptical views. In addition, the inoculation intervention has hardly any effect.

Regarding political orientation, our results show a highly significant main effect on all five dependent variables. People who are very conservative or with political views leaning to the right are more skeptical concerning climate change. There were no substantial interaction effects with political orientation and message conditions (the significant interaction effects had only small effect sizes and show no clear pattern).

Polarizing long-term mindsets and interventions

Figure 1 exemplarily shows the polarizing effects of all three intervening variables on the acceptance of anthropogenic global warming as well as the ambiguous effects of interventions on the same dependent variable. An inoculation preceding misinformation intervention increases AGW acceptance among free-market supporters when compared to the control group. An inoculation intervention does so among participants with strong populist attitudes. None of the interventions increase AGW acceptance among participants with a political orientation toward the right. In contrast, the misinformation-only intervention consistently decreases AGW acceptance in comparison to the control group.

Figure 1.

Predicted response from linear regression of observed data (dependent variable: AGW acceptance).

Dashed line with triangles represents control group, dotted line with diamonds represents group receiving inoculation-only intervention, solid line with squares represents group receiving misinformation-only intervention, and dot-dashed line with squares represents group receiving inoculation plus misinformation. Horizontal axes represent (a) free-market support, (b) populist attitudes, and (c) political orientation. For “free-market support” and “populist attitudes,” 1 corresponds to strong disagreement and 5 corresponds to strong agreement; for “political orientation,” 1 corresponds to a left orientation and 7 corresponds to a right orientation. Vertical axes represent AGW acceptance.

In addition, Figures A2 and A3 (in Supplemental material) indicate polarizing effects of the two additional intervening variables on attitudes toward climate change as well. The only exception here is the effect of populist attitudes on the attribution of climate change to human influences. Figure A2 in the Supplemental material shows ambiguous results for the depolarizing effects of interventions. Participants who strongly agree with populist attitudes have higher rates of human influence or attribution and policy support if they were assigned to the inoculation-only or inoculation preceding misinformation groups than if they were assigned to the control or misinformation-only groups, which does not apply to the other two dependent variables. Figure A3 in the Supplemental material highlights a polarizing effect of misinformation compared to the other experimental conditions. It does not, however, provide evidence for a depolarizing effect of interventions in comparison to the control group.

5. Discussion

The aim of our study was to replicate one of the experiments conducted by Cook et al. (2017) for Germany. Furthermore, we extended their study by including further intervening variables. Cook et al. (2017) found in their experiment that misinformation had a polarizing effect, especially on those groups in society who support a free market. In addition, they were able to neutralize this effect with inoculation messages preceding misinformation. However, their effect sizes were rather small.

First of all, our study confirms the importance of pre-existing beliefs and worldviews for climate-related attitudes: participants who support unregulated markets, agree with populist attitudes, and describe themselves as politically right-wing are particularly skeptical with regard to climate change. In this vein, our study replicates previous findings of Cook et al. (2017). Moreover, up to now, the relationship between conservatism, economic liberalism, and climate skepticism has not yet been demonstrated so clearly for Germany. Therefore, our study confirms again the influence of long-term dispositions on attitudes toward single issues, such as climate change. These dispositions determine how incoming information is processed, perceived, and remembered (Bolsen and Druckman, 2015), and they are one reason why existing attitudes are relatively stable over time and difficult to change. This also points to the fact that climate-relevant attitudes are linked to core personal values and personal identity and that particularly defensive reactions to persuasive messages toward more climate protection can be expected from these groups of people. Linked to that, climate change communication can only be successful in these groups if advocacy for climate change and climate protection can be reconciled with these core personal values.

Regarding the effects of misinformation messages and most notably the effects of inoculation messages, we could not replicate the findings of Cook et al. (2017); At least in our setting, the misinformation message and also inoculation preceding misinformation had hardly any effect on the climate-related attitudes under study. Similar to the results of Cook et al. (2017), among supporters of a free market, we find a small effect of inoculation. A similar trend can also be observed for conservative participants, as well as for respondents supporting populist attitudes. However, these effects are very small and in most cases non-significant. While Cook et al. (2017) interpreted their results in terms of inoculation being an effective tool in neutralizing the effects of misinformation, our findings suggest a more conservative interpretation.

With that said, it is interesting to ask what reasons may explain our divergent results and the small effects: First, it can be assumed that climate-related attitudes, like most attitudes, are relatively stable and robust. Against this background, it is not surprising that a single dose of information is not able to change these attitudes substantially and permanently. Second, support for climate-related attitudes and the acceptance of anthropogenic climate change are comparatively high among our German sample, which is in line with previous research and other surveys (e.g. European Commission, 2017). Even German participants with a very strong support for unregulated markets express relatively little climate skepticism compared to their US-counterparts in Cook et al. (2017). A generally less skeptic sample might also be less inclined to change its attitudes based on a skeptical misinformation message in the first place, because it is inconsistent with prior beliefs. Third, our respondents mainly have a high formal education (Statistisches Bundesamt, 2019a). This high level of formal education might affect the participants’ attitudes toward climate change as well as their response to inoculation and misinformation, since previous research showed that highly educated Germans tend to be well informed and concerned about climate change, while those segments of the German population with a lower formal education tend to be disengaged from the issue of climate change, rarely seek information about climate change, and use entertainment-oriented media (Metag et al., 2015). Therefore, future research should take into account the pluralism within populations. For an optimized impact of inoculation, it is relevant to know how inoculation techniques can be tailored to the attitudes, previous knowledge and needs of different societal groups and how these techniques can be embedded in their daily routines.

The results show that inoculation (i.e. awareness and information campaigns) might help in some cases and under certain conditions to diminish the effects of misinformation. Be it with regard to the use of fake experts in the context of climate change or with regard to a broader range of issues, the inoculative provision of knowledge about common misinformation techniques can be understood as one approach to increased media literacy (c.f. Jeong et al., 2012). Media literacy interventions, such as inoculation, aim at increasing recipients’ knowledge and skills that are necessary for critical thinking with respect to media content, techniques, characteristics, or structures (Jeong et al., 2012). This knowledge and these skills become especially important in the context of societal and political issues such as climate change as they are a necessary prerequisite to live up to the ideal of the engaged citizen in democratic societies by enabling civic participation in a digitalized environment (Mihailidis and Thevenin, 2013). It remains unclear which forms of inoculation are most suitable to strengthen media literacy in everyday life: be it, for example, through gamified web services, journalistic work, or school education.

Of course, our study comes with several limitations regarding the generality of the findings (Simons et al., 2017). First, although we used a scientific panel for our study, our sample is not representative of the German population. Whereas the distribution of age and gender are quite close to that of the population, our respondents are, as mentioned earlier, highly educated (Statistisches Bundesamt, 2019a, 2019b; Bundesinstitut für Bevölkerungsforschung, n.d.). Second, the stimuli used were rather short and the attention filters only provided us with some superficial information about how deeply the respondents processed the given information. Further research should extend and vary this design by using, for example, longer or other texts. Another interesting variation in design would be a longitudinal approach. Here, effects of differing time frames or a repeated exposure to the stimuli could be measured. Such a design would be closer to the real patterns of media usage. We also did not factor in the possible effects that the respective sources of the inoculation and misinformation interventions might have had on the perceived credibility of said interventions. In addition, more signatories signed the stimulus used in the original study—a fact that might partly explain the small effect of misinformation in our case. Third, the temporal and cultural specificity (Simons et al., 2017) could also be a confound in comparison to the original study. Since Cook et al. (2017) conducted their study, public awareness of the consensus has been steadily increasing (Cook et al., 2018). Particularly in Germany, the number of climate skeptics is also considerably lower than in the United States. In addition, the support of a free market is a political ideology that is not as prevalent in Germany as in the US setting. Therefore, only few of our respondents strongly support unregulated markets. Both temporal and cultural factors might, therefore, explain the comparatively smaller effects of misinformation and inoculation in our sample. Specificity is certainly another aspect that future research should consider, since the effectiveness of certain inoculation messages might not only vary within populations but also across cultures.

Click here for additional data file.

Supplemental material, sj-pdf-1-pus-10.1177_09636625211024550 for The effect of misinformation and inoculation: Replication of an experiment on the effect of false experts in the context of climate change communication by Hannah Schmid-Petri and Moritz Bürger in Public Understanding of Science