Public Conceptions of Scientific Consensus.

Despite decades of concerted efforts to communicate to the public on important scientific issues pertaining to the environment and public health, gaps between public acceptance and the scientific consensus on these issues remain stubborn. One strategy for dealing with this shortcoming has been to focus on the existence of scientific consensus on the relevant matters. Recent science communication research has added support to this general idea, though the interpretation of these studies and their generalizability remains a matter of contention. In this paper, we describe results of a qualitative interview study on different models of scientific consensus and the relationship between such models and trust of science, finding that familiarity with scientific consensus is rarer than might be expected. These results suggest that consensus messaging strategies may not be effective.

Introduction

In the epilogue of their influential Merchants of Doubt, Oreskes and Conway offer something of a justification for our trust of science. Some tasks — like buying a home — involve ceding trust to others. The stakes are high. If the officials in question are incompetent (or dishonest), we risk financial ruin. Yet we do it anyway. Why? Their (short) answer: because we don’t have much of a choice. We don’t have the expertise or access needed to do the title search, for example. So we “trust someone who is trained, licensed, and experienced to do it for us” (2010a, 272). Our trust of science, they suggest, is similarly compelled:

If we don’t trust others or don’t want to relinquish control, we can often do things for ourselves. We can cook our own food, clean our own homes, do our own taxes, wash our own cars, even school our own children. But we cannot do our own science. So it comes to this: we must trust our scientific experts on matters of science, because there isn’t a workable alternative. (272; our italics)

A cynical reaction is tempting: if the last few decades have revealed anything about modern society, it’s that many feel all too willing to reject scientists’ conclusions on all manner of subjects — from the safety of vaccines to the existence and threat of anthropogenic climate change (ACC). More recently, even the question of whether simple face masks are safe to wear and effective at reducing the spread of diseases like COVID-19 have been controversial (Funk and Tyson 2020; van Green and Tyson 2020). In this light, one might be tempted to reject their analogy; there is an alternative to trusting science: not trusting science.1

On the other hand, perhaps the analogy is apt. The force of the injunction to trust some purported authority turns in part on one’s take on the ‘workability’ of not trusting that authority. One doesn’t have to purchase a home, after all, or trust banks to hold one’s money. One doesn’t have to avail oneself of life-saving vaccines. Are these poor financial or health decisions? From the perspective of one who already trusts such entities, the answer may well be ‘yes’; they may even regard the alternatives as simply unworkable. But without that trust, it is difficult to make the case for trust from the negative consequences of not trusting without begging the question about whether trust is warranted. Given that trusting can make us vulnerable (Baier, 1986; Jones, 1996), some might reasonably judge that it is better to play it safe. In any case, it scarcely requires much sophisticated empirical study to recognize that telling people they should trust science because they have no choice is unlikely to be a productive means of producing such trust.2

What are the better alternatives for cultivating trust in science communication? This is a (very general3) question that many science advocates and communication researchers have been trying to answer for decades. The lack of significant success over this long period testifies to the question’s difficulty. In recent years, however, a science communications strategy has emerged with both conceptual–normative and (apparent) empirical support: to communicate about socially-contentious scientific issues framed as matters of scientific consensus. This basic idea has seen some uptake in the context of various public outreach projects on climate change4 and many members of the news media seem eager to adopt it as a panacea for our science-communication ills.5

Unfortunately, we believe that there’s reason for caution about consensus-framing as a general strategy for science communication. While it is possible to articulate a prima facie compelling normative justification for this strategy — showing why the existence of a scientific consensus (of a certain kind) concerning a claim provides a kind of epistemic warrant for accepting that claim in question — such a justification requires that the messaging takes a form that appears unlikely to be generally effective. This is because (as we will argue) scientific consensus, as a concept, seems not to be broadly understood. We arrive at this conclusion as a result of an interview study that members of this research team undertook in order to gain a more robust sense of the prevalent conceptions of scientific consensus in the American lay-public, details of which we present below.6 Given certain normative assumptions about how one should communicate science (or anything) to a wider public, we arrive at a dilemma for consensus-framed science communication: in the prevailing conditions, we should expect it to be either unsupportable or ineffective.

The plan of the paper is as follows. In §2, we will return to the question of the public’s trust of science and consider the normative justification for accepting propositions on which there is a scientific consensus of a certain kind. Crucial to what follows is the distinction between consensus and mere agreement — a distinction that those practicing and researching science communication have not consistently drawn. We will argue that only when understood as a consensus (in a certain robust sense that sets it apart from mere agreement) can consensus-framing properly convey epistemic warrant.7 In §3, we describe our qualitative study that suggests that, framed as such, this epistemic warrant will likely be lost on a significant portion of the lay public; §4 assembles and discusses our dilemma and considers possible responses. We conclude in §5 with some tentative thoughts about next steps for both philosophers and science communication researchers.

Trust of Science

From Trust of Individual Scientists to Trust of Scientific Consensus

Consider first a simple case: a layperson’s trust of an individual scientist to accurately inform them of a particular scientific conclusion relevant to their lives — for example, whether drinking a glass of red wine every night would harm their health in some way. It is familiar that the general social epistemic task in evaluating testimony (in general) involves assessing testifiers on at least two dimensions — their competence and honesty. While the sort of basic plausibility filters we typically employ (Lipton, 1998) no doubt have some role to play — most of us would probably reject out of hand claims that a glass of wine will kill us or that it will cure our ails — in many scientific contexts, it seems likely that the two dimensions of trustworthiness will need to do most of the epistemic heavy lifting. Science, after all, has been known to produce deeply counterintuitive knowledge.8

When it comes to the competence dimension, it is controversial whether the task is realistic for those without much scientific training. Some suggest that the challenge is in principle meetable, however. Oreskes and Conway gesture in this direction shortly after offering their brief justification for lay trust of science:

because scientists are not (in most cases) licensed, we need to pay attention to who the experts actually are — by asking questions about their credentials, their past and current research, the venues in which they are subjecting their claims to scrutiny, and the sources of financial support they are receiving. (2010, 272)

In a similar spirit, Anderson (2011) describes various criteria for judging honesty and epistemic responsibility, arguing that lay assessment of these qualities is possible even for those with relatively modest educational attainment (cf. Feinstein, 2011; Keren, 2018).

On the other hand, the perception of expertise can sometimes be a matter of motivated cognition (Kahan, Jenkins-Smith, and Braman 2011; Suldovsky 2016; Suldovsky, Landrum, and Stroud 2019; Stewart 2019). Complicating matters further is research suggesting that science, as a profession, occupies a somewhat ambiguous position in the public consciousness. As a general matter, while the public tends to accord scientists considerable competence (only engineers rank higher), they occupy only a middling position when it comes to “warmth” (Fiske & Dupree, 2014, 13,595). Such affective dimensions of trust cannot be easily discounted. Epistemic trust is tied up for many with a moral sense of trust (“What kind of people are these folks?” “Do they have my best interests at heart?”). As de Melo-Martín and Intemann note, “When we trust, we are vulnerable to others. Hence, trust is risky; our trust can be betrayed. If people trust scientific experts to produce and disseminate sound knowledge and scientists fail to do so, people will have incorrect beliefs and make inadequate decisions” (2018, 90). Recognizing that the aims and values of a given scientist may not cohere with one’s own — and that, being people, scientists are as apt as anyone to dissemble or mislead (given the right incentives and character flaws) — might lead one to withhold their epistemic trust.

Such complications at the individual level suggest an alternative locus for the prima facie trustworthiness of science: the scientific community (as a somehow united whole)9 — or, to construe things more narrowly: scientific consensus (concerning a particular issue). It is at this community level that particular scientific claims are vetted via peer-review and less formal post-peer-review practices. It is at this level that replications are attempted, disputes are prosecuted, papers are cited (positively and critically), results used as a platform for further work, and so on. When “the knowledge machine” of the scientific enterprise (Strevens, 2020) is firing on all cylinders, it is arguably reasonable to identify a kind of social objectivity attached to results on which there is robust scientific consensus (Longino, 1990). Think of this as the outline of a normative argument for the ex ante epistemic value of scientific consensus and thus a justification for the use of a consensus messaging strategy (CMS). The argument would need filling out to be fully plausible, of course; but suppose we grant the conclusion for a moment.

That such a normative case can be made does not, of course, entail that we’d be wise to adopt a CMS in response to our science communication challenges. Some science communication researchers, however, have recently offered descriptive, empirical support for CMSs on the basis of the “pivotal role” that perceived scientific consensus plays in the acceptance of science (Lewandowsky, Gignac, and Vaughan 2013). Van der Linden et al., (2015), citing the foregoing study, argue that “perceived scientific agreement [is] a ‘gateway belief’ that either supports or undermines other key beliefs about climate change, which in turn, influence support for public action” (2; see also van der Linden, Leiserowitz, and Maibach 2019). These results — including their generality and real-world efficacy — remain controversial (Landrum & Slater, 2020; Kahan, 2017; Landrum, Hallman, and Jamieson 2019; cf. van der Linden, Leiserowitz, and Maibach 2017). But the basic appeal of the underlying idea is obvious — particularly in cases like ACC. Thanks in large part to the well-funded campaigns to cast doubt on climate science (Oreskes and Conway 2010a; Brulle 2014), the public consistently underestimates the level of scientific consensus on ACC (Hamilton, 2016, 201; Leiserowitz et al., 2016) It stands to reason that if they came to believe that there was a scientific consensus on ACC, they would also tend to accept that ACC was occurring.10Mutatis mutandis, the hope goes, for other pieces of socially-contentious science.

Distinguishing Consensus from Mere Agreement

We will not attempt to evaluate the descriptive case for CMSs here — not directly, at least. Before describing our own empirical study that we contend bears on the tenability of CMSs, however, let us return to the normative case for their adoption: should the existence of a robust scientific consensus on X warrant a belief that X is true?11 This evidently depends both on what we mean by ‘consensus’ and what we may presume about the relevant background beliefs — e.g., how one conceives of consensus as coming about. The attentive reader of the empirical literature on CMSs may have noticed an occasional slide between talk of consensus and talk of agreement. Consider again van der Linden (2015) quoted above; here’s more of the context of that quotation:

We posit that belief or disbelief in the scientific consensus on human-caused climate change plays an important role in the formation of public opinion on the issue. This is consistent with prior research, which has found that highlighting scientific consensus increases belief in human-caused climate change [here they cite (Lewandowsky, Gignac, and Vaughan 2013)]. More specifically, we posit perceived scientific agreement as a “gateway belief” that either supports or undermines other key beliefs about climate change. (2; our emphasis)

This sort of conflation between agreement and consensus is also evident when one examines the stimuli for the studies in question, where participants are asked to estimate the level of agreement on climate change as a matter of a precise percentage. While treating consensus and percent agreement as functionally equivalent is methodologically expedient, there are serious questions about whether doing so is warranted.

To see this, consider a parallel to our normative question above: should the nearly unanimous agreement of a group of people on X warrant a belief that X is true? Surely the only reasonable answer to such a schematic question is (at best): it depends. How was this agreement reached? How diverse is this agreeing group — in their values, ideologies, prior commitments, &c.? What is the nature of their expertise (if any)? How relevant is it to the issue at hand, for instance? While the question of the social epistemology of consensus has received only sporadic philosophical attention over the years (for some exceptions, see 1990; 2002; Beatty 2006; 2017; Solomon, 2007; Odenbaugh, 2012; Miller, 2013; 2019; Stegenga, 2016), the non-identity of consensus with mere agreement is widely granted. Ditto for the claim that for consensus to deserve our epistemic respect, it should amount to more than mere agreement. Miller, for example, asks when a consensus is “knowledged-based or epistemically justified” (2013; 2019), offering a broadly abductive answer (“when knowledge is the best explanation” of the consensus) and suggesting conditions under which we might expect knowledge (rather than accident, bias, or various sorts of social pressure) to provide the best explanation of the consensus in question — including a condition of “social diversity” à la Longino (1990). Others offer broadly similar accounts (Stegenga, 2016) or note conditions under which consensus should not be taken as reliably indicative of the truth (Beatty, 2006).12

Here, we submit, understanding something about “how science works” as a social enterprise may be pivotal for appreciating the prima facie epistemic significance of scientific consensus — or at least being in a position to ask the right questions Anderson, 2011; Oreskes, 2019, ch.2). One of the more salient features of the scientific enterprise uncovered in the last century is its tendency toward self-scrutiny via a balance, of sorts, between competition, skepticism, and collaboration within the scientific community (Merton, 1973; Kuhn, 1962; Longino, 1990; Kitcher, 1990; Strevens, 2017, 2020) — a balance which, to an approximation, has the potential to keep in check individual “pigheadedness” (or even harness it for good, as discussed in Morton 2014) when certain conditions concerning the composition and activity of the community are met. Now, again, while there is clearly much more to be said about these conditions and the nature and limits of the epistemic warrant that scientific consensus can provide, the core point should seem quite plausible: matters of scientific consensus only provide such warrant in the context of a fairly rich set of background beliefs about what scientific consensus is and how it is formed. While such background beliefs are presumably common amongst the readers of this journal, it is an open question what mental model of scientific consensus prevails among the wider public. This is the question that we approach empirically in the study described in the next section.

Before turning to the study, it is worth reflecting on two further practical problems that a CMS which treats consensus and agreement as synonymous would face. First, we simply don’t have reliable survey data on the level of agreement among domain experts on all (or even most) scientific issues. A widely discussed poll mentioning “AAAS scientists” (Funk and Rainie 2015) is in fact a poll of AAAS members — subgroups of which include AAAS Members (a broad group including journalists, humanists, science communicators, among presumably many other non-scientists), Working Ph.D. Scientists, and Active Research Scientists.13 Depending on one’s view of whose agreement is relevant — is it all working scientists or only specialists? — such surveys, where they exist, will be of questionable value.

Second, even if we had the more fine-grained surveys on various issues, previous research on public conceptions of consensus suggests that many people have a very low tolerance for dissent. Aklin and Urpelainen report that “the scientific community can only convince the public about the existence of a problem with a high degree of consensus [meaning agreement]. In other words, even a modest amount of scientific dissent significantly decreases public support for environmental policy” (2014, 174). This makes intuitive sense. In a scientifically sophisticated vernacular, ‘consensus’ is as much a qualitative as quantitative matter; just as it involves a conception of a rigorous process of contestation and a fair hearing of the evidence, we would submit that it also (as a byproduct) involves an increasing marginalization of dissenting voices. Treated as a purely quantitative matter, on the other hand, a member of the lay public might reasonably wonder (e.g., concerning ACC): “What do those 3% of apparently dissenting scientists say? What evidence do they have? Shouldn’t we consider this as well?” (Landrum & Slater, 2020, 3). It is thus an open question whether matters on which science-savvy observers recognize a consensus would be treated as such by the lay public if the issue was discussed in terms of agreement, say, on the order of a mere 75%.14

Thus, a CMS using ‘consensus’ — abjuring the infirmities of a percentage-agreement gloss and potentially signaling the existence of a more robust process of formation — would seem to be preferable, both normatively and practically. But is it workable? This is a matter on which further direct experimental study is needed. The study we describe below concerning how members of the lay public conceptualize scientific consensus bears on the workability question indirectly. To it we now turn.

The Study: Public Conceptions of Scientific Consensus

Aims

Our primary aims in this study were (1) to examine what models exist in the general public for scientific consensus and to determine how sophisticated such models are; and (2) to determine whether and how scientific consensus figures into the public’s trust of science. We chose semi-structured interviews and an analysis methodology based in grounded theory, as explained below, to capture qualitative data to answer these questions and to develop further hypotheses concerning the public’s conception of scientific consensus.

Methods

The authors and team of student researchers (24) conducted a total of 70 semi-structured interviews between September of 2018 and December of 2019 from a variety of backgrounds and locations in the U.S, including data from 16 different states. The researchers initially employed convenience sampling via acquaintance to collect interviews, and then, in an attempt to increase the range of age, education attainment, religiosity, and political ideology represented, moved to purposive sampling later in the process of data collection. In particular, the purposive sampling targeted participants with lower levels of education and conservative political ideologies as those populations were underrepresented in the original set of data. Demographic information for the sample can be viewed in Table S1 in the online supporting material.15 While we need to be cautious about generalizing these results to the entire U.S. population (especially to habitually underrepresented communities), this is a respectable sample size for a qualitative study of this nature. They are meant to explore participants’ views in greater depth than can be achieved using quantitative measures. One particular way in which our sample fails to be demographically representative is in their relatively high level of education attainment, which might incline one to expect greater sophistication in conceptions of science.

The student researchers were trained in interview methodology and normed by the first two authors through a series of practice interviews. Interviews were then conducted either face-to-face or via videoconferencing, audio recorded with participant consent, and transcribed and checked by the authors. The semi-structured interviews used open-ended questions inviting participants to share their understanding of science, scientific consensus, and reasons for trusting (or not) scientific results. Early questions were fairly general and designed to provide participants opportunities for mentioning scientific consensus (or concepts in the vicinity) naturally without prompting. Subsequent, more-focused questions addressed whether participants were familiar with the idea of a scientific consensus, and (if so) asked them to describe their conception of that term. The interview also included questions (some about two hypothetical scenarios) designed to allow the researchers to gauge the sophistication of participants’ understanding of scientific consensus. The full interview script can be found in the online supporting material. After participating in the interview, participants were given a survey to collect demographic information and data concerning participants’ understanding of science as a social enterprise (to be used in a future analysis).

The authors coded the relevant questions of the transcribed interviews and entered the resulting data into spreadsheets. Simple descriptive coding schemes were pre-determined based on the interview questions (e.g. codes for mentioning consensus when discussing trust in science or not), but many codes having to do with level of sophistication in conception of scientific consensus and definitions of “science” were developed through an inductive process of reading and re-reading transcripts, identifying recurring themes or words, and finding appropriate categories into which response types could be grouped. This common technique for qualitative interview coding borrows from grounded theory (Glaser & Strauss, 1967; Birks and Mills 2015). Further information on coder norming, the coding protocol, and inter-rater reliability (mean Krippendorf’s Alpha for all raters on all variables = 0.90) can be found in a detailed methods section in the online supporting material.

For this analysis, we focused on four variables: (1) Approach to science, (2) Consensus in response to trust, (3) Familiarity with consensus, and (4) Sophistication of consensus model. Our inductive coding practice generated sub-categories into which we sorted participants for each of the four main variables. Each variable and the corresponding results are briefly described below, with discussion about how the qualitative and quantitative data relate to our research questions and hypotheses. The final codebook with full explanations can be found in the online supporting material.

Results and Discussion

1. Approach to Science variable. Our research questions and aims were centered on participants’ conceptualization of scientific consensus, but in order to contextualize their views on this subject and mask our focus, interviews began with questions about how participants understand science. Most responses (44%) fell into a heterogeneous category we labeled “Muddled.” This category included responses identifying science only as a subject of academic study or (to our surprise) the natural world itself.16 Other common responses in this category saw science as an effort to “prove something is true” but without any evident conception of how scientists went about this.

The “Broad” category of responses (24%) included any that characterized science as the pursuit of knowledge or understanding broadly without any mention of concrete outcomes. These responses tended to include statements like “science is studying what happens in the world.” “Process/Method-Oriented” and “Outcome-Oriented” approaches to science were both relatively common (21% and 9% of interviewees, respectively). “Process/Method-Oriented” responses generally focused on the distinctive methods of science — like experimentation, testing of hypotheses, or systematic observation. “Outcome-Oriented” approaches tended to focus on the “products of science,” such as discoveries, understanding, knowledge, cures for diseases, or technological advancements.

The least common type of response was labeled “Enterprise-Oriented” — this category was intended to encompass conceptions of science that highlighted the sense in which it is a social enterprise aimed at producing, revising, and curating knowledge and understanding of certain features of the world. The “Enterprise-Oriented” category was developed prior to interview coding, as a possible category that we hypothesized might be attributed to participants who connected their trust of certain pieces of science to the question of whether a consensus existed on that science. Only one of the 70 participants expressed an “Enterprise-Oriented” approach to science.

2. Consensus in Response to Trust variable. Interviewers asked participants whether they trusted science, and then asked participants to explain their response. In some cases, interviewers asked participants if they trusted individual scientists or science as a whole. Very few of the interviewees (3) spontaneously mentioned a conception of scientific consensus (including general agreement among scientists) as a reason to trust science. An additional six interviewees did mention consensus as a reason to trust science after the prompt regarding science as a whole versus individual scientists (coded as “Mixed” in our coding scheme). The vast majority (87%) of respondents, however, gave various other reasons to trust or distrust science. Some of these were based on ideas about science having the “facts” or being “concrete.” An example of this can be seen in the excerpt below:

Interviewer: Do you feel like you generally trust science?

Participant Z1: Yes.

Interviewer: Why?

Participant Z1: It’s concrete.

Interviewer: Could you say more?

Participant Z1: I feel science is concrete in terms of it’s not religion or philosophy or political viewpoints. It’s science and math. It’s more concrete.

Other responses were more focused how science is portrayed in politics, or in media representations, as is represented in the response below:

Participant AP1. Yeah, I trust science. I think it depends on, I guess, what it is. Like I’m a firm believer, I like vaccines and I don’t believe in that if I get a shot I’m going to become dyslexic. I don’t believe in the common media portrayals of science.… So, I definitely do trust science, I just don’t trust them in [the] media’s portrayal of science, if that makes sense.

Participant CM1. I would say [I trust science], I have no reason not to trust it. I think I start not to trust it when it becomes political, you know? So when you have politicians starting to argue about science like okay, like what? And again, I think that’s my natural inclination to be suspicious of politics in general because you know, they’ll say whatever they want to say in order to advance their interests, whether it’s completely... I’m not saying it’s a lie, but there’s definitely a lot of half truths that float around up there.

In some cases, trust in science was described as justified for reasons of methodology and “proof,” as in the following example:

Participant SJ3: I trust science because...they do an experiment. Trial and error...they don’t just say, okay, it’s scientifically proven, but they have a reason behind each…each theory, or each reasoning. So, for example, people say organic food is better, but there are scientific reasons…you can prove that certain organic foods are better to eat. They have these reasonings behind it.

Our results suggest, in answer to our second research question, that it is relatively rare for members of the lay public to connect their trust of science or scientific claims with beliefs about scientific consensus. For the most part, consensus seemed to be unrelated to participants’ thinking about the grounds for trusting science.

Familiarity with Consensus variable: During the interviews, researchers asked participants if they were familiar with the idea of scientific consensus. This occurred after questions regarding trust of science, how new ideas become accepted in science, and a scenario about whether participants would be inclined to accept results from new research, giving the participants ample opportunity to bring up consensus (or cognate ideas) naturalistically (vanishingly few did). In response to this question, 30 participants (43%) indicated that they were familiar with the term ‘scientific consensus’. These responses were coded as cons_fam (“consensus familiar”) regardless of the accuracy of the participants’ subsequent definition of the term. With this question we were only trying to get a sense of the proportion of interviewees who would recognize the term if it was given to them. Fifteen participants asked for a definition or to be reminded of what the term meant, and then expressed some understanding or recognition after the reminder. These responses were labeled cons_np (for “needed prompt”) and were considered distinct from the 25 cases (36%) in which participants did not know what scientific consensus was prior to a definition and expressed at most acquiescence (and sometimes confusion) when given the definition (labeled cons_unfam).

Sophistication variable: While interviewers asked participants to describe their conception of scientific consensus, various parts of the interview were designed to elicit further detail in the participants’ models of consensus from which its sophistication could be judged. Our inductive coding approach generated four categories of levels of sophistication.

The first level of sophistication, labeled Unsure/No View, was applied when a participant reported being unfamiliar with consensus, did not express much recognition, or did not evince a distinctive view when offered a basic definition by the interviewer and or in the scenarios designed to encourage them to think about the scientific community (or sub-communities). Generally, these participants accepted the minimal characterization offered by interviewers (see below), but offered little else. This code was compatible with a participant expressing some claims about the likely formation, distribution, or relevance of consensus on prompting, but this usually happened as a clear guess associated with the interviewer’s definition. The following example represents a typical Unsure/No View response:

Interviewer: Are you familiar with the idea of scientific consensus?

Participant M3: No.

Interviewer: By consensus I mean something like general agreement.

Participant M3: Okay.

Interviewer: How common do you suppose consensus is in science?

Participant M3: Depending on the issue, I’m sure there’s a lot of it.

Interviewer: And is there one topic or subject or issue that you think has a significant amount of consensus?

Participant M3: Not really.

Interviewer: Okay, do you have a sense of how scientific consensus comes about?

Participant M3: There has been improvement throughout the years. It’s kinda hard to debate it. So, I would say that the longer the study, you have more.

Only seven of the 70 interviewees (10%) were categorized as having this level of sophistication. Far more common (47%) was the second level of sophistication, which we labeled Muddled. In these cases, the participants thought of consensus in normatively non-standard ways, often at opposite ends of a spectrum of necessary agreement. In some cases, participants believed that 100% agreement between scientists, with no toleration for dissent, was necessary for consensus. In others, participants thought that just a small plurality of scientists, perhaps multiple people working in the same lab, or one other scientist convinced by the evidence, constituted a consensus. Some participants evidently conceived of scientific consensus as something pertaining to the level of agreement in the general public (e.g., “it’s when the masses, the majority of the people accept something scientific as true.”). This code also encompassed cases in which a more standard conception of scientific consensus was expressed, but was accompanied by non-standard beliefs about how consensus was reached, such as through a group of privileged insiders, through only the scientists deemed most intelligent, through governmental “approval” or peer review, or as the manifestation of a kind of “groupthink” as in the example below:

Interviewer: How common do you supposed consensus is in science?

Participant SJ1: Probably fairly — it’s kind of like groupthink.

Interviewer: Do you have a sense of how scientific consensus comes about?

Participant SJ1: Yeah, I think it’s what I said before, that the more often someone states something as fact, the more apt people are to accept it as fact, whether it is or it isn’t.

More standard understandings of scientific consensus were categorized as Mainstream. The 22 participants (31%) whose responses were coded with this third level of sophistication thought of scientific consensus as general, strong agreement of the relevant agents. Here’s typical response for this category:

Interviewer: Are you familiar with the idea of scientific consensus?

Participant C2 : Yes. That means that the greater body of the scientists agree on a conclusion.

Interviewer: How common to you supposed scientific consensus is in science?

Participant C2: It’s tough to answer that. There’s all sorts of questions. Some of it — the consensus is easy. Others — the consensus is much more difficult because the evidence isn’t convincing enough. So it’s common to have it, it’s common not to have it.

A clear, mainstream understanding of the term is present here. Our use of this category tolerated some minor, non-standard models of how consensus comes about, such as suggestions that all relevant scientists might meet in person to discuss a subject and reach a consensus. Generally speaking, it was compatible with a loose identification of consensus as general agreement.17

The final, and most nuanced model of scientific consensus was labeled Sophisticated, and was seen in eight of the participants in this study (11%). This code built upon the Mainstream category; recipients added an appreciation of certain nuances of consensus that contribute to its epistemic significance. This could include a more complete understanding of how consensus comes about or a recognition of the compatibility of consensus with minority or outsider dissent; responses in this group might also recognize the desirability of social diversity among the relevant agents, and/or their relative independence in forming their views. These nuances appeared in response to questions and scenarios throughout the interview, as in the example below:

Participant SS1: [How common scientific consensus is] obviously ranges on the topic, what the field of study is. There are certain fields where there’s a lot more research, a lot more money pumped into it. So a good example I would say just like climate science. That’s where there’s a really large consensus on that field. Other fields don’t have that same certainty.... There’s always going to be people on the other side of that is going to disagree with you, but when you have a majority of the people.

(later in interview) Interviewer: So, imagine that all the scientists in a certain corporation that conducts medical research agreed on the cause of an illness. [Do] you regard that as a consensus on what you would be inclined to accept their conclusions?

Participant SS1: No, because it was just from one. You said one corporation? ... No, it has to be outside sources. They have that obviously incentive to sell that product.... I don’t find that to be credible at all…. If they had overwhelming evidence from outside of the corporation [I would find that credible].

Overall, converting our four sophistication codes to numbers (1–4, from least to most), the average sophistication score across our 70 interviews was 2.4. While we do not take the numerical values we associated with our category descriptions to constitute a well-defined scale — there is clearly room to disagree about whether a “muddled” view of consensus is “better or worse” than having no view at all — this average being noticeably below a Mainstream of 3.0 conveys something important about the overall sophistication of our participants’ mental models of scientific consensus. Or, put another way, our observation was that a majority (57%) of our interviewees either lacked a pre-existing view of what scientific consensus was or harbored significant misunderstandings about it. Especially when we reflect on the fact that even a Mainstream model of scientific consensus that treats it as (potentially) little distinguished from mere agreement may lack the sophistication we posit is necessary for generating the relevant epistemic warrant, we face the worrying possibility that nearly 90% of our participants lacked what was needed to appreciate the significance of scientific consensus.

Breaking out our latter three variables by the five categories in the Approach to Science variable, we observe a noticeable trend towards greater recognition and sophistication concerning scientific consensus for those with what we would consider more sophisticated conceptions of the scientific enterprise. Those with muddled views of science (44% of our participants) were unlikely to associate consensus with their trust of science and, indeed, tended to be unfamiliar with the concept itself (see Table 1 below).

Table 1

Summary Results by Approach to Science18

1. Approach to Science(percentage of total participants)	2. Consensus in response to Trust?	3. Familiar with Consensus?	4. Sophistication (mean score)
Muddled (44%)	90% no 6% mixed 3% yes	29% familiar (fam) 16% needed prompt (np) 55% unfamiliar (unfam)	2.0
Broad (24%)	88% no 6% mixed 6% yes	59% fam 24% np 18% unfam	2.6
Outcome-Oriented (9%)	67% no, 17% mixed 17% yes	33% fam 33% np 33% unfam	2.7
Process-Oriented (21%)	87% no, 13% mixed, 0% yes	53% fam, 27% np 20% unfam	3.0
Enterprise-Oriented ( 1%)	100% no	100% fam	4.0 (single result)

Limitations

As with all qualitative studies with this methodology and sample size, limitations exist in the generalizability of the results. Although we aimed for diversity through our purposive sampling, people of color, politically conservative individuals, and those with less education are underrepresented in this sample. These results are also not readily generalizable to populations outside of the U.S. We note, however, that our participants overrepresent those demographic groups — such as those with higher levels of educational attainment — that one might expect to possess a more nuanced understanding of the scientific enterprise. If this is the case, our results may, in fact, overestimate the level of sophistication about scientific consensus in the general public.

Furthermore, it is possible that there are views or models of consensus that were not drawn out by our particular interview protocol. For example, consensus might matter functionally to members of the general public when it comes to their trust of science, though it is rarely explicitly thought to matter. We did attempt, in the creation of this interview protocol, to give respondents ample opportunity to mention consensus or neighboring concepts, but we can rule out neither this possibility nor the possibility that particular ways of asking questions masked the role consensus plays in some participants’ trust of science.

A dilemma for CMSs

These limitations in mind, our results point to a (two-tier) dilemma for the advisability of using CMSs to communicate with the public about science. The first horn of the dilemma stems from the observation that the idea of scientific consensus often seemed simply unfamiliar to our study participants. When the concept is recognized at all, participants as a whole did not show much sophistication in their grasp of it. Moreover, as we noted above, it was only in the vast minority of cases that the existence of a consensus came up as relevant to a participant’s trust of science, even after prompting. Though we need to be cautious about drawing significant conclusions from these findings, at the very least they should temper expectations for the efficacy of CMSs for generating trust in scientific messages. Indeed, they may suggest an explanation for the inconsistent results in efforts to replicate that model in other contexts and in other ways (see, e.g., Deryugina and Shurchkov 2016; Bolsen & Druckman 2018; Landrum, Hallman, and Jamieson 2019; Chinn and Hart 2021b). More empirical research is clearly needed on this point.

A natural way of responding to the lack of recognition of (or sophistication about) the concept of consensus is to replace it in our scientific messaging strategies with mere agreement. Perhaps the persuasive effect of a rich conception of scientific consensus could be triggered instead by messages focusing on measures of agreement among scientists on a given issue. This leads to the second horn of dilemma — itself another dilemma: framing a CMS in terms of agreement will likely either fail to be a generally workable strategy or fail to be a normatively acceptable strategy.

Our case against workability was sketched above (§2.2): While we have (arguably) good measurements of the (impressively high) extent of agreement among climate scientists about ACC (Oreskes, 2004; Cook et al., 2016), other issues have not been studied at this level of detail, making percent-agreement effectively unavailable as an alternative for many scientific issues. Or worse, as we suggested above, it could be that levels of agreement noticeably below 100% will induce boomerang / reactance effects stemming from questions about the nature of the disagreement (Zhou, 2016; Chinn and Hart 2021a). Even in the case of ACC, with its near unanimity in the scientific community, climate change skeptics have (apparently successfully) employed a “Galilean Gambit” (Landrum & Slater, 2020, 3) to magnify the significance of even extreme minority views.19

The normative case against framing a CMS (when workable) in terms of mere agreement is, we think, intuitive. Suppose that mere agreement should not be regarded as providing epistemic warrant except against the backdrop of a range of background beliefs about the epistemic context of this agreement, processes that likely brought it about, and so on. Suppose further that such a backdrop cannot be assumed (or that we know it to be rare). Then, at best, representing a fact as supported by mere scientific agreement is tantamount to asserting something on grounds that one knows to be unjustified. While this is not a case of straightforward lying — one is not attempting to create false beliefs in another — it does appear to be a kind of dishonesty. It is thus prima facie wrong. We think this is true even if one believes the claim being asserted (and believes that it would be good for the recipient of our assertion to believe it). Consider an analogy: suppose we know that climate-denier Dave will reflexively believe anything that Tom Hanks asserts (whatever the truth of such assertions are). We might then be tempted to argue to Dave that he should believe that climate change is real because Tom Hanks has said it is. Doing so constitutes a kind of manipulation and thus arguably offends against his intellectual autonomy (cf. Riley, 2017; Fricker, 2021).

Now, of course, there’s room to resist this line of argument or the conclusion we draw from it. Perhaps when the stakes are high enough, the prima facie wrong of the dishonesty can be overcome by the social benefit of getting people to believe in a certain way. Such believers might not count as knowing (being, in a certain sense, “Gettierized”), but this may be a matter of indifference when it comes to the social good that is brought about by their true belief. That looks at least plausible in the case of climate change — on which more presently.

One might also argue that it’s possible to avoid insincerity while still using others’ false beliefs; returning to our analogy, we could effectively sidestep the matter of the evidential relevance of Tom Hanks. Rather than arguing as above, for example, one might instead say, “Look Dave: you think that everything Tom Hanks says is correct, right? I think that’s nonsense, myself, but have you heard that he thinks that climate change is real? So by your lights, you should believe that it’s real!” First, it’s not obvious to us that this completely avoids the manipulation; but grant for the sake of argument that it does. Is this sort of maneuver possible in the case of glossing consensus as mere agreement? Perhaps if we already knew that beliefs about the epistemic significance of mere agreement were widespread, we could simply appeal to these beliefs even if we found them to be evidentially dubious. But we don’t seem to know this. Indeed, as Intemann has pointed out, “[c]limate skeptics have rejected the empirical evidence for a scientific consensus precisely because they are dubious of the processes and practices that have produced agreement in climate science” (2017, 193). Without a pre-existing peg to hang our hat on — viz. that mere agreement is epistemically weighty — we would again presumably be in a position of falsely representing that the agreement is evidentially relevant to the target belief.

Perhaps it’s implausible to regard glossing consensus as mere agreement as dishonest. It might be more akin to a harmless idealization or speaking in a language that members of the lay public can more readily understand (see, for example, Oreskes and Conway 2010b, 687). In an editorial in Public Understanding of Science, that journal’s editor suggested that the field should rethink “the very meaning of key terms like ‘quality’ and ‘accuracy’. Accuracy of science communication was traditionally defined as adherence to the specialist message, but is this still the case?…We probably need a new notion of accuracy” (Bucchi, 2017, 891). Charitably interpreted, we can read this as an encouragement to science communicators to consider more carefully and strategically how certain messages will likely be received — e.g., instead of talking about the extent to which the existence of anthropogenic climate change is confirmed or very highly probable, characterizing our epistemic state as knowing that it is occurring. As before, however, it is not clear how this sort of approach would work in the case of communicating the consensus about climate change. While mere agreement and consensus may of course overlap — the scientific consensus about climate change involves a high degree of agreement — the former is not a mere idealization of the latter.

Let us consider a final way of resisting this horn of our dilemma. In a fascinating and provocative series of articles, John (2018; 2019; 2021) has explored the limits of norms of sincerity and openness when it comes to science communication and expert testimony. In cases, for example, where non-experts harbor a “false ‘folk philosophy of science’” it might be that sincerity on certain matters will create in them false beliefs; likewise, “as in Climategate, transparency and openness may destroy warranted trust…. If we care about the promotion of true belief, we should not demand that scientists are transparent and open” (2018, 7). Indeed, John argues, there are situations in which one may need to choose “between making an honest assertion and making an effective assertion,” (9) (i.e., an assertion that would be in a non-expert’s epistemic interest to believe). Perhaps glossing consensus as mere agreement is like this: a way of producing a true belief in the lay public by way of a false assertion, a case of ‘well-leading’ rather than ‘misleading’ (10).

It would take us too far afield to evaluate John’s arguments in any depth. But even granting their basic thrust, much more would need to be said in favor of the effectiveness of an agreement-framed-CMS. Recall that this question arises in the context of the second horn of the second-tier dilemma — concerning an issue, like ACC, on which the scientific community and (even more) relevant experts agree. On this issue, the effectiveness of agreement-framed-CMSs for at least the immediate acceptance of ACC has been something of a mixed bag (see citations in §2.1); even when significant effects show up, effect sizes are small, and no one yet knows whether the relevant belief revisions would occasion changes in one’s actions relevant to climate change (for a review of the relevant literature, see Landrum & Slater 2020). More empirical research is needed here, as John agrees (2018, 10).

Aside from this “immediate” question of efficacy — can agreement-framed-CMSs shift basic beliefs about ACC (and like matters)? — we have a number of concerns about the longer-term efficacy of such strategies stemming from possible downstream consequences of representing that agreement as epistemically significant. One obvious worry for pursuing such strategies vigorously is that doing so might serve to entrench a faulty norm of acceptance: that scientific matters should only be accepted where there is near-unanimity. This would in turn make communication more difficult on issues discussed in §2.2 — that is, issues either about which we lack good information about the level of agreement of individual scientists or on which the level of agreement, while compatible with there being a robust consensus, may not surpass a heightened bar. Another worry is that it may put communicators in the precarious position of needing to defend the epistemic significance of agreement against objections like those gestured towards by Intemann above. Responses that open the door to accusations of dishonesty or manipulation might further corrode trust of such communicators. While this efficacy question is a good deal more difficult to study empirically, it too should be thought through and investigated carefully.

Conclusion & next steps

To summarize the overall structure of our dilemma is that if a CMS is sophisticated (abjuring a facile identification of scientific consensus and mere agreement), then the results of our study lead us to doubt that it will be effective; if the CMS, on the other hand, takes the simple approach and equates consensus and agreement, then it will either be difficult to employ in a broad range of cases or will transgress the sincerity norm in science communication (for communicators who accept our earlier points, anyway). The conclusion of the previous section was that even if this norm admits of exceptions in certain cases, we need to be cautious about potential downstream consequences for public trust and contributing to a more challenging communication environment overall.

Reflection on our dilemma brings us to a final, tentative point. We saw that greater sophistication in one’s view of science tended to coincide with it being more likely that one would be aware of the idea of scientific consensus and demonstrate greater sophistication in one’s grasp of the concept. This is not overly surprising. The fact that consensus was so rarely associated with our study participants’ trust of science suggests, though, that science educators and communicators could do more to produce an understanding of science that helps make more salient how healthy and robust forms of consensus come about, why such consensus should be seen as epistemically significant, and why such significance is compatible with the existence of minority dissent. It seems to us very plausible that a grasp of certain of the social–institutional features of the scientific enterprise — particularly, the balance between cooperation and competition — would provide an apt background for judging whether a consensus is likely to be indicative of the truth or could be explained away as groupthink, a bandwagon effect, or a conspiracy (Intemann, 2017; Slater, Huxster, and Bresticker 2019).

One of our next steps is to attempt to test this hypothesis by making use of the survey data concerning participants’ grasp of the social enterprise of science we collected after each interview. We also intend to undertake a deeper coding effort on these interviews to further explore the public’s perceptions of science and scientific consensus. Meanwhile, we believe that philosophers of science and epistemologists have an important role to play in contributing to the important and ongoing empirical research on effective (and acceptable) science communication strategies going forward.