Persuading republicans and democrats to comply with mask wearing: An intervention tournament.

Many people practiced COVID-19-related safety measures in the first year of the pandemic, but Republicans were less likely to engage in behaviors such as wearing masks or face coverings than Democrats, suggesting radical disparities in health practices split along political fault lines. We developed an "intervention tournament" which aimed to identify the framings that would promote mask wearing among a representative sample of Republicans and Democrats in the U.S. from Oct 14, 2020, to Jan 14, 2021 (N = 4931). Seven different conditions reflecting different moral values and factors specific to COVID-19-including protection from harm (self), protection from harm (community), patriotic duty, purity, reviving the economy, threat, and scientific evidence-were implemented to identify which framings would "win" in terms of promoting mask wearing compared to a baseline condition. We found that Republicans had significantly more negative attitudes toward masks, lower intentions to wear them, and were less likely to sign or share pledges on social media than Democrats, which was partially mediated by Republicans, compared to Democrats, perceiving that the threat of COVID-19 was lower. None of our framing conditions significantly affected Republicans' or Democrats' attitudes, intentions, or behaviors compared to the baseline condition, illustrating the difficulty in overcoming the strength of political polarization during COVID-19.

Early in 2020, the novel coronavirus quickly spread around the world, and within months, over 28 million cases and 910,000 COVID-19-related deaths had been recorded worldwide (Worldometers.info, 2020). In the first year of the pandemic, governments implemented a wide variety of community interventions to slow the spread of the virus, including school and workplace closures, stay-at-home orders, and public information campaigns to encourage greater observation of public safety behaviors like wearing face coverings, social distancing, and avoiding large gatherings of people. Many countries enacted a similar set of interventions, however, responses to these measures have been particularly divisive with a large partisan gap in the U.S. The political divisions have seeped into public attitudes on COVID-19 interventions, with partisan divides shaping disparate compliance and enforcement of these safety practices.

A wealth of data indicates that individual responses to the COVID-19 pandemic are related to political affiliation and beliefs. A survey of 3000 American citizens in late March of 2020 found partisanship (measured by party affiliation, intended 2020 Presidential vote, and self-rated ideology)1 is “the most consistent factor that differentiates Americans' health behaviors and policy preferences” (Gadarian, Goodman, & Pepinsky, 2021, p 0.2). This partisan division has persisted over the course of the pandemic and is evident with regards to preventive, pro-social health behaviors recommended by the CDC, such as wearing a face covering when in public. A Pew Research study in late June of 2020 confirmed that Republicans and Democrats perceived pandemic-related risks very differently, with less than half of Republicans worried about the health effects of COVID-19, a decline from April, compared to Democrats whose rate of concern remained high and stable from April to June (Pew Research Center, 2020). A late July 2020 poll by NBC News Survey Monkey showed 86% of Democrats reported wearing a face mask every time they left their homes and might be in contact with others, compared to 48% of Republicans (Wronski, 2020). Given the evidence that masks effectively reduce the spread of COVID-19 (CDC, 2020), these differences in attitudes and health behaviors can have substantial consequences on public health. In order to achieve compliance with mask wearing guidelines, more research on messaging that accounts for political divides is critical.

As communities continue to open up while COVID-19 remains a threat, particularly to unvaccinated individuals, it is urgent to understand how to persuade both Republicans and Democrats to engage in recommended health behaviors. While there are studies that have examined the effectiveness of different types of messaging about COVID-19 health behaviors (Capraro & Barcelo, 2020; Jordan, Yoeli, & Rand, 2020), there is a dearth of research that takes political affiliation into account. Here, we present an intervention tournament that includes seven different intervention conditions relative to a control to test which framings most effectively promote mask wearing among representative samples of Republicans and Democrats in the U.S. (N = 4, 931). While Republicans are more resistant to wearing masks than Democrats, polls show that neither party reported total compliance with pre-vaccine mask wearing guidelines (Pew Research Center, 2020). Moreover, from a theoretical point of view, we expect that different frames may be more or less effective depending on individuals' party affiliation as discussed below. As such, our research examined both Republicans and Democrats.

The current study builds on work that examines moral framings as a means to reduce attitudinal polarization between political groups (Day, Fiske, Downing, & Trail, 2014; Feinberg and Willer, 2013, Feinberg and Willer, 2015; Voelkel & Feinberg, 2018). According to Moral Foundations Theory (MFT), liberals and conservatives tend to emphasize different values when it comes to determining what is moral (Graham et al., 2011; Graham, Haidt, & Nosek, 2009). Liberals typically endorse moral values that stress protection from harm and fairness, whereas conservatives tend to adhere to moral values related to ingroup-loyalty, respect for authority, and protection of purity and sanctity (Graham, Haidt and Nosek, 2009, Graham et al., 2011; but see Schein & Gray, 2015, for an alternative perspective). In past work, researchers have successfully employed these differences to shift political attitudes on both sides of the political spectrum. Feinberg and Willer (2013), for example, examined the impact of moral messaging (purity and harm) on environmental attitudes, which tend to show a partisan split. They found that when framing pro-environmental arguments in terms of harm, a more liberal moral value, liberals reported significantly stronger pro-environmental attitudes than conservatives. However, when the argument was framed in terms of purity, a more conservative moral value, the significant difference between liberals' and conservatives' environmental attitudes was eliminated. In another set of studies, Feinberg and Willer (2015) found that conservatives held more positive attitudes toward same-sex marriage and universal healthcare—policies that conservatives don't typically support—when the arguments they read were framed in terms of conservative moral values (loyalty and purity). Likewise, liberals' attitudes toward military spending and adopting English as the nation's official language were more positive when arguments were framed in terms of fairness.

The present research expands on this work by testing the effectiveness of moral framings to persuade individuals to wear masks or face-coverings in the context of the COVID-19 pandemic before the vaccine was widely available in the U.S. As detailed below, we also include additional framings specific to the COVID-19 pandemic that reflect economic considerations, the danger of the virus, and effectiveness of masks that also show Republican-Democrat political divisions during COVID-19. Specifically, we developed a total of seven carefully constructed message framings which were pitched against a control condition in an intervention tournament of representative samples of Democrats and Republicans that aimed to identify the framings that best promote wearing a mask or face covering among Democrats and Republicans (Bruneau, Kteily, & Falk, 2018; Lai et al., 2014, Lai et al., 2016). Bruneau et al. (2018) coined the term “intervention tournament” as a method for testing not just a single strategy but pinpointing the best strategies out of a pool of strategies evaluated simultaneously. Below we advance specific hypotheses regarding which conditions may be more effective in persuading Republicans and Democrats to wear a mask or facial covering. We also examined which framings “win”—i.e., produce the highest mask wearing intentions and behaviors among Republicans and Democrats (see methods for details). While our tournament was conducted within the U.S., it may offer some insight into encouraging health behaviors in other countries, particularly given that similar political divides regarding COVID-19 behaviors have been found worldwide (Youngs, 2020).

Our seven conditions reflect different moral and COVID-19-specific framings. First, we built upon past framing work (Day et al., 2014; Feinberg and Willer, 2013, Feinberg and Willer, 2015; Voelkel & Feinberg, 2018) and used moral foundations theory (Graham, Haidt and Nosek, 2009, Graham et al., 2011) to develop four different moral-frame interventions, including individual harm, group harm, loyalty, and purity (Graham, Haidt and Nosek, 2009, Graham et al., 2011). Past research suggests that messages are more effective when they are designed to be consistent with people's moral inclinations (Day et al., 2014; Feinberg and Willer, 2013, Feinberg and Willer, 2015; Voelkel & Feinberg, 2018). Therefore, when directed at Republicans, messages that reflect moral foundations such as ingroup-loyalty and purity may be effective, while those that align with the moral foundation of liberals, such as harm, may not. Likewise, a message that focuses on harm may be effective for Democrats, while messages framed in terms of conservative moral values may not. Drawing on the model of moral motives (Janoff-Bulman & Carnes, 2013), we also included a hybrid group harm condition that integrates both having concern for the group and concern for harm at the same time, which may be appealing to both Republicans and Democrats.

In addition to our morally-framed messages, we also included three additional COVID-19 framing conditions that tap into specific concerns that Republicans and Democrats have regarding wearing face masks or coverings. While the aforementioned morally-framed messages tap into world-views that are differentially appealing to individuals, another viable approach is to target the specific psychological mechanism(s) that may underlie why Republicans or Democrats are reluctant to wear facial masks or coverings, or what has been termed “wise interventions” (Walton & Wilson, 2018). Based on extant polling data, we designed three additional conditions that are targeted to address these mechanisms.

First, we included an economic framing condition, as many Republicans are more worried about financial losses and shutdown due to the COVID-19 pandemic than public health (Shepard, 2020). A July 2020 poll of over 50,000 Americans by NBC News Survey Monkey finds that 67% of Republican respondents view the coronavirus outbreak as more of an economic crisis than a health crisis, compared to 17% of Democrats (Wronski, 2020). As such, it is possible that a message that highlights the benefit of mask wearing for being able to open the economy more quickly may be effective among Republicans and less so among Democrats.

We also included a condition that highlights the devastating threat that COVID-19 continues to have on the U.S. in terms of cases and deaths (The New York Times, 2020). Ironically, although research has typically found that conservatives are more psychologically, perceptually, and neurologically sensitive to threat than liberals (Carraro, Castelli, & Macchiella, 2011; Hibbing, Smith, & Alford, 2014; Jost, 2017; Jost, Glaser, Kruglanski, & Sulloway, 2003; Kanai, Feilden, Firth, & Rees, 2011; Oxley et al., 2008; Vigil, 2010), polls show that Republicans are less likely to perceive COVID-19 as threatening as compared to Democrats. For example, a poll conducted in July 2020 found that only 64% of Republicans, compared to 95% of Democrats, are at least somewhat concerned that they or a family member will be infected with the coronavirus (Blood & Swanson, 2020), and 63% of Republicans believe the coronavirus outbreak has been made a bigger deal than it really is, compared to 18% of Democrats (Mitchell, Jurkowitz, Oliphant, & Shearer, 2020). Recent work shows that feeling realistic threat to physical or financial safety as a result of COVID-19 is related to greater adherence to public health guidelines among people in general (Kachanoff, Bigman, Kapsaskis, & Gray, 2021). Thus, highlighting the realistic threats of COVID-19 may effectively encourage mask wearing among threat-sensitive Republicans. We note that this prime may also be effective for Democrats, as threat has been shown to “tighten” individuals and cultures and promote norm abiding behavior (Gelfand et al., 2011; Jackson et al., 2019).

Finally, we included a condition that highlights the scientific evidence behind mask wearing. In the early stages of the pandemic, messaging from health officials in the U.S. largely told people that masks were not necessary (Gregorian, 2020), and President Donald Trump sent mixed messages on masks through late July of 2020 (Breuninger, 2020). Later, however, there was consensus among experts and evidence that face coverings do diminish the spread of the virus (CDC, 2020). Despite this, a June 2020 poll indicated that there was lingering confusion among Republicans with 47% of Republicans, compared to 31% of Democrats, saying that it was harder to tell what information was true three months in than it was in the first few weeks of the outbreak (Mitchell et al., 2020). To address lingering confusion and disbelief about the effectiveness of masks, our final experimental condition makes clear that there is definitive scientific evidence in support of mask wearing, which may promote these behaviors among Republicans.2 Reminders of the effectiveness of masks may also promote mask wearing among Democrats given that they too have been exposed to conflicting information.

Our seven experimental messages, four moral and three specific to COVID-19, were compared to a control condition for a total of eight conditions. All eight conditions have been piloted extensively (see methods section below). We examined the effectiveness of the messages using four dependent variables: a) mask wearing attitudes, b) mask wearing intentions, c) signing a pledge to commit to wearing a mask/face covering, and d) willingness to share the pledge with social networks. For c) we created a separate website to host the pledge (see https://covidpledge.wixsite.com/sign) to maximize the realism of this measure.

Methods

Ethics information

This research was approved by the Institutional Review Board at the University of Maryland, College Park. All participants provided informed consent before participating in the study. We worked with Qualtrics to collect our data, so compensation was provided by the participants' panel company (hosted by the Qualtrics survey platform).

Design and hypotheses

We pre-registered our original study design and hypotheses (https://osf.io/4k8bf). The manuscript was accepted as a JESP registered report. For our tournament, participants were randomly assigned to one of eight conditions. Each condition included a standard message encouraging wearing a mask or face covering to minimize the spread of COVID-19 and an experimental framing about why masks are important. Each framing targeted a specific moral value or COVID-19-specific factor that may (or may not) be important to Democrats and Republicans. In addition to a message and framing, each condition included an image that represents the condition that was piloted. This image was displayed at the top of every subsequent page throughout the survey to reinforce the manipulation message. The eight conditions included the following:

Control (condition 1)

This was the baseline condition and included the standard message with no additional justification. Building on Capraro and Barcelo (2020), the message was as follows:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering.”

The image that accompanied the message for this condition was:

Protection from Harm (Self) (Condition 2)

This condition highlighted the liberal moral value ‘harm’ as justification for engaging in prevention behaviors. It specifically stated:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because it will keep you safe.”

The image that accompanied the message for this condition was:

Protection from harm (community) (condition 3)

This condition focused on preventing harm to others as the justification for wearing a mask or face covering. Although this framing touches on a liberal value (harm) and thus should be effective for Democrats, it is targeted toward the community which is a conservative moral foundation (ingroup-loyalty) and thus may also be effective among Republicans (Graham et al., 2009). This condition included the following message:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because it will keep our communities safe.”

The image that accompanied the message for this condition was:

Patriotic duty (condition 4)

This condition was designed to tap into the moral foundation of ‘ingroup-loyalty’ at a broader level—i.e., making patriotic sacrifices for one's country. As noted, loyalty and sacrifice for one's group are more important to conservatives than liberals, and thus this may be more effective when targeted at Republicans. This condition included the following message:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because it is our patriotic duty to make sacrifices for our great country.”

The image that accompanied the message for this condition was:

Purity (condition 5)

This condition employed the conservative moral value ‘purity’, which is based in the psychological desire to avoid contamination, and thus should be more effective when targeted at Republicans. This condition included the following message:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because it will keep our bodies from being contaminated by a disgusting virus.”

The image that accompanied the message for this condition was:

Reviving the economy (condition 6)

This condition highlighted the importance of following health guidelines for a successful reopening of the economy. It may be especially relevant to Republicans as polls show that two-thirds of Republicans view the pandemic as more of an economic crisis than a health crisis (Wronski, 2020). Participants read:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because it will help us to reopen our economy more quickly.”

The image that accompanied the message for this condition was:

Threat (Condition 7)

This condition emphasized the threat that COVID-19 continues to pose to Americans and the severity of the potential consequences of contracting the virus. It aimed to activate threat, given that Republicans have ironically been less likely to perceive COVID-19 as threatening as compared to Democrats (Blood & Swanson, 2020; Mitchell et al., 2020). This prime may also be effective among Democrats as well since threat has been shown to tighten individuals and promote norm abidance more generally (Gelfand et al., 2011; Jackson et al., 2019). The message for this condition read:

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because COVID-19 has killed over 211,000 Americans and continues to spread rapidly.” (Note: the number of deaths at survey launch was 211,000. We kept this number up to date over the course of data collection.)

The image that accompanied the message for this condition was:

Scientific Evidence (Condition 8)

As the messages from health and political officials regarding the importance of masks were unclear and often conflicting during earlier stages of the COVID-19 crisis (Breuninger, 2020; Gregorian, 2020), this condition emphasized that there is clear scientific evidence showing that masks effectively reduce the spread of the virus (CDC, 2020), and may be effective for Republicans and Democrats (but see footnote 2).

“Months after the start of the COVID-19 pandemic, many areas of the U.S. are opening to some degree and some segments of the population are starting to move around relatively freely. However, since a cure for COVID-19 has not been found and COVID-19 remains a serious threat, it is important to wear a mask or face covering because scientific evidence has proven that they can effectively prevent the spread of the virus.”

The image that accompanied the message for this condition was3 :

Pilot Data.

Prior to launching the intervention tournament, we sampled 601 American citizens through the online platform Prolific Academic (298 Republicans, 303 Democrats; M age = 35.7, SD age = 13.6; 51.2% women, 47.3% men, 1.5% other; 76.5% White, 8.2% Asian, 7.5% Black, 4.8% Hispanic, 3% multiracial/other) to pilot the stimuli. The purpose of the pilot was to ensure our manipulations were valid and not to specifically test the hypotheses. We asked all participants to evaluate each condition in a randomized order on the degree to which the text and image reflected the message. We asked participants to rate on a scale from 1 (Not at all) to 3 (Moderately) to 5 (Extremely) “to what extent do the text and image of each condition convey that wearing a mask or face covering is important because …” (with the remaining part of the sentence reflecting each particular condition; e.g., because it will protect you from harm (in the Protection from Harm (Self) condition), and associated phrases for each condition). We also asked participants whether the text and image were respectful using the question: “Do you consider this message to be…” Disrespectful (1) to Respectful (5). Finally, we asked each participant to choose the three messages that they thought were most effective and the three that were least effective. Prior to our data collection, our criteria for excluding a condition were: a) if either the Democrats or Republicans sample's rating on how well the text and image reflected the message was significantly lower than the scale mid-point (3); b) if the rating on respectfulness of the message was significantly lower than the scale mid-point; or c) if the condition was rated in the top three for ineffectiveness by both Republicans and Democrats.

In addition to the seven experimental conditions and control condition depicted above, we also piloted four additional conditions. Two of these conditions were designed to appeal to the moral authority of religion to encourage mask wearing. Since research has found that views of God as punitive versus loving are influenced by threat (Caluori, Jackson, Gray, & Gelfand, 2020), we designed conditions that highlighted both the loving and punishing sides of God: “because God wants us to protect each other” (Loving God) and “because God tells us that it's a sin to harm each other” (Punishing God). We also tested a condition derived from cultural tightness-looseness theory, which suggests that people desire tighter norms and social order during times of threat (Gelfand et al., 2011; Jackson et al., 2019): “because it will help us to maintain a tight social order in our communities” (Tight Social Order). Finally, we tested a condition designed to appeal to the American value of freedom. Kachanoff et al. (2021) found that perceiving the coronavirus pandemic as a symbolic threat to one's American identity predicted less adherence to social distancing recommendations. Based on this, we designed a condition that aimed to reframe mask wearing as an action that would protect Americans' freedom: “because it protects our freedom to control our personal space” (Freedom).

Based on our criteria for exclusion, we examined whether the mean ratings for each condition were significantly lower than the mid-point (3) on the extent to which the text and image reflected the message and were respectful using one-tailed t-tests. We also calculated the percentage of people who listed the condition in the top three for ineffectiveness. All of the results are in a table presented in Appendix A. The results showed that the Protection from Harm (self), Protection from Harm (community), Patriotic Duty, Purity, Reviving the Economy, Threat, and Scientific Evidence conditions were all viable conditions (e.g., all of these conditions were not rated below the mid-point by either Democrats or Republicans on either item and were not rated among the three least effective by either Republicans or Democrats).

The four other conditions that we piloted, however, were excluded. The Punishing God condition was rated as below the mid-point on the text and image reflecting the message for the combined sample and among both Democrats and Republicans, was rated as below the mid-point on respectfulness among Democrats and was listed among the top 3 most ineffective messages by both Democrats and Republicans. The Loving God condition was also listed among the top 3 most ineffective messages by both Democrats and Republicans. The Tight Social Order condition was rated below the mid-point on whether the text and image reflect the message by Democrats and was listed among the top 3 most ineffective messages by both Democrats and Republicans. Finally, the Freedom condition was rated below the mid-point in terms of the image and text reflecting the message for the combined sample and for Democrats. Based on all of our criteria, we excluded the Punishing God, Loving God, Tight Social Order, and Freedom conditions.

We note that while the text and image for the Scientific Evidence condition were deemed acceptable based on our criteria, two anonymous reviewers suggested that the image (biceps and an X through a picture of the virus, as seen in Footnote 3) could be further improved. Based on this suggestion, we piloted a new image (N = 195) that was adapted from the Center for Disease Control (CDC) which shows the effectiveness of masks at blocking germs from others (Appendix B) through the online platform Prolific Academic (see image for Condition 8 above). This new image was also deemed acceptable and was used for the intervention tournament.4

Dependent Variables

Our dependent variables included a) mask wearing attitudes (see Table 1 ), b) mask wearing intentions (see Table 1), c) signing a pledge to commit to wearing a mask/face covering, and d) willingness to share the pledge on social media.

Table 1

Mask wearing attitude and intention items.

Measure
a) Mask Wearing Attitudes
Do you consider wearing a mask or face covering to be… (1 to 7 scale)
	1. Unimportant. . . . .Important
	2. Bad. . . . .Good
	3. Foolish. . . . .Wise
	4. Negative. . . . .Positive
	5. Undesirable. . . . .Desirable
	6. Unnecessary. . . . .Necessary
b) Mask Wearing Intentions (adapted from Capraro and Barcelo, (2020))
In the next week, I intend to… (1 – Not at all to 7 – Very much)
	1. wear a mask or face covering any time I leave home.
	2. wear a mask or face covering any time I am engaged in essential activities and/or work, and physical distancing and staying at home are not possible.
	3. wear a mask or face covering any time I'm around people outside my household.
	4. wear a mask or face covering any time I can't maintain 6 ft from others outside my household.
	5. wear a mask or face covering inside public buildings.
	6. wear a mask or face covering in public outdoor spaces.

We checked whether the items measuring Mask Wearing Attitudes (6 items) and Mask Wearing Intentions (6 items) formed single factors using maximum likelihood factor analysis with an oblimin rotation.5 A pilot study conducted through the online platform Prolific (N = 589)6 illustrated a clear one-factor solution and high reliability for both variables (Mask Wearing Attitudes: one-factor solution (λ1 = 5.17, λ2–6 < 1.0), accounting for 86.15% of the variance, item loadings of 0.69 or greater, α = 0.96; Mask Wearing Intentions: one-factor solution (λ1 = 4.41, λ2–6 < 1.0), accounting for 73.43% of the variance, item loadings of 0.70 or greater, α = 0.92). In the final sample, we confirmed that each scale was suited for factor analysis (KMO sampling adequacy coefficients >0.90, ps < 0.001). We confirmed a one-factor solution and high reliability for both variables (Mask Wearing Attitudes: one-factor solution (λ1 = 5.07, λ2–6 < 1.0), accounting for 84.50% of the variance, item loadings of 0.69 or greater, α = 0.95; Mask Wearing Intentions: one-factor solution (λ1 = 4.61, λ2–6 < 1.0), accounting for 76.82% of the variance, item loadings of 0.76 or greater, α = 0.93).

Behavioral Measure – Signing a Pledge

We included a behavioral measure that asks participants to sign (initials only) a pledge to wear a mask or face covering to prevent the spread of COVID-19. We designed a website (https://covidpledge.wixsite.com/sign) modeled after existing ones. Participants' signatures were recorded in a Qualtrics survey accessible only to the research team, which enabled participants to maintain anonymity. The pledge was presented in the survey with the following information:

Signing a pledge helps us successfully take action. We have created a pledge that people can sign to commit to wearing a mask or face covering, which helps prevent the spread of COVID-19.

If you sign the pledge, only your initials and today's date will be recorded.

This pledge is completely optional. If you are not interested, please continue to the next page.

Would you like to sign the pledge? If you are interested, the pledge (https://covidpledge.wixsite.com/sign) will open on the next page (we have embedded it into the survey for your convenience). Once you have signed, please continue with the survey.

Once directed to the page, participants then chose: “Yes, I would like to sign the pledge” or “No, I would not like to sign the pledge”.

Behavioral measure – sharing with social network

In addition to asking participants to sign a pledge, we also included a behavioral measure that asked participants to share the pledge with their social networks. To avoid potential privacy concerns, rather than having participants share the pledge directly from our survey, we embedded a link to our pledge website into the Qualtrics survey along with a “copy” button and the message: “Would you be willing to share this pledge with your social network? If yes, please copy the link to the pledge and share it with your social network when the survey is over.” Participants' sharing behavior was measured by whether or not they copy the link to the pledge.

Exploratory variables

For exploratory analyses, which were pre-registered, we included measures of age, family income (1 = Below 30 k, 2 = 30-60 k, 3 = 60-90 k, 4 = 90-120 k, 5 = Above 120 k), socioeconomic status (SES, “Which letter corresponds to where you think you stand in society?”, letters displayed vertically from A to K corresponding to different rungs of the McArthur ladder), gender (man or woman; responses to an option for other were excluded), religiosity (from 1 = not at all religious to 7 = extremely religious), race (White/Caucasian, Black, Hispanic, Asian, Pacific Islander, Multiracial, and other; for analyses, we compared White/Caucasian to the remaining categories), health conditions (e.g., cardiovascular diseases, diabetes, and cancer; from 0 to 5 and above), timing of the 2020 presidential election (before and after), mask mandate in one's state (yes or no), and stay-at-home order in one's state (yes or no). We measured perceived threat using three items on a response scale (“How concerned are you by the spread of the new coronavirus (COVID-19)?”, from 1 = not at all concerned to 7 = extremely concerned, “How dangerous do you think the Coronavirus is?”, from 1 = not at all dangerous to 7 = extremely dangerous, “How contagious do you think the Coronavirus is?”, from 1 = not at all contagious to 7 = extremely contagious). We measured objective COVID-19 cases and deaths at the county-level. We matched Zip Codes to county codes, then matched response dates to the nearest monthly data point for county cases and deaths per capita starting on the first day of data collection (from Oct 14, 2020 to Jan 14, 2021). To achieve a normal distribution of data, we log transformed cases and deaths. We also measured whether participants were infected by the Coronavirus (yes vs. no or unsure).

Power analysis

An a-priori power analysis for a MANOVA with 2 dependent variables (mask wearing attitudes and intentions), power of 0.95, α = 0.01, and a small effect size (Cohen's f  = 0.005) indicated that we needed a total sample of 4477 participants in order to detect statistically significant effects. The dependent variables in this power analysis were considered to be a function of the main effect of political party, the main effects of 7 dummy coded variables that directly compare each framing condition to the control condition, and the interaction effects between political party and each of the 7 dummy coded variables. For the dichotomous variables (pledge signing and willingness to share pledge on one's social network), a power analysis for a z-test for a 2 (pledge signing vs. not) X 2 (political ideology groups) X 8 (framing and control conditions) contingency table, a-priori power of 0.95, α = 0.01, and a small effect size (OR = 1.50) indicated that we need a total sample of at least 2703 participants in order to detect statistically significant effects. We used the standard small effect size in power analyses as we were not aware of any work that examines the effectiveness of the types of framings that we are implementing to encourage mask wearing among Republicans and Democrats. We also specified an α level of 0.01 to correct for multiple comparisons across tests.

Sampling

We contracted Qualtrics to recruit 5000 individuals for this study (2500 Republicans; 2500 Democrats) as our target sample size. Within each political group, participants were randomly assigned to one of our eight conditions. Within each condition, Qualtrics recruited a sample of Americans that fulfilled quotas for region, gender, age, race, and education level matched to the demographics of the specific political group. The Republican sample followed the demographic trends of Republican/Republican leaning voters: region (16% Northeast, 23% Midwest, 41% South, 23% West), gender (54% men, 46% women),7 race (81% White, 5% Black, 7% Hispanic, 7% Other), and education (35% HS or less, 35% some college, 19% college degree, 10% postgrad) (Pew Research Center, 2014, Pew Research Center, 2020a). The Democrat sample followed the demographic trends of Democrat/Democrat leaning voters: region (21% Northeast, 22% Midwest, 34% South, 26% West), gender (40% men, 60% women), race (59% White, 19% Black, 13% Hispanic, 8% Other), and education (28% HS or less, 31% some college, 22% college degree, 19% postgrad) (Pew Research Center, 2014, Pew Research Center, 2020a). As registered voters tend to be older than the average population, rather than using the age distribution of register voters, we used a censused-matched age distribution (provided by Qualtrics) for both political groups: 18–34 (~33%), 35–55 (~33%), 55+ (~33%). Recruiting a sample based on representative quotas within each condition, rather than across the whole sample, allowed us to compare the effects of the framings as closely as possible.

To recruit our sample, Qualtrics targeted respondents who have previously indicated that they are Republican or Democrat. To confirm that participants identified as Republican or Democrat, respondents began the survey by indicating their party affiliation (i.e., Republican, Democrat, Independent, other, or none). When we began the survey, we also asked participants to report where they fall on the political spectrum (1 = very liberal, 5 = neutral, 9 = very conservative), and participants who chose Republican and 6–9 on the political spectrum qualified as Republican and participants who chose Democrat and 1–4 on the political spectrum qualified as Democrat. However, we found that screening participants on both political party and political ideology made our rejection rate much higher than Qualitrics originally anticipated. Of our first 2865 respondents, 1690 (59%) were deemed ineligible due to their political party and/or ideology. Of 908 Republicans who responded, 604 (66.5%) were conservative, 194 (21.4%) were neutral, and 110 (12.1%) were liberal. Out of 1236 Democrats, only 571 (46.2%) classified themselves as liberal, 434 (35.1%) were neutral, and 231 (18.7%) were conservative. Due to the nested nature of our quotas (participants were divided by political party, then randomly assigned to an experimental condition, and then within each experimental condition, there were specific quotas for region, gender, age, race, and education level to gather a representative sample), we decided to deviate from our pre-registered plan and classified participants as Republican or Democrat based only on their self-reported political party in order to increase the feasibility of the data collection. Across the final sample, we found general trends toward matching among Republicans (conservative Republicans = 1869; moderate Republicans = 416; liberal Republicans = 178; mean ideology = 7.01, 95% CI [6.93, 7.09]) and Democrats (liberal Democrats = 1591; moderate Democrats = 599; conservative Democrats = 278; mean ideology = 3.51, 95% CI [3.43, 3.60]). We also found no evidence that ideology interacted with our interventions across the four dependent variables (ps > 0.05; Appendix C). We screened out respondents who were not US citizens and/or not Republican/Democrat. Finally, participants also reported their age, gender, race, education level, and region.

We excluded data from participants who failed any of our quality/attention checks, as pre-registered on OSF. These include quality checks provided by Qualtrics, such as checks for speeding through the survey or straight-lining (e.g., answering all “2”s). We also included our own attention checks throughout the survey that ask people to choose a specific response option (e.g., “Choose strongly agree” or “Choose the response option furthest to the right”). Finally, we included an open-ended question at the end of the survey that asks participants to write the first five words that come to mind when they think about the meaning of masks. This served as a quality check, and we excluded and replaced participants who wrote gibberish (i.e., non-words or irrelevant, grammatically incorrect phrases). Thus, any participant who failed even one of these checks was excluded. Qualtrics replaced participants who failed any of these attention/quality checks with new participants in the correct conditions and quotas.

We screened the data to identify participants who met at least one of the four predetermined exclusion criteria discussed above. Specifically, participants' data were excluded if a) the time they took to complete the survey was less than half the median of the first 100 responses or 3 standard deviations above the median8 ; b) the same answer choice was selected across most or all of the questions (i.e. straight-lining), as per Qualtrics policy; c) they did not choose the indicated response to one of the attention check questions interleaved among other questions (e.g. “From the following answers, please select “3”); and d) they did not pass the quality check question (“What are the first five words that come to mind when you think about masks?”). Responses were considered low-quality if they met any of the following criteria: 1) wrote gibberish/non-words in any of the five questions, 2) wrote non-responses (e.g., “none”, “don't know”) in three or more of the questions, 3) repeated the same word(s) multiple times, or 4) wrote responses that are irrelevant to the question. After cleaning the data, our final sample size consisted of 4931 participants (2463 Republicans; 2468 Democrats). Data collection occurred from Oct 14, 2020, to Jan 14, 2021.

Results

The descriptive statistics for all demographic variables (i.e., gender, age, race, region, religion, education, family income, and socioeconomic status) within each political group are presented in Table 2 . The sample characteristics are representative of Republicans and Democrats in the US (deviation from quotas <3%). The descriptive statistics for all of the outcomes and covariates within each political group are presented in Table 3 and the outcomes across the experimental conditions are presented in Table 4 .

Table 2

Sample Characteristics across Political Party.

Name	Republican (n = 2463)	Democrat (n = 2468)
Gender
Man	1261 (51%)	978 (40%)
Woman	1202 (49%)	1490 (60%)
Age	47.87 [47.14, 48.60]	45.71 [45.98, 46.44]
Race
Asian	132 (5%)	133 (5%)
Black	118 (5%)	499 (20%)
White	2006 (81%)	1460 (59%)
Hispanic	153 (6%)	309 (13%)
Multiracial	25 (1%)	49 (2%)
Other	23 (<1%)	12 (<1%)
Pacific Islander	6 (<1%)	6 (<1%)
Region
Midwest	570 (23%)	537 (22%)
Northeast	396 (16%)	516 (21%)
South	1040 (42%)	835 (34%)
West	457 (19%)	580 (24%)
Religiosity	4.58 [4.50, 4.66]	3.76 [3.67, 3.84]
Religion
Agnostic	84 (3%)	216 (9%)
Atheist	90 (4%)	226 (9%)
Buddhist	15 (1%)	36 (1%)
Christian	2073 (83%)	1658 (66%)
Hindu	10 (<1%)	17 (<1%)
Jewish	49 (2%)	83 (3%)
Muslim	26 (1%)	56 (2%)
Sikh	2 (<1%)	4 (<1%)
Other	160 (6%)	218 (9%)
Highest Education
College (No Degree)	592 (24%)	601 (24%)
Graduate (4-year)	506 (21%)	560 (23%)
Grammar School	22 (1%)	10 (<1%)
High School	872 (35%)	712 (29%)
M.A./M.S.	191 (8%)	340 (14%)
M.D./J.D.	35 (1%)	50 (2%)
Ph.D.	31 (1%)	37 (2c%)
Technical/Vocational	214 (9%)	158 (6%)
Family Income	2.51 [2.46, 2.56]	2.46 [2.41, 2.51]
Socioeconomic Status (SES)	6.07 [5.98, 6.16]	5.82 [5.73, 5.92]

Note: Values are either counts (relative frequencies) or means [95% CI]. Values for Religion do not add up to the N for each political party because participants were allowed to choose multiple religions.

Table 3

Descriptive Statistics on Dependent Variables and Covariates across Political Party.

Name	Republicans (n = 2463)	Democrats (n = 2468)
Outcomes
Attitude	5.64 [5.57, 5.71]	6.66 [6.63, 6.69]
Intention	5.73 [5.67, 5.80]	6.63 [6.60, 6.66]
Signed Pledge
Yes	1330 (54%)	1755 (71%)
No	1133 (46%)	713 (29%)
Shared Pledge
Yes	418 (17%)	624 (25%)
No	2045 (83%)	1844 (75%)
Covariates
Election
Before	1487 (60%)	1732 (70%)
After	976 (40%)	736 (30%)
Stay-at-Home Order
Yes	665 (27%)	847 (34%)
No	1564 (64%)	1376 (56%)
Unsure	234 (10%)	245 (10%)
Mask Mandate
Yes	1962 (80%)	2093 (85%)
No	412 (17%)	297 (12%)
Unsure	89 (4%)	78 (3%)
Infection (COVID-19)
Yes	120 (5%)	89 (4%)
No (Untested)	1520 (62%)	1451 (59%)
No (Tested)	776 (32%)	865 (35%)
Unsure	47 (2%)	63 (3%)
Health Conditions	0.37 [0.34, 0.40]	0.39 [0.36, 0.42]
Perceived Threat	4.98 [4.91, 5.05]	6.17 [6.13, 6.21]
Cases (Logged)	7.87 [7.85, 7.90]	7.82 [7.80, 7.85]
Deaths (Logged)	7.44 [7.40, 7.50]	7.51 [7.46, 7.55]

Note: Values are either counts (relative frequencies) or means [95% CI].

Table 4

Descriptive Statistics on Dependent Variables across Interventions and Control Condition.

	Attitudes (Mean; 95% CI)	Intentions (Mean; 95% CI)	Signed Pledge (Y / N)	Shared Pledge (Y / N)
Control	6.18 [6.07, 6.29]	6.18 [6.08, 6.29]	392 / 222 (64%)	129 / 485 (21%)
Harm (Self)	6.16 [6.04, 6.27]	6.22 [6.11, 6.32]	408 / 212 (66%)	137 / 483 (22%)
Harm (Community)	6.20 [6.09, 6.31]	6.16 [6.05, 6.27]	398 / 215 (65%)	131 / 482 (21%)
Patriotic Duty	6.08 [5.97,6.20]	6.18 [6.07, 6.28]	379 / 235 (62%)	129 / 485 (21%)
Purity	6.15 [6.03, 6.27]	6.19 [6.08, 6.30]	386 / 234 (62%)	125 / 495 (20%)
Economy	6.20 [6.08. 6.31]	6.24 [6.14, 6.34]	393 / 220 (64%)	130 / 483 (21%)
Threat	6.17 [6.06, 6.28]	6.23 [6.13, 6.33]	358 / 260 (58%)	145 / 473 (24%)
Scientific Evidence	6.05 [5.93, 6.18]	6.06 [5.94, 6.18]	371 / 248 (60%)	116 / 503 (19%)

Note: Values are either counts (relative frequencies) or means [95% CI]. Y = Yes. N = No.

We conducted a MANOVA with the political party variable, the categorical intervention variable that contrasts each of the intervention conditions to the control condition using SPSS's contrast feature, and the interaction terms between the contrast coded intervention variable and political party as predictors.9 We included mask wearing attitudes and mask wearing intentions as dependent variables.

We found a statistically significant main effect of political party such that Republicans reported less positive attitudes (mean difference = −1.02, 95% CI [−1.09, −0.94], p < .001) and lower intentions (mean difference = −0.89, 95% CI [−0.96, −0.82], p < .001) to wear a mask compared to Democrats (η2 = 0.13, p < .001). We found no statistically significant effect of the intervention conditions on attitudes and intentions to wear a mask (p = .157). We also found no statistically significant interaction between the intervention conditions and political party (p = .912). The results of the MANOVA are presented in Table 5 . Effect sizes are displayed as unstandardized mean differences between the intervention and control condition on attitudes and intentions. The effects of the intervention and control conditions on attitudes and intentions in the original response scale across the entire sample are illustrated in Fig. 1, Fig. 3 and the effects across political parties are illustrated in Fig. 2, Fig. 4 . Because none of the interventions were significantly different from the control condition, there were no “winning” conditions in the tournament.

Table 5

MANOVA on Attitudes and Intentions to Wear a Mask.

			Mean Difference [95% CI]
Predictor	V	F(df)	Attitudes	Intentions	Ps
(Intercept)	0.962	61,531.75 (2; 4914)			< 0.001
Political Party (PP)	0.130	366.26 (2; 4914)			< 0.001
Intervention vs. Control(All Participants)	0.004	1.37 (14; 9830)			0.157
Harm (Self)			−0.02 [−0.17, 0.13]	0.04 [−0.10, 0.18]	.794a, .591i
Harm (Community)			0.03 [−0.13, 0.18]	−0.02 [−0.16, 0.12]	.751a, .756i
Patriotic Duty			−0.09 [−0.25, 0.06]	−0.001 [−0.14, 0.14]	.237a, .985i
Purity			−0.03 [−0.18, 0.12]	0.006 [−0.14, 0.15]	.708a, .929i
Economy			0.02 [−0.13, 0.17]	0.07 [−0.08, 0.21]	.785a, .365i
Threat			−0.004 [−0.16, 0.15]	0.06 [−0.09, 0.20]	.954a, .447i
Scientific Evidence			−0.13 [−0.28, 0.02]	−0.13 [−0.27, 0.02]	.094a, .082i
Intervention vs. Control(Republicans)	0.006	0.98 (14; 4910)			0.474
Harm (Self)			−0.02 [−0.31, 0.26]	0.06 [−0.21, 0.32]	.866a, .679i
Harm (Community)			0.08 [−0.20, 0.37]	0.03 [−0.24, 0.29]	.572a, .854i
Patriotic Duty			−0.16 [−0.45, 0.12]	−0.03 [−0.29, 0.23]	.263a, .828i
Purity			0.006 [−0.28, 0.29]	0.03 [−0.23, 0.29]	.969a, .828i
Economy			0.08 [−0.20, 0.37]	0.13 [−0.14, 0.39]	.572a, .347i
Threat			0.07 [−0.21, 0.35]	0.14 [−0.12, 0.40]	.631a, .295i
Scientific Evidence			−0.16 [−0.45, 0.12]	−0.16 [−0.42, 0.10]	.263a, .223i
Intervention vs. Control (Democrats)	0.005	0.85 (14; 4920)			0.613
Harm (Self)			−0.02 [−0.13, 0.10]	0.02 [−0.09, 0.13]	.778a, .690i
Harm (Community)			−0.03 [−0.15, 0.08]	−0.07 [−0.18, 0.04]	.567a, .221i
Patriotic Duty			−0.02 [−0.13, 0.09]	0.03 [−0.09, 0.14]	.709a, .648i
Purity			−0.06 [−0.18, 0.05]	−0.02 [−0.13, 0.10]	.263a, .777i
Economy			−0.04 [−0.15, 0.07]	0.006 [−0.11, 0.12]	.493a, .920i
Threat			−0.08 [−0.19, 0.03]	−0.03 [−0.14, 0.08]	.171a, .603i
Scientific Evidence			−0.10 [−0.21, 0.01]	−0.09 [−0.20, 0.02]	.088a, .119i
PP * Intervention vs. Control (Republicans vs. Democrats)	0.002	0.54 (14; 9830)			0.912
Harm (Self)			0 [−0.31, 0.31]	0.04 [−0.25, 0.33]	1 a, .785i
Harm (Community)			0.11 [−0.20, 0.42]	0.10 [−0.19, 0.39]	.483a, .495i
Patriotic Duty			−0.14 [−0.45, 0.17]	−0.06 [−0.34, 0.22]	.369a, .679i
Purity			0.07 [−0.24, 0.37]	0.05 [−0.23, 0.33]	.674a, .730i
Economy			0.12 [−0.19, 0.43]	0.12 [−0.16, 0.41]	.441a, .400i
Threat			0.15 [−0.15, 0.45]	0.17 [−0.11, 0.45]	.328a, .238i
Scientific Evidence			−0.06 [−0.37, 0.25]	−0.07 [−0.35, 0.21]	.700a, .627i

Note: V = Pillai's Trace. Subscript indicates the p-values for the hypothesis tests and corresponding mean differences describing the influence of the interventions on attitudes. Subscript indicates the p-values for hypothesis tests and corresponding mean differences describing the influence of the interventions on intentions. The mean differences for the interaction between political party and the intervention vs. control condition represent the differences between the differences (z-tests).

Fig. 1

Attitudes to Wear a Mask across Intervention Conditions.

Fig. 3

Intentions to Wear a Mask across Intervention Conditions.

Fig. 2

Attitudes to Wear a Mask across Intervention Conditions and Political Parties.

Fig. 4

Intentions to Wear a Mask across Intervention Conditions and Political Parties.

Finally, we conducted a hierarchical loglinear model (HLM) analysis to examine whether there was a three-way interaction between the seven framing conditions (relative to the control condition), participants' political party (Democrat or Republican), and participants' pledge signing behavior (signed or not).

The HLM is an extension of the χ2 test and models a multidimensional structure of categorical variables using an iterative proportional-fitting algorithm developed for SPSS (Version 24; IBM, 2021). The χ2 test evaluates the difference between observed and expected cell counts of one or more categorical variables given a probabilistic distribution of values. With a single variable, the χ2 tests if the difference between the observed and expected cell counts across the response categories. With more than one variable, the χ2 test evaluates a continency table (i.e., pairs of responses across categorical variables) and tests for independence between variables. Unlike a MANOVA, a HLM does not differentiate between dependent and independent variables. In line with the tradition of χ2 tests, we specify the model for the contingency table as a variable-by-variable interaction (e.g., row * column) rather than label them as dependent and independent variables as we do in other sections of the paper.

The HLM is split up into K-way effects and partial associations. In the current study, the K-way effects represent χ2 tests for the main effects (pooled), two-way interactions (pooled), and the three-way interaction. The K-way effects include χ2 tests of lower order effects with and without the inclusion of higher order effects. The HLM decomposes the main effects and two-way interactions by providing partial associations or additional χ2 tests that evaluate each main effect and interaction. The main effects test whether the observed distribution of values for each of the variables (i.e., political party, intervention condition, and pledge signing) significantly differs from the expected distribution of values. The two-way interaction effects tests whether one of the variables (e.g., pledge signing) is predicted by one of the other two variables (e.g., political party or intervention condition). The three-way interaction effect tests whether one of the variables (e.g., pledge signing) is predicted by the interactive effect of the other two variables (e.g., intervention condition and political party).

We calculated the K-way effects, which tests if there are any main effects (K = 1), two-way interactions (K = 2), or a three-way interaction between signing the pledge, the intervention condition, and political party (K = 3). We found evidence for significant main effects and two-way effects (ps < 0.001) but not a three-way interaction (p = .577; Table 6 ). Because K-way effects are pooled across effects and intervention conditions, they provide only a limited amount of information. Accordingly, we used partial associations to help us better understand which main effects or interactions were significant overall, which were significant across intervention conditions, and which were significant within and across political parties (see Table 7 ). Effect sizes are displayed as odds ratios (OR), or the odds of an outcome compared to the odds of an alternative outcome. An odds ratio equal to 1 is equivalent to a probability of 50% (i.e., a coin flip). We found a statistically significant main effect for signing the pledge, such that there were statistically significantly greater odds of signing the pledge compared to not signing the pledge (OR = 1.30, 95% CI [1.27, 1.34], p < .001). We found a statistically significant two-way interaction between political party and signing the pledge, such that the odds of signing the pledge was lower among Republicans compared to Democrats (OR = 0.83, 95% CI [0.81, 0.85], p < .001). We found marginally significant evidence that the interventions (vs. the baseline control condition) influence pledge signing behavior (p = .064). There was a significant main effect for the threat intervention decreasing the odds of signing the pledge (OR = 0.90, 95% CI [0.84, 0.97], p = .008), an effect that was in the opposite direction of our predictions. The effects of the interventions and control condition on signing the pledge to wear a mask across the entire sample and across political parties are illustrated in Fig. 5, Fig. 6 .

Table 6

Hierarchical Loglinear Model (K-way Effects) on Signing a Pledge to Wear a Mask.

	K	df	Likelihood Ratio		Pearson
			χ2	p	χ2	p
K-way and Higher Order Effects	1	31	489.10	<0.001	473.39	<0.001
	2	22	174.30	<0.001	172.37	<0.001
	3	7	5.69	0.577	5.68	0.577
K-way Effects	1	9	314.80	<0.001	301.03	<0.001
	2	15	168.62	<0.001	166.68	<0.001
	3	7	5.69	0.577	5.68	0.577

Note. Number of iterations = 2. K-way and Higher Order Effects include tests of all effects at level K and higher.

Table 7

Hierarchical Loglinear Model (Partial Associations) on Signing a Pledge to Wear a Mask.

Effect	df	Partial χ2	OR	p
Signed Pledge	1	314.68	1.30 [1.27, 1.34]	<0.001
Political Party (PP)	1	0.005		0.943
Intervention	7	0.11		1
PP * Signed Pledge	1	155.52	0.83 [0.81, 0.85]	<0.001
PP * Intervention vs. Control	7	0.527		0.999
Intervention vs. Control * Signed Pledge(All Participants)	7	13.34		0.064
Protection from Harm (Self)			1.08 [1.00, 1.17]	0.053
Protection from Harm (Community)			1.05 [0.97, 1.13]	0.238
Patriotic Duty			0.98 [0.91, 1.06]	0.689
Purity			0.99 [0.92, 1.07]	0.868
Reviving the Economy			1.04 [0.96, 1.12]	0.389
Threat			0.90 [0.84, 0.97]	0.008
Scientific Evidence			0.94 [0.87, 1.02]	0.134
Intervention vs. Control * Signed Pledge(Republicans)	7	7.70		0.360
Protection from Harm (Self)			0.93 [0.68, 1.28]	0.663
Protection from Harm (Community)			1.07 [0.78, 1.48]	0.666
Patriotic Duty			0.80 [0.58, 1.10]	0.177
Purity			0.85 [0.62, 1.16]	0.301
Reviving the Economy			0.95 [0.69, 1.30]	0.729
Threat			0.79 [0.58, 1.09]	0.151
Scientific Evidence			0.77 [0.56, 1.06]	0.112
Intervention vs. Control * Signed Pledge(Democrats)	7	11.33		0.125
Protection from Harm (Self)			1.35 [0.94, 1.93]	0.102
Protection from Harm (Community)			1.03 [0.73, 1.46]	0.871
Patriotic Duty			1.07 [0.75, 1.51]	0.713
Purity			1.05 [0.74, 1.49]	0.771
Reviving the Economy			1.11 [0.78, 1.58]	0.556
Threat			0.77 [0.55, 1.07]	0.122
Scientific Evidence			0.94 [0.66, 1.32]	0.701
PP * Intervention vs. Control * Signed Pledge(Republicans vs. Democrats)	7	5.69		0.577
Protection from Harm (Self)			1.06 [0.98, 1.15]	0.169
Protection from Harm (Community)			0.95 [0.88, 1.03]	0.247
Patriotic Duty			1.04 [0.96, 1.12]	0.378
Purity			1.02 [0.94, 1.10]	0.635
Reviving the Economy			1.00 [0.93, 1.09]	0.912
Threat			0.96 [0.89, 1.03]	0.252
Scientific Evidence			1.01 [0.94, 1.09]	0.763

Note: OR = Odds ratios [95% CIs] and chi-square (χ2) tests for the Republican and Democrat subgroups were calculated using logistic regression.

Fig. 5

Probability of Signing the Pledge for Each Intervention Condition.

Fig. 6

Probability of Signing the Pledge across Intervention Condition and Political Party.

We next conducted the same analyses for whether participants were willing to share the pledge with their social network. For the K-way effects, we again found main effects and two-way interactions (ps < 0.001) but no three-way interaction (p = .435) (Table 8 ). For the partial associations (Table 9 ), we found a statistically significant main effect for sharing the pledge, such that there were statistically significantly lower odds of sharing the pledge compared to not sharing the pledge (OR = 0.51, 95% CI [0.53, 0.50], p < .001). We also found a statistically significant interaction between political party and sharing the pledge, such that the odds of sharing the pledge was lower among Republicans compared to Democrats (OR = 0.88, 95% CI [0.85, 0.92], p < .001). There was no statistically significant evidence that the intervention (vs. the control) conditions influenced pledge sharing behavior (p = .653). The effects of the interventions and baseline control condition on sharing the pledge to wear a mask are illustrated across the entire sample (Fig. 5, Fig. 7 ) and by political party (Fig. 6, Fig. 8). As with mask attitudes and intentions, because none of the interventions were significantly different from the control condition, there were no “winning” conditions in the tournament for signing and sharing the pledge.

Table 8

Hierarchical Loglinear Model (K-way Effects) on Sharing a Pledge to Wear a Mask.

	K	df	Likelihood Ratio		Pearson
			χ2	p	χ2	p
K-way and Higher Order Effects	1	31	1813.67	<0.001	1686.50	<0.001
	2	22	63.52	<0.001	63.41	<0.001
	3	7	6.94	0.435	6.94	0.435
K-way Effects	1	9	1750.15	<0.001	1623.09	<0.001
	2	15	56.58	<0.001	56.47	<0.001
	3	7	6.94	0.435	6.94	0.435

Note. Number of iterations = 2. K-way and Higher Order Effects include tests of all effects at level K and higher.

Table 9

Hierarchical Loglinear Model (Partial Associations) on Sharing a Pledge to Wear a Mask.

Effect	df	Partial χ2	OR	p
Signed Pledge	1	1750.03	0.51 [0.53, 0.50]	<0.001
Political Party (PP)	1	0.01		0.943
Intervention	7	0.11		1
PP * Shared Pledge	1	51.56	0.88 [0.85, 0.92]	<0.001
PP * Intervention vs. Control	7	0.318		1
Intervention vs. Control * Shared Pledge(All Participants)	7	5.06		0.653
Protection from Harm (Self)			1.03 [0.94, 1.13]	0.510
Protection from Harm (Community)			1.02 [0.93, 1.11]	0.743
Patriotic Duty			0.99 [0.90, 1.08]	0.761
Purity			0.97 [0.88, 1.06]	0.487
Reviving the Economy			1.00 [0.91, 1.09]	0.957
Threat			1.07 [0.98, 1.17]	0.135
Scientific Evidence			0.94 [0.85, 1.03]	0.169
Intervention vs. Control * Shared Pledge(Republicans)	7	4.09		0.769
Protection from Harm (Self)			0.96 [0.63, 1.44]	0.827
Protection from Harm (Community)			1.03 [0.69, 1.55]	0.881
Patriotic Duty			0.76 [0.50, 1.17]	0.211
Purity			0.79 [0.52, 1.21]	0.288
Reviving the Economy			0.82 [0.54, 1.25]	0.351
Threat			0.99 [0.66, 1.49]	0.976
Scientific Evidence			0.90 [0.60, 1.37]	0.631
Intervention vs. Control * Shared Pledge(Democrats)	7	7.91		0.341
Protection from Harm (Self)			1.17 [0.81, 1.68]	0.404
Protection from Harm (Community)			1.02 [0.70, 1.47]	0.928
Patriotic Duty			1.22 [0.85, 1.76]	0.277
Purity			1.09 [0.75, 1.57]	0.656
Reviving the Economy			1.19 [0.83, 1.72]	0.341
Threat			1.30 [0.91, 1.87]	0.148
Scientific Evidence			0.84 [0.57, 1.23]	0.361
PP * Intervention vs. Control * Shared Pledge(Republicans vs. Democrats)	7	6.94		0.435
Protection from Harm (Self)			1.00 [0.92, 1.10]	0.969
Protection from Harm (Community)			0.95 [0.87, 1.04]	0.265
Patriotic Duty			1.07 [0.98, 1.18]	0.145
Purity			1.03 [0.94, 1.13]	0.535
Reviving the Economy			1.05 [0.95, 1.15]	0.332
Threat			1.02 [0.93, 1.11]	0.666
Scientific Evidence			0.94 [0.85, 1.03]	0.164

Note: OR = Odds ratios [95% CIs] and chi-square (χ2) tests for the Republican and Democrat subgroups were calculated using logistic regression.

Fig. 7

Probability of Sharing the Pledge across Each Intervention Condition.

Fig. 8

Probability of Sharing the Pledge across Intervention Condition and Political Party.

Other exploratory measures: threat and demographics

Our exploratory measures and analyses were pre-registered on OSF (https://osf.io/e89fv). We explored whether the interventions were more or less effective based on sample demographics such as income, age, gender, race, religiosity, and pre-existing health conditions. We included a time variable for the 2020 presidential election (before or after), which occurred during data collection. We also examined whether participants' reports of whether masks and stay-at-home orders were mandatory in their communities to see if interventions were more or less effective based on community responses. Finally, since communities have been differentially affected by COVID-19, we measured the number of county-level cases and deaths using zip codes, as well as perceptions of COVID-19 threat and personal infection, to see if the interventions were more or less effective based on perceived and actual levels of threat.10 Although the demographic variables, the community response variables, perceived threat, and county-level COVID-19 cases all correlated with some of the outcomes (Table 10 ), we still found no statistically significant evidence that our interventions produced an effect when including these covariates in our models (Table 11, Table 12, Table 13 ).

Table 10

Pearson correlations between the dependent variables and covariates.

	Attitude		Intention	Signed Pledge	Shared Pledge
Age		0.13***	0.10***	0.10***	−0.05**
Family Income		0.04*	0.01	0.02	0.09***
SES		0.05**	0.00	0.03	0.05**
Gender (Male vs. Female)		0.06***	0.09***	0.03†	−0.06***
Religiosity		−0.03	−0.03	−0.01	0.06***
Race (White vs Non-White)		−0.11***	−0.14***	−0.05***	−0.04**
Health Conditions		0.06***	0.05***	0.09***	0.04*
Election		−0.11***	−0.11***	−0.06***	0.04**
Mask Mandate		0.14***	0.19***	0.05***	0.07***
Stay-at-home Order		0.16***	0.15***	0.11***	0.14***
Perceived Threat		0.73***	0.71***	0.37***	0.15***
Cases		−0.05***	−0.05***	−0.03*	0.04**
Deaths		0.00	−0.03	−0.02	0.00
Had COVID		−0.05**	−0.05***	−0.01	0.01

Note: † p = .05; * = p < .05; ** p < .01; *** p < .001.

Table 11

MANOVA with Covariates on Attitudes and Intentions to Wear a Mask.

			Adjusted Mean Diff. [95% CI]
Predictor	V	F(df)	Attitudes	Intentions	p
(Intercept)	0.04	78.31 (2, 4315)			<0.001
Intervention vs. Control	0.003	0.86 (14; 8632)			0.605
Harm (Self)			0.01 [−0.11, 0.12]	0.07 [−0.04, 0.18]	.911a, .198i
Harm (Community)			0.07 [−0.04, 0.19]	0.04 [−0.07, 0.16]	.225a, .433i
Patriotic Duty			−0.04 [−0.16, 0.07]	0.05 [−0.06, 0.16]	.452a, .354i
Purity			0.02 [−0.10, 0.13]	0.07 [−0.05, 0.18]	.755a, .244i
Economy			0.01 [−0.11, 0.12]	0.05 [−0.06, 0.16]	.901a, .376i
Threat			−0.002 [−0.12, 0.11]	0.06 [−0.05, 0.17]	.977a, .260i
Scientific Evidence			−0.04 [−0.16, 0.07]	−0.01 [−0.13, 0.10]	.451a, .800i
Political Party (PP)	0.01	16.16 (2; 4315)			<0.001
PP * Intervention vs. Control	0.002	0.55 (14; 8632)			0.903
Age	0.01	17.37 (2; 4315)			<0.001
Family Income	0.002	4.92 (2; 4315)			0.007
SES	0.003	7.18 (2; 4315)			0.001
Gender (Male vs. Female)	0.001	2.93 (2; 4315)			0.054
Race (White vs. Non-White)	0.01	12.31 (2; 4315)			<0.001
Health Conditions	0.001	2.01 (2; 4315)			0.135
Election	0.001	2.21 (2; 4315)			0.110
Mask Mandate	0.021	46.86 (2; 4315)			<0.001
Stay-At-Home Order	< 0.001	0.39 (2; 4315)			0.676
Perceived Threat	0.497	2131.13 (2; 4315)			<0.001
Cases	0.001	1.41 (2; 4315)			0.245
Had COVID	0.003	5.56 (2; 4315)			0.004

Note: V = Pillai's Trace. Subscript indicates the p-values for the hypothesis tests and corresponding mean differences describing the influence of the interventions on attitudes. Subscript indicates the p-values for hypothesis tests and corresponding mean differences describing the influence of the interventions on intentions.

Table 12

Logistic Regression with Covariates on Signing a Pledge to Wear a Mask.

	b	SE	Wald	df	p	OR	95% CI
							Lower	Upper
(Constant)	−2.23	0.54	17.35	1	< 0.001	0.11
Intervention vs. Control			6.23	7	0.514
Harm (Self)	0.02	0.19	0.01		0.921	1.02	0.70	1.48
Harm (Community)	0.05	0.19	0.06	1	0.812	1.05	0.72	1.52
Patriotic Duty	−0.12	0.19	0.38	1	0.539	0.89	0.61	1.29
Purity	−0.12	0.19	0.39	1	0.533	0.89	0.61	1.29
Economy	−0.20	0.19	1.11	1	0.292	0.82	0.56	1.19
Threat	−0.30	0.19	2.42	1	0.120	0.74	0.51	1.08
Scientific Evidence	−0.24	0.19	1.58	1	0.209	0.79	0.54	1.14
Political Party (PP)	0.13	0.08	3.10	1	0.078	1.14	0.99	1.33
PP * Intervention vs. Control			3.54	7	0.831
Age	0.01	0.00	14.94	1	< 0.001	1.01	1.00	1.01
Gender (Male vs. Female)	−0.03	0.07	0.18	1	0.670	0.97	0.85	1.11
Race (White vs. Non-White)	−0.09	0.08	1.25	1	0.265	0.92	0.78	1.07
Health Conditions	0.14	0.05	8.86	1	0.003	1.15	1.05	1.26
Stay-At-Home Order	0.19	0.08	6.34	1	0.012	1.21	1.04	1.41
Mask Mandate	0.04	0.10	0.17	1	0.682	1.04	0.86	1.26
Perceived Threat	0.51	0.03	351.75	1	< 0.001	1.66	1.57	1.75
Cases	−0.06	0.06	1.02	1	0.313	0.94	0.83	1.06

Note: OR = Odds ratio.

Table 13

Logistic Regression with Covariates on Sharing a Pledge to Wear a Mask.

	b	SE	Wald	df	p	OR	95% CI
							Lower	Upper
(Constant)	−4.37	0.63	47.82	1	< 0.001	0.01
Intervention vs. Control			4.31	7	0.743
Harm (Self)	0.07	0.23	0.11	1	0.744	1.08	0.69	1.68
Harm (Community)	0.16	0.23	0.53	1	0.467	1.18	0.76	1.84
Patriotic Duty	−0.08	0.24	0.12	1	0.728	0.92	0.58	1.46
Economy	−0.23	0.24	0.92	1	0.337	0.80	0.50	1.27
Purity	−0.19	0.24	0.65	1	0.419	0.83	0.52	1.31
Threat	0.02	0.23	0.01	1	0.934	1.02	0.65	1.60
Scientific Evidence	−0.05	0.24	0.05	1	0.823	0.95	0.60	1.51
Political Party (PP)	0.36	0.09	17.19	1	< 0.001	1.43	1.21	1.69
PP * Intervention vs. Control			6.33	7	0.502
Age	−0.01	0.00	11.43	1	0.001	0.99	0.99	1.00
Family Income	0.12	0.03	14.19	1	< 0.001	1.13	1.06	1.20
SES	−0.01	0.02	0.45	1	0.501	0.99	0.96	1.02
Gender (Male vs. Female)	−0.28	0.08	11.57	1	0.002	0.76	0.64	0.89
Religiosity	0.08	0.02	17.87	1	< 0.001	1.08	1.04	1.13
Race (White vs. Non-White)	−0.06	0.09	0.52	1	0.471	0.94	0.79	1.11
Health Conditions	0.01	0.05	0.05	1	0.815	1.01	0.92	1.11
Stay-at-home Order	0.33	0.08	16.23	1	< 0.001	1.39	1.19	1.64
Mask Mandate	0.37	0.12	9.11	1	0.003	1.45	1.14	1.85
Perceived Threat	0.26	0.03	57.48	1	< 0.001	1.30	1.21	1.39
Cases	0.11	0.07	2.47	1	0.116	1.12	0.97	1.28

Note: OR = Odds ratio.

Finally, we ran a pre-registered image-only control condition of a mask with no text promoting the importance of wearing face masks in a separate study (https://osf.io/e89fv, see Appendix D). With this new control condition, we tested whether our original control condition with the text and an image may have been too strong given that it did state that wearing masks was important (without a rationale for why which was the purpose of the intervention conditions) (N = 636; Republican = 316; Democrat = 320; mean attitudes = 6.06, 95% CI [5.95, 6.18]; mean intentions = 6.23, 95% CI [6.13, 6.32]; signed pledge [Y, N] = 401, 235 [63%]; shared pledge [Y, N] = 98,538 [15%]). We used the following image for this new control condition:

We found a statistically significant main effect for the interventions (vs. the new control condition) on attitudes and intentions to wear a mask (p = .001, Table D1). However, upon conducting multiple comparison tests, we found only one marginally statistically significant effect, such that the scientific evidence condition decreased intentions to wear a mask (mean difference = −0.17, 95% CI [−0.31, −0.03], p = .02); which was in the opposite direction of our hypothesis. We did not find any effects of the interventions on signing or sharing a pledge to wear a mask (Table D2, Table D3), nor did we find any interactions between the interventions and political party.

Table D1

MANOVA with Exploratory Control Condition on Attitudes and Intentions to Wear a Mask.

			Mean Difference [95% CI]
Predictor	V	F(df)	Attitude	Intention	p
(Intercept)	0.962	70,711.32 (2; 5548)2.01 (16; 11,098)			< 0.001
Intervention vs. Control2	0.006				0.010
Control1			0.12 [−0.04, 0.27]	−0.05 [−0.19, 0.09]	.13a, .51i
Harm (Self)			0.10 [−0.06, 0.25]	−0.01 [−0.15, 0.13]	.21a, .91i
Harm (Community)			0.14 [−0.01, 0.29]	−0.07 [−0.21, 0.07]	.07a, .33i
Patriotic Duty			0.02 [−0.13, 0.18]	−0.05 [−0.19, 0.09]	.75a, .49i
Economy			0.09 [−0.06, 0.24]	−0.04 [−0.18, 0.10]	.26a, .57i
Purity			0.14 [−0.01, 0.29]	0.02 [−0.12, 0.16]	.08a, .80i
Threat			0.11 [−0.04, 0.26]	0.01 [−0.13, 0.15]	.15a, .91i
Scientific Evidence			−0.01 [−0.17, 0.14]	−0.17 [−0.31, −0.03]	.86a, .02i
Political Party (PP)	0.132	422.74 (2; 5548)0.90 (16; 11,098)			< 0.001
PP * Intervention vs. Control	0.003				0.570

Note: Control1 = Registered Control (Image and Text). Control2 = Exploratory Control (Image Only).

Table D2

Logistic Regression with Exploratory Control Condition on Signing a Pledge to Wear a Mask.

	b	SE	Wald	df	p	OR	95% CI
							Lower	Upper
Constant	0.15	0.04	16.14	1	< 0.001	1.17
Intervention vs. Control2			8.03	8	0.431
Control1	0.19	0.16	1.37	1	0.241	1.21	0.88	1.66
Harm (Self)	0.12	0.16	0.55	1	0.460	1.13	0.82	1.54
Harm (Community)	0.26	0.16	2.59	1	0.108	1.30	0.94	1.78
Patriotic Duty	−0.03	0.16	0.04	1	0.852	0.97	0.71	1.33
Economy	0.02	0.16	0.02	1	0.893	1.02	0.75	1.40
Purity	0.13	0.16	0.68	1	0.408	1.14	0.83	1.57
Threat	−0.04	0.16	0.07	1	0.785	0.96	0.70	1.31
Scientific Evidence	−0.07	0.16	0.19	1	0.666	0.93	0.68	1.28
Political Party (PP)	0.77	0.06	182.24	1	< 0.001	0.47	0.41	0.52
PP * Intervention vs. Control			6.95	8	0.542

Note: Control1 = Registered Control (Image and Text). Control2 = Exploratory Control (Image Only).

Table D3

Logistic Regression with Exploratory Control Condition on Sharing a Pledge to Wear a Mask.

	b	SE	Wald	df	p	OR	95% CI
							Lower	Upper
Constant	−1.63	0.05	1002.28	1	< 0.001	0.20
Intervention vs. Control2			7.35	8	0.499
Control1	0.42	0.22	3.45	1	0.063	1.51	0.98	2.35
Harm (Self)	0.37	0.22	2.72	1	0.099	1.45	0.93	2.24
Harm (Community)	0.45	0.22	4.04	1	0.044	1.56	1.01	2.41
Patriotic Duty	0.14	0.23	0.37	1	0.543	1.15	0.73	1.82
Economy	0.19	0.23	0.64	1	0.422	1.20	0.77	1.89
Purity	0.21	0.23	0.87	1	0.352	1.24	0.79	1.94
Threat	0.41	0.22	3.38	1	0.066	1.51	0.97	2.33
Scientific Evidence	0.31	0.23	1.92	1	0.166	1.37	0.88	2.13
Political Party (PP)	0.49	0.07	52.06	1	< 0.001	0.61	0.54	0.70
PP * Intervention vs. Control			7.23	8	0.512

Note: Control1 = Registered Control (Image and Text). Control2 = Exploratory Control (Image Only).

Discussion

Given the threat of COVID-19 and its emerging variants, finding methods to encourage people to engage in effective health behaviors—such as wearing a mask or face covering—is critical to managing the risk of infection (CDC, 2020). However, in the United States, there has been a political divide in the likelihood to follow recommended health behaviors used to fight COVID-19, with Republicans being less concerned about the virus (Pew Research Center, 2020) and less likely to follow recommended health behaviors (Wronski, 2020) compared to Democrats. Therefore, finding solutions to persuade individuals across the political spectrum to adhere to the recommended health behaviors is critical for dealing with the spread of COVID-19. Based on moral foundations theory (Graham, Haidt and Nosek, 2009, Graham et al., 2011) and past message framing traditions (Day et al., 2014; Feinberg and Willer, 2013, Feinberg and Willer, 2015; Voelkel & Feinberg, 2018), we developed seven different messages and tested them in a tournament to determine which ones could persuade a representative sample of Republicans and Democrats in the U.S. to wear masks from October 2020 to January 2021. These messages promoted masks with subsequent rationales either focusing on protection from harm (self), protection from harm (community), patriotic duty, purity, reviving the economy, threat, and scientific evidence.

Across multiple dependent variables, we found no evidence that the interventions persuaded Republicans or Democrats in the U.S. to wear masks compared to a baseline message condition. None of the messages significantly increased attitudes and intentions to wear a mask or behaviors to sign and share a pledge to wear a mask. Indeed, in contrast, we only found a marginal effect that the threat message was associated with less pledging behavior compared to the baseline message, which was in the opposite direction of our hypotheses. The strongest effect on mask attitudes and behaviors was by far political party affiliation, such that Republicans reported having more negative attitudes and intentions toward wearing masks and were less likely to sign and share a pledge as compared to Democrats. Quite clearly, the interventions were not able to override the deep-seated partisanship that has existed in the U.S. during the COVID-19 pandemic.

There are several possible reasons for the null effects of our interventions. First, the interventions may have been too weak. Though many framing studies involve reading a brief message or nudge for only a few minutes (Thaler & Sunstein, 2021), it is possible that this approach is limited in contexts where attitudes are strongly engrained. Indeed, other interventions also found no evidence for changing COVID-19 related health attitudes or behaviors when using brief messages targeting selfish versus altruistic motivation (Sasaki, Kurokawa, & Ohtake, 2021), prevention versus promotion motivation (Utych, 2021), social norms (Bilancini, Boncinelli, Capraro, Celadin, & Paolo, 2020), or appeals to civic responsibility or limited health care system capacity (Pink et al., 2020). The messages might have been more effective if they were delivered in multiple instances over time or came from an elite, trusted source (see Pink, Chu, Druckman, Rand, & Willer, 2021). Second, and relatedly, the timing of the study may have impacted our results. We conducted the intervention tournament across four months (from Oct 14, 2020, to Jan 14, 2021), after attitudes had become polarized, and this may have been further amplified during the U.S. Presidential election and continued through the attack on the Capitol on January 6, 2021. Finally, while our interventions were grounded in theory on moral foundations and wise interventions, it is possible that there are other interventions that would have been more effective. There is some evidence, for example, that using messages that focus on the community-wide benefits of face coverings (Capraro & Barcelo, 2020), promote reasoning over emotional responses (Capraro & Barcelo, 2021), discuss COVID-19 as a public (vs. personal) threat (Jordan et al., 2020; c.f., Miyajima & Murakami, 2021), evoke empathy through storytelling about how the virus affected the elderly (Pfattheicher, Nockur, Böhm, Sassenrath, & Petersen, 2020), or use written reflection exercises (Hume, John, Sanders, & Stockdale, 2020) show some efficacy in promoting COVID-19 health behaviors. Future research using meta-analysis will be useful in discerning which interventions had the most powerful impact during COVID-19 and whether they were effective for both Republicans and Democrats.

The most robust finding of the tournament was that Republicans were much more resistant to mask wearing across our dependent variables than Democrats, raising the question of why this was the case. Though it is likely multiply determined, Republicans' resistance to virus mitigating interventions may be due to the lack of perceived threat of COVID-19. Collective threats motivate norm abiding behavior when there is a reliable fear signal (Gelfand, 2021). When this signal is thwarted or manipulated, strict abidance of new norms is less likely to evolve. Consistent with this, we found Republicans did, in fact, perceive lower threat of COVID-19 and this partially mediated the partisan differences in mask wearing attitudes, intentions, and behaviors (see Appendix E, Fig. 9 ). We note that this meditation analysis was exploratory and not pre-registered, and we caution drawing conclusions about the casual direction from this model as the variables were measured cross-sectionally. We did test an alternative model by switching the order of the mediator (i.e., perceived threat) and the outcomes (see Appendix F). We found that the proportion of mediation accounted for by the indirect effect was significantly smaller for the alternative model (Table F9).

Fig. 9

Perceived Threat Mediates the Effect of Political Party on Mask Wearing Attitudes and Behaviors. Note: We used 1000 bootstrap samples for estimating SEs.

Table F9

Comparing Outcome and Mediator Models.

	% Mediation
	Outcome Model			Mediator Model			Difference			Z	p
Attitudes	80.80 [73.50, 87.30]			59.20 [54.60, 63.90]			21.60 [13.28, 29.92]			5.09	< 0.001
Intentions	82.40 [76.40, 89.90]			53.80 [49.60, 58.80]			28.60 [20.43, 36.77]			6.86	< 0.001
Signed Pledge	80.50 [70.60, 88.20]			13.60 [10.90, 16.00]			66.90 [57.74, 76.06]			14.31	< 0.001
Shared Pledge	51.70 [37.50, 62.50]			2.80 [1.70, 3.40]			48.90 [36.37, 61.43]			7.65	< 0.001

Note: The outcome model represents the variables as outcomes (e.g., Political Party → Perceived Threat → Attitudes) whereas the mediator model represents the variables as mediators (e.g., Political Party → Attitudes → Perceived Threat).

At first glance, the lower perceived COVID-19 threat levels among Republicans may seem puzzling given the large body of evidence in psychology and neuroscience that has shown that conservatives are more sensitive to threats such as pathogens and negative stimuli when compared to liberals (Aarøe, Petersen, & Arceneaux, 2020; Hibbing et al., 2014; Jost, 2017; Mendez, 2017; cf., Bakker, Schumacher, Gothreau, & Arceneaux, 2020; Brandt, Wetherell, & Reyna, 2014; Elad-Strenger, Halperin, & Saguy, 2019). Yet these studies were not able to account for some important contextual moderators, including the effect of leadership on perceived threat during an actual pandemic. Indeed, during COVID-19, the pathogen threat signal was repeatedly diminished by U.S. President Donald Trump who failed to adequately convey that the threat was dire. As Trump said on March 2020 “Just stay calm. It will go away.” Even by January 2021, well after more than 300,000 Americans had died, Trump complained of the Center for Disease Control and Preventions supposed exaggerations. As the death toll climbed to staggering levels in the United States, those Americans who supported Trump felt even more empowered to ignore the warning signs. In this respect, they were more tightly following their leader than any fear instincts about pathogens. It's worth noting that the condition where we did attempt to activate COVID-19 threat (e.g., Condition 7: “It is important to wear a mask or face covering because COVID-19 has killed over 211,000 Americans and continues to spread rapidly”) showed some evidence of backfiring among both Republicans and Democrats. Threat messages may evoke people's fear of the virus, but inadvertently promote maladaptive behaviors (e.g., denial or avoidance; Rippetoe & Rogers, 1987; Witte, 1992).

In addition, previous studies also generally have looked at how conservatives and liberals react to threats in isolation of other threats. It is possible that when examining multiple threats simultaneously that conservatives are more sensitive to threats of perceived infringements of freedom and/or other types of threat (e.g., conflict. Brandt et al., 2020; Kahn, Björklund, & Hirschberger, 2020) and/or that conservatives express their fear in different ways (e.g., xenophobia) that support their party's values (Brandt et al., 2014).

The results also have some important practical implications. Given that none of the seven interventions worked, we need to continue to develop more ecologically valid interventions to help promote mask wearing and social distancing as the pandemic continues to evolve. As research shows, behavior change with light-touch interventions is particularly challenging when individual attitudes are already unsupportive to begin with (Dewies, Schop-Etman, Rohde, & Denktaş, 2021), which was the case for Republicans in our sample.

As scientists, we overestimated the extent to which participants would overcome the partisan divide and assumed that at least some conditions had the potential “to win”. Indeed, in a separate forecasting study in which over 1000 participants predicted the effects of the interventions presented in this study, we found that academics, behavioral science practitioners, and laypeople alike overpredicted the results of our tournament (Dimant, Pieper, Clemente, Dreber, & Gelfand, 2021). In particular, when examining the accuracy of predictions across political parties, forecasters predicted larger effects for Democrats than we found, yet were more accurate in their predictions for Republicans. Existing research suggests that political polarization runs deep in the U.S., that Democrats and Republicans hold incorrect beliefs about each other, and that it is highly challenging to reduce polarization via behavioral interventions (Dimant, 2021). During events that require an urgent response, such as a global pandemic, holding accurate beliefs about the outcomes of behavioral interventions is crucial to reduce the costs and maximize the success of identifying effective behavioral interventions.

Data availability and code statement

All data and analysis codes are available through the Open Science Framework (https://osf.io/xr9ka/). The study was pre-registered prior to data collection and approved as a pre-registered report at JESP.

Declaration of Competing Interest

The authors declare no competing interests.

Conditions	Degree to which the text and imagereflected the message			Respectfulness			Rank order(% of bottom three)
	Combined	Dem.	Rep.	Combined	Dem.	Rep.	Dem.	Rep.
Punishing God	t (600) = −6.84***	t (302) = −7.91***	t (297) = −2.28*	t (600) = −5.97***	t (302) = −7.78***	t (297) = −1.56	78%	67%
Loving God	t (600) = 2.37	t (302) = −0.46	t (297) = 3.76	t (600) = 4.55	t (302) = −0.09	t (297) = 6.40	50%	44%
Tight Social Order	t (600) =−1.01	t (302) =−3.97***	t (297) = 2.34	t (600) =−1.98	t (302) =−2.25*	t (297) = 4.68	55%	44%
Freedom	t (600) =−3.26***	t (302) =−4.79***	t (297) =−0.21	t (600) = 11.99	t (302) = 7.44	t (297) = 9.49	24%	28%
Protection from Harm (Self)	t (600) = 5.47	t (302) = 1.25	t (297) = 6.48	t (600) = 31.67	t (302) = 23.47	t (297) = 21.46	5%	13%
Protection from Harm (Community)	t (600) = 17.42	t (302) = 11.93	t (297) = 12.70	t (600) = 41.00	t (302) = 34.63	t (297) = 25.02	3%	8%
Patriotic Duty	t (600) = 12.31	t (302) = 6.96	t (297) = 10.54	t (600) = 13.04	t (302) = 5.90	t (297) = 12.74	20%	16%
Purity	t (600) = 19.11	t (302) = 12.43	t (297) = 14.62	t (600) = 13.82	t (302) = 6.39	t (297) = 13.74	6%	8%
Reviving the Economy	t (600) = 4.64	t (302) =0.85	t (297) = 5.62	t (600) = 14.34	t (302) = 5.74	t (297) = 15.97	9%	13%
Threat	t (600) = 13.36	t (302) = 8.19	t (297) = 10.76	t (600) = 10.98	t (302) = 6.99	t (297) = 8.52	6%	7%
Scientific Evidence	t (600) = 4.74	t (302) = −1.08	t (297) = 4.74	t (600) = 23.99	t (302) = 17.49	t (297) = 16.46	11%	16%
Control	t (600) = 5.98	t (302) = 2.28	t (297) = 6.13	t (600) = 39.51	t (302) = 29.85	t (297) = 26.31	33%	34%

Note: Bolded ratings are those that are significantly lower than the scale mid-point (3) and that were ranked in the bottom three in terms of effectiveness. Conditions that were omitted in the main study are in bold. Dem. = Democrat. Rep. = Republican. * p < .05, *** p < .001.

Outcome	Test Statistic (Name)	Test Statistic (Value)	df	p
Attitudes and Intentions (Pooled)	Pillai's Trace (F)	0.02 (0.76)	112 (9718)	0.9726
Attitudes	F	1.07	56 (4859)	0.9966
Intentions	F	1.35	56 (4859)	0.8225
Signed Pledge	χ2	69.97	7	0.0992
Shared Pledge	χ2	69.96	7	0.0994

Table E1

Political Party → Perceived Threat → Attitudes.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.82		0.04		0.75		0.89		22.38		< 0.001		80.8
Direct		c		0.20		0.02		0.14		0.25		6.98		< 0.001		19.2
Total		c + a × b		1.02		0.04		0.94		1.10		25.16		< 0.001		100.0

Table E2

Political Party → Perceived Threat → Attitudes.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Perceived Threat		a		1.19		0.04		1.11		1.27		29.34		< 0.001
Perceived Threat		→		Attitude		b		0.69		0.01		0.66		0.72		47.96		< 0.001
Political Party		→		Attitude		c		0.20		0.03		0.14		0.25		6.98		< 0.001

Table E3

Political Party → Perceived Threat → Intentions.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.74		0.03		0.68		0.80		23.16		< 0.001		82.4
Direct		c		0.16		0.02		0.11		0.20		6.40		< 0.001		17.6
Total		c + a × b		0.89		0.04		0.82		0.97		24.67		< 0.001		100.0

Table E4

Political Party → Perceived Threat → Intentions.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Perceived Threat		a		1.19		0.04		1.12		1.27		31.15		< 0.001
Perceived Threat		→		Intentions		b		0.62		0.01		0.59		0.65		41.43		< 0.001
Political Party		→		Intentions		c		0.16		0.02		0.11		0.20		6.40		< 0.001

Table E5

Political Party → Perceived Threat → Signed Pledge.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.14		0.01		0.12		0.15		19.80		< 0.001		80.5
Direct		c		0.03		0.01		0.01		0.06		2.33		0.020		19.5
Total		c + a × b		0.17		0.01		0.14		0.20		12.54		< 0.001		100.0

Table E6

Political Party → Perceived Threat → Signed Pledge.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Perceived Threat		a		1.19		0.04		1.12		1.27		31.53		< 0.001
Perceived Threat		→		Signed Pledge		b		0.12		0.004		0.11		0.12		26.52		< 0.001
Political Party		→		Signed Pledge		c		0.03		0.01		0.01		0.06		2.33		0.020

Table E7

Political Party → Perceived Threat → Shared Pledge.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.04		0.004		0.03		0.05		9.63		< 0.001		51.7
Direct		c		0.04		0.01		0.01		0.06		3.14		0.002		48.3
Total		c + a × b		0.08		0.01		0.06		0.11		7.10		< 0.001		100.0

Table E8

Political Party → Perceived Threat → Shared Pledge.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Perceived Threat		a		1.19		0.04		1.12		1.27		31.58		< 0.001
Perceived Threat		→		Shared Pledge		b		0.04		0.00		0.03		0.04		10.25		< 0.001
Political Party		→		Shared Pledge		c		0.04		0.01		0.01		0.06		3.14		0.002

Table F1

Political Party → Attitudes → Perceived Threat.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.71		0.03		0.65		0.76		23.9		< 0.001		59.2
Direct		c		0.49		0.03		0.43		0.55		15.5		< 0.001		40.8
Total		c + a × b		1.19		0.04		1.12		1.27		31.5		< 0.001		100.0

Table F2

Political Party → Attitudes → Perceived Threat.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Attitude		a		1.02		0.04		0.94		1.09		26.2		< 0.001
Attitude		→		Perceived Threat		b		0.69		0.01		0.67		0.72		59.1		< 0.001
Political Party		→		Perceived Threat		c		0.49		0.03		0.43		0.55		15.5		< 0.001

Table F3

Political Party → Intentions → Perceived Threat.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.64		0.03		0.59		0.70		23.5		< 0.001		53.8
Direct		c		0.55		0.03		0.49		0.61		16.7		< 0.001		46.2
Total		c + a × b		1.19		0.04		1.11		1.27		30.0		< 0.001		100.0

Table F4

Political Party → Intentions → Perceived Threat.

Path Estimates
												95% Confidence Interval
						Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Intentions		a		0.89		0.04		0.83		0.97		24.4		< 0.001
Intentions		→		Perceived Threat		b		0.72		0.01		0.69		0.74		57.9		< 0.001
Political Party		→		Perceived Threat		c		0.55		0.03		0.49		0.61		16.7		< 0.001

Table F5

Political Party → Signed Pledge → Perceived Threat.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.16		0.02		0.13		0.19		10.5		< 0.001		13.6
Direct		c		1.03		0.04		0.96		1.10		27.3		< 0.001		86.4
Total		c + a × b		1.19		0.04		1.11		1.27		30.1		< 0.001		100.0

Table F6

Political Party → Signed Pledge → Perceived Threat.

Path Estimates
											95% Confidence Interval
					Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Signed Pledge		a		0.17		0.01		0.14		0.20		12.5		< 0.001
Signed Pledge		→		Perceived Threat		b		0.95		0.04		0.87		1.04		22.5		< 0.001
Political Party		→		Perceived Threat		c		1.03		0.04		0.96		1.10		27.3		< 0.001

Table F7

Political Party → Shared Pledge → Perceived Threat.

Mediation Estimates
								95% Confidence Interval
Effect		Label		Estimate		SE		Lower		Upper		Z		p		% Mediation
Indirect		a × b		0.03		0.01		0.02		0.04		6.19		< 0.001		2.84
Direct		c		1.16		0.04		1.08		1.23		31.19		< 0.001		97.16
Total		c + a × b		1.19		0.04		1.12		1.26		31.93		< 0.001		100.00

Table F8

Political Party → Shared Pledge → Perceived Threat.

Path Estimates
											95% Confidence Interval
					Label		Estimate		SE		Lower		Upper		Z		p
Political Party		→		Shared Pledge		a		0.08		0.01		0.06		0.11		6.95		< 0.001
Shared Pledge		→		Perceived Threat		b		0.41		0.04		0.32		0.49		9.84		< 0.001
Political Party		→		Perceived Threat		c		1.16		0.04		1.08		1.23		31.19		< 0.001