Moral expressions, sources, and frames: Examining COVID-19 vaccination posts by facebook public pages.

Taking advantage of 3 million English-language posts by Facebook public pages, this study answers the following questions: How did the amount of COVID-19 vaccine-related messages evolve? How did the moral expressions in the messages differ among sources? How did both the sources and the five moral foundations in posts influence the number of likes to posts, after controlling for the public page's features (e.g., age, followers)? Our research findings suggest that moral expression is prevalent in the COVID-19 vaccination posts, surpassing nonmoral content. Media sources, despite the high volume of posts, on average elicited fewer likes than all other sources. Although care and fairness were the two most used moral foundations, they were negatively related to likes. In contrast, the least used two moral values of authority and sanctity were positively related to likes. We conclude with a discussion of theoretical contributions and a recommendation of possible interventions.

Introduction

As the world continues to fight against COVID-19, people's willingness to receive vaccination has sparked controversy across the world. Individuals are divided by their moral preferences such as religious beliefs and political ideologies (Funk & Gramlich, 2021). People who choose not to get vaccinated against COVID-19 express a variety of concerns regarding the vaccines, including serious health risks, distrust in government authorities, and information overload (Puri et al.. 2020). The latest global vaccine data showed that vaccine hesitancy remains high in certain countries despite sufficient supply and incentives offered by state or local governments (World Health Organization, 2021). Even within people who have been vaccinated, there is still a considerable amount of confusion related to the effectiveness of the vaccines and health impact.

Vaccination has been viewed as a moral issue of right or wrong through the moral lens (Smith & Graham, 2019). For example, receiving COVID-19 vaccine has been viewed as the right thing to do as a high vaccination rate in society can offer protection toward ourselves and also the most vulnerable (Rosenfeld & Tomiyama, 2022). However, the literature has documented how anti-vacciners use their religious beliefs to justify their decision not to receive vaccination (Hussain et al., 2018). Moral values move beyond personal preferences and could be used to explain human behaviors related to vaccination. When a message about vaccination appears, individuals apply the moral lens to determine how they respond. If the moral values attached to a message resonate with individuals, they are more likely to show approval of the message (Wang & Liu, 2021). If the moral values do not align, it indicates a violation of individuals’ beliefs and could result in highly emotional and angered responses to dismiss the message (Wang & Lewis, 2021). Existing research about COVID-19 vaccination messages tend to focus on data-driven analyses such as topic modelling or sentiment analysis (e.g., Zhang et al., 2022), the moral lens is firmly driven by theoretical concerns that have been empirically verified over the years (Graham et al., 2013). Taking this theory-driven lens, moral analyses would allow us to unpack what moral values are more prevalent in the COVID-19 vaccination context and likely to trigger reactions from individuals. Implications could be drawn to suggest how to draft persuasive messages to conduct vaccine intervention campaigns.

Our paper starts from a review of the moral foundation theory and its relevance to the topic of COVID-19 vaccine discourses. By using moral expression, moral sources, and moral framing as our conceptual lenses, we provide a framework in which communication studies can be connected to moral value studies. The method section explains both the big data we used and the computational analysis we ran on the data. After presenting both descriptive and inferential findings, we drew conclusions and made suggestions on understanding the limits and the structure of social media ecosystems, utilizing moral values in social media discourses, and addressing COVID-19 vaccine related controversies.

Literature review

Communication studies can contribute to moral value studies by examining how moral values are expressed, what are the sources of such moral expressions, and how such expressions morally frame the issues. The concept of moral expression refers to the usage of moral values in expressing views and opinions. The concept of moral sources stipulates that information sources play a motivating or empowering role in helping people see issues through moral lenses. Communication is one major way to motivate. The concept of moral framing takes a step further to examine how different moral frames influence the reactions of users. Our paper aims to examine each concept through a set of empirical analyses of social media data.

Moral expression and COVID-19 vaccination

Haidt and Joseph (2004) developed moral foundations theory (MFT), which is the “first draft” of human's morality. MFT identifies several distinct and pervasive psychological systems that exist among individuals and societies, which vary in their salience. These moral values influence how human beings make judgements about what is right or wrong, resulting in divisions of political ideologies and views regarding other social issues (Graham et al., 2013; Shim et al., 2021; Wilhelm et al., 2020). MFT proposes five foundations: care/harm, fairness/cheating, in-group/loyalty, authority/respect, and purity/sanctity.

The first foundation, care/harm, captures intuitions of sympathy, compassion, and nurturance in response to caring for the vulnerable (e.g., children and the elderly). This particular foundation makes individuals sensitive to signs of suffering and cruelty. Violation of care results in anger and outrage which motivates individuals to punish the offenders (Mooijman et al., 2018).

The second foundation, fairness/cheating, captures the notion of rights and justice. It makes individuals sensitive to signs of exploitation and motivates them to shun or punish cheaters. This foundation emphasizes the need to preserve what is right morally and to follow rules.

The third foundation of loyalty/betrayal focuses on forming and maintaining coalitions based on shared identities, which make individuals want to trust and reward in-group members and ostracize those who betray the group. A collective identity motivates individuals to take certain actions through norms of obligation, reciprocity, and shared grievances (Polletta & Jasper, 2001).

The fourth foundation of authority/subversion captures the notion of traditional social hierarchies and maintaining social order, which makes individuals sensitive to signs of rank or status. It emphasizes respect toward leadership (politicians, government, or opinion leaders) and the need to obtain legitimacy. When the violation of authority occurs, transgression can be evaluated morally to offer criticism (Jasper, 2017).

The last foundation, sanctity/degradation, captures the notion of spiritual concerns and moral disgust if such spiritual values are violated. This moral value has been primarily tied to religious beliefs, emphasizing the sacredness of a particular issue or a figure (Strickler & Danigelis, 2002, June). Sanctity thus helps to justify social causes that individuals may support (Haidt & Joseph, 2004).

MFT can serve to evaluate moral expression, i.e., how people talk about situations using moral ways in everyday life. The connection between moral foundations and the discussion of vaccination is logical because vaccination has never been a purely medical issue (Buckee et al., 2021) and discussed in diverse ways, driven by an array of non-medical concerns (Larson, 2018). Moral concerns are particularly relevant given how much the pandemic has profound impacts on human life in all aspects (Kwon & Park, 2022). The care/harm moral foundation is related to vaccine hesitancy because some people are concerned about the harm vaccines could bring while other people are worried about the risks brought by non-vaccinated individuals. For example, misinformation claims that COVID-19 vaccine will cause 97% of recipients to become infertile (Loomba et al., 2021). The fairness/cheating moral foundation is relevant to firstly, vaccine distribution, especially among vulnerable groups and marginal regions; and secondly, the influence behind mandatory vaccination decisions, such as the perceived unfair voice from pharmaceutical companies. Anti-vaccination sources often claim that mandatory vaccination violates parental civil liberties and shows distrust in authorities who mandate compulsory vaccination, including scientists and governments. This distrust implies a connection to the authority/subversion moral foundation (Kata, 2010) and was found to negatively relate to vaccine confidence (Sturgis et al., 2021).

The loyalty/betrayal moral foundation is present when vaccine debates become politically driven and divided along party affiliations. In the context of the COVID-19 pandemic, vastly different coping strategies adopted by opposing sides in geopolitics (e.g., U.S. vs China) may lead to vaccine discussions involving nationalist appeals. Another claim often used by anti-vaccination sources is that vaccines include “contaminants” (Kata, 2010) and anything that is unnatural or impure should be avoided, linking to the sanctity/degradation moral foundation. For instance, misinformation claims that COVID-19 vaccine will alter DNA in humans (Loomba et al., 2021), making the human species impure. When examining the considerable counter-movement to question the safety of common vaccinations taken by children, Amin et al. (2017) found that vaccine-hesitant parents are more likely to hold purity and liberty-subversion moral foundations.

The Internet, starting from its early days (Zhang, 2005), has provided virtual spaces in which people can express their views on a variety of issues. Because of the enormous user base, the expressions found on social media are increasingly listened to as genuine voices and on-the-ground sentiments (e.g., Troisi et al., 2022). Various actors including media, governments, organizations, and companies increasingly use social media to reach and persuade their audiences. Vaccine-related content on social media became trending topics widely discussed by a large range of actors. Johnson et al. (2020) examined the linkage network among Facebook pages that discussed vaccination and found that anti-vaccination clusters were highly entangled with undecided clusters, whereas pro-vaccination clusters were more peripheral. This large-scale social network analysis does not provide much insights on the discussion content though. Among other prior research that examines the content of social media discussion, Faasse et al. (2016) analyzed comments to a prominent Facebook post about childhood vaccination and found that anti-vaccination comments contained greater analytical thinking and lower anxiety words than pro-vaccination comments. When comparing Facebook pages from high-HPV vaccine coverage counties to low-HPV counties in the U.S, Zhang et al. (2021) found that public pages from high-HPV coverage counties devoted more posts to raise awareness, increase knowledge, and target women to take actions. However, the existing studies have yet to examine the moral foundations of vaccine discourse and take full advantage of the scale of social media data. We thus ask a first research question regarding moral expression on the issue of Covid-19 vaccines, focusing on public pages on Facebook.

What moral values are prevalent in Facebook public pages' messages about COVID-19 vaccines?

Sources and moral expression on social media

Social media platforms such as Facebook are increasingly popular sources of health information while at the same time the content is not necessarily scientific (Kalimeri et al., 2019, May; Puri et al., 2020). When studying hesitancy towards other vaccines, scholars have examined not only information sources such as social media channels but also how social media users expressed their views about vaccines on these channels (e.g., Faasse et al., 2016; Johnson et al., 2020). Without verification, misinformation could spread on social media like wildfire and impact risk perception (Piltch-Loeb et al., 2021). Existing literature has demonstrated how Facebook users may be discouraged from getting vaccinated due to the influence of poor-quality information and emotional stories (Buller et al., 2019). Moreover, the easy access of social media enables a range of sources to become vocal on issues traditionally viewed as suitable for experts to discuss (e.g., scientific and medical issues). Vaccine used to be a health technology that is defined by health experts and authorities. However, as policies such as compulsory vaccination directly impact individuals, various parties take to social media channels to voice their views and attempt to convince others (e.g., Puri, et al., 2020; Smith & Graham, 2019). These parties can be viewed as information sources that express their views in moral ways and have impacts on the users who are exposed to the sources’ messages.

Source effects in communication studies have been long recognized. Persuasion studies posit source characteristics as important factors that influence the persuasiveness of messages. The line of source credibility studies (e.g., Wathen & Burkell, 2002) found trustworthiness, competence, dynamism and objectivity to be four important source factors that influence attitude changes of the message receivers. In the domain of vaccine studies, although media along with family/friends and physicians/medical professionals were reported as common information sources regarding vaccines (Gargano et al., 2015), building people's trust in these social actors has been increasingly challenging (Pagliaro et al., 2021; Yaqub et al., 2014). Distrust in sources such as government and scientists/medical professionals was found to be linked to vaccine hesitancy (Allington et al., 2021).

One approach to understand the lack of trust in information sources such as authorities is to take the moral perspective and examine how different sources make moral expressions. This investigation is important because in an era of information overload, disinformation and cultural wars, users often rely on their judgement of the sources to decide whether they accept the messages sent by the sources. When judging sources, users mobilize their moral intuitions and emotional feelings - for example, those who dislike the authority value consequently evaluate the government sources as untrustworthy. Moreover, for unfamiliar sources, users gauge the source's alignment to their own moral stances through reading the sources' moral expressions. Traditionally, there are a range of social actors who are deemed as sources that often express moral values, including religious entities, families, and civic organizations (Cheng, 2004). Although media organizations ideally should function as objective channels to convey sources' messages, communication studies once and again show that media can employ moral framing strategies in their reporting, presenting vastly different moral ways of understanding the same issue (Hopp et al., 2020). Similar to media organizations, other sources such as governments or scientists may express certain moral values and trust declines if people perceive them to be deviant from their moral beliefs.

Social media allow various social actors to have their own channels to communicate with their audiences. Other than media organizations, medical professionals and governments, a large array of social actors express their voices on health issues on the Internet. For instance, Li and colleagues (2021) found that non-governmental organizations (NGOs) in the United States were actively contributing to the discussion on Covid-19, not only shaping the issue discussion patterns but also evolving in the tie formation. In addition to the abundance of information sources, the interactive nature of social media means that users actively choose which sources they follow. On Facebook, users make the initial decisions of following a public page and can change their decisions to unfollow any time. The multitude and interactivity of information sources on social media suggest that the sources' moral expressions may differ and the source's influence on user reactions may vary. We thus ask:

How do different Facebook public pages express moral values in the messages about COVID-19 vaccines?

Social media reactions such as likes, shares, and comments are useful indicators of user responses to social media content. Although the reaction buttons are often put side-by-side on social media interfaces, these indicators mean different things and involve different psychological and social mechanisms. For instance, liking a post means paying at least a minimum amount of attention because a user needs to notice a post before liking it. As Facebook never really had the dislike button, clicking likes sometimes is interpreted as expressing agreement. No matter whether likes mean attention-paying or agreeing, we could examine the relationship between moral values and likes in a linear way. Comments require significantly more efforts than likes and comments have more complex relationships with posts’ content. For instance, people often leave unfavourable comments and do not click likes to indicate their disagreement with the content. Shares require slightly more efforts than likes and shares often have an audience in mind. In short, both comments and shares have distinctive mechanisms that would be more complicated if we link them to moral values. For instance, simply linking moral content to number of comments can be misleading because the substance of comments could be mostly against the moral values expressed in the posts. Simply linking moral content to number of shares is inadequate because the moral atmosphere of the audience groups needs to be considered to make meaningful interpretations. We thus focus on likes as our dependent variable and asks this question:

How do different Facebook public pages influence user like?

Moral framing and its influence on social media reactions

The moral expressions, as found in the social media communication domain, are expected to reveal the ways people understand vaccination issues. Audiences who see these expressions will cognitively and emotionally respond to the content's moral foundations. Moral framing refers to the employment of moral values, normative beliefs, and religious or cultural tenets to justify the truthfulness or the legitimacy of a message (Jasper, 2008). Social media messages with moral foundations become value-based and can appeal to individuals' core morality which thus influences people's reactions (Hopp et al., 2021).

Moral framing functions as a mobilization strategy where people assign meanings to a message and interpret relevant events and conditions to gather support (Snow & Benford, 1988). Wang and Liu (2021) studied the issue of police brutality during the 2019–2020 Hong Kong anti-extradition protest and found that tweets that used moral expressions have differential effects on how viral a message would become. They found that moral framing that uses care/harm, fairness/cheating, and authority/subversion were prominent and could influence the popularity and approval of tweets, as these moral values were more likely to have emotional appeals. They also pointed out that the effects of moral framing depend on the recipients of the messages, and are bounded by group identity. Mutlu et al. (2020) found that Twitter users tend to share tweets involving the virtue dimension more than those involving the vice dimension, in the context of Syrian White Helmets-related tweets. On social media, moral foundations have been found in vaccine-related expression. Broniatowski et al. (2020) studied 204 anti-vaccination Facebook pages from 2009 to 2019 and found that moral framing remained as one of the five most popular strategies used by Facebook users to mobilize supporters.

Social media messages can trigger people's various reactions. Communication scholars have primarily focused on peer reaction indicators related to viral reach which captures the volume of message sharing (Weismueller et al., 2022) and forwarding/retweeting by social media users(Alhabash & McAlister, 2015). Other metrics have been examined recently to capture affective evaluation of audiences, such as their emotional responses (Utz, Otto, & Pawlowski, 2020). Among various reactions, like was found to convey agreement to likeminded content as opposed to other reaction indicators such as recommend or respect (Stroud et al., 2017). Furthermore, scholars have conceptualized likes on social media as an indicator of visibility (Kim, 2018). This study focuses on the frequency of likes on Facebook as it captures both an explicit response of social media users and also an attitude evaluation of messages (Alhabash & McAlister, 2015).

Given the mobilization mechanism discussed above, we would expect that the moral framing of vaccine discussion on social media affects people's reaction. On one hand, when a moral frame of vaccines is consistent with an individual's existing attitude or belief, people are more likely to respond to the message positively through sharing and liking. On the other hand, studies have shown that anti-vaccine social media users feel morally outraged about the practice of vaccination (Smith & Graham, 2019; Orr et al., 2020). The use of moral frames thus could also trigger people's emotional reactions to the message (Wang & Lewis, 2021), some of which may provoke liking and others may inhibit liking. To unpack how moral frames influence people's reactions, we propose the below hypothesis:

Moral values in Facebook public pages' messages about COVID-19 vaccines influence user like.

Method

Our study was pre-registered on OSF registries, to follow the open science best practice (Dienlin et al., 2021). Our data come from Facebook pages archived by CrowdTangle. According to Facebook,1 “pages are places on Facebook where artists, public figures, businesses, brands, organizations and non-profits can connect with their fans or customers.” In contrast, “groups are a place to communicate about shared interests with certain people.” Facebook pages are often seen as broadcasting channels that send messages out to their followers while Facebook groups are deemed as communities that emphasize common interests and interactions among group members.2 One study that examines public pages vs. groups related to vaccines concludes that “public pages may work more efficiently to have targeted vaccine campaigns for organizations or institutions than setting up public group discussions.” (Zhang et al., 2021, p. 10) When using the CrowdTangle API to search through its database, we included all post types in English Language, regardless whether the post was branded content or not. We only included verified accounts because we are interested in content by legitimate users.

Search criteria

We set the search timeframe from 11 March 2020 to 10 August 2021. The period covered the entire process of launching and implementing COVID-19 vaccines. On March 11 in 2020, WHO declared COVID-19 a pandemic. On May 21 in 2020, the Trump administration and AstraZeneca announced a collaboration. As the first public governmental action about COVID-19 vaccines, it started to trigger more discussion about vaccines. Several vaccines (e.g. Pfizer BioNTech; Modern; Johnson & Johnson) were investigated, produced with expansion, and used globally. On August 10 in 2021, 30% of people have been vaccinated at least one dose of COVID-19 vaccines.3 From 11 March 2020 to 10 August 2021 most people in the world were impacted by COVID-19, and topics about vaccination were extensively discussed online. Based on news articles on COVID-19 and important public documents regarding the pandemic, we assembled a list of search keywords (see Supplementary Material Table S1). For better covering the targeted posts, this list includes general search keywords that refer to COVID-19 vaccination (e.g. COVID Vaccine, pandemic Vax, coronavirus) and specific search keywords that refer to specific relevant brands or technologies (e.g. Pfizer, Johnson vaccine, mRNA).

Keyword validation

In order to validate our keywords, we used stratified sampling based on the proportions of posts in different months to sample the data. Our sample size was set to 500 posts. Two researchers who specify in social media research coded the sample. Two simple codes were used: “1” represents vaccine-related posts; “0” represents irrelevant posts. We first coded 50 posts and conducted an inter-coder reliability test. Krippendorff's alpha was 0.64 and Cohen Kappa coefficient was 0.66. After further discussions, we coded another 50 posts and only one coding result was different. Eventually the validity test showed that 486 posts (97.2%) were vaccine-related with only 14 irrelevant posts. The result shows that our keyword list has a high accuracy rate. Keywords in irrelevant posts included herd immunity, COVID drug, flu vaccine, etc.

eMFD

Our data was analyzed using a dictionary-based approach, specifically, the extended Moral Foundation Dictionary4 (eMFD, Hopp et al., 2021). A dictionary-based model is a straightforward computational way to analyze textual content, which would calculate a value for text according to an existing dictionary. The dictionary guided by Moral Foundation Theory was first built in 2012 (Graham & Haidt, 2012). To build this dictionary, experts discussed, reviewed literature and manually selected words that they thought best exemplified the upholding or violation of moral foundations. This dictionary was then included in the Linguistic Inquiry and Word Count dictionary (LIWC, Pennebaker et al., 2001), and broadly used. Following the same procedure, an extended version, MFD2.0,5 was constructed by Frimer et al. (2017).

eMFD differs significantly from the previous generations of moral foundation dictionaries. It employed crowd-sourced workers, instead of experts, to annotate the moral foundations of passages from news articles. These passages were later parsed out to words and each of the words was given a moral value based on its probability to appear in passages annotated with the particular moral foundation. Generating moral values based on passages gives eMFD an advantage in being context-sensitive, compared to previous MFDs that only focus on single words. In addition, eMFD relies on a large number of ordinary people and their intuitive moral judgment to generate moral foundations. This strategy is consistent with many studies (e.g., de Waal, 2013, p. 289; Diller & Boornazian, 2015) that suggest the intuitive nature of human moral judgment, and in contrast to previous MFDs' reliance on expert judgment (see Weber et al., 2018 for more discussions on expert vs. crowd-worker coding). Given Facebook pages' broadcasting tendency and the popularity of Covid vaccine topics, we believe eMFD serves as an appropriate tool for our analysis. Following Hopp et al.’s (2021) suggestion, we used the single probability approach in identifying moral framing which allows the use of moral scores in inferential analysis. Using eMFD, textual documents are first preprocessed by applying tokenization, stop-word removal and lowercased. Next, the model compares each word in the document against the constructed dictionary for word scoring. The average of all words' values in a specific moral foundation would be the passage's predictive probability for this moral foundation.

Independent variables

Our independent variables include both basic information about the posts and moral foundation values of the posts. CrowdTangle dataset provides post demographics such as post age and page likes and followers at posting. As these demographics may naturally influence the reactions the posts received (e.g., the older the posts are, the more likely the posts are reacted to), we included them as our first set of independent variables. Post age (in days) was transformed from the date the post was published (M = 222.16; S. D. = 107.81). Followers at posting refers to the page's number of followers when the post was published (M = 1384108.00; S. D. = 6048091.00). This measure can be seen as an indicator of the page's popularity and is expected to influence individual posts' reaction regardless of the content of the posts.

Facebook asks each public page to categorize itself. The categories have three levels6 : Level 1 includes business, community organizations, interest, media, non-business places, public figures and others. Level 2 further defines the page such as which kinds of business (e.g., food and beverage) or which roles of public figures (e.g., persons or government officials). Level 3 goes even further to specify the pages such as that a food and beverage page can specify it is an African restaurant or an ice cream shop. The original Facebook page categories are not always reasonable for the purpose of our analyses. For instance, “media/news companies” is a level-2 label under “business”, not under “media”. Another example is that the level 2 “public and government” category is under “business”, not under “community organizations” or “non-business places”. Considering the focus of our study on the pandemic, we have done a thorough re-coding to re-categorize the pages based on the level-3 labels. Eventually, we have 7 categories: (1) “media” including arts, books, movies, music, news, TV, podcast, theatre, and so on; (2) “government and political entities” including city halls, community centres, police stations, political parties, and so on; (3) “health” including hospitals, all types of doctors, clinics, pharmacies, and so on; (4) “NGOs” including non-profits, education entities, religious places, community organizations, and so on; (5) “business” including all business other than the first 4; (6) “public figures” including authors, artists, journalists, celebrities, bloggers, politicians, government officials, and so on; (7) “others” including all remaining categories such as locations, landmarks and events.

eMFD provides six major measures, including one overall moral-immoral ratio and the probabilities of five moral foundations. Moral-nonmoral ratio indicates the ratio of moral words to non-moral words in each post. The mean of this ratio is 1.17 (S. D. = 0.99), which indicates that on average, our posts contain 1.17 times more moral words than non-moral words thus a moral foundation analysis of the posts is justified. In addition to the general amount of moral content, the eMFD generates a probability score that denotes the average of each post belonging to one of the five moral foundations. On average, the posts in our dataset showed highest probability in care (5%), followed by a second-tier of moral foundations including fairness and loyalty (3%). Authority had a probability of 2%. The lowest value is sanctity (1%).7

Dependent variable

CrowdTangle data provide a range of reactions, including number of likes, shares, comments and the aggregation of the three (named total reactions). In addition, five emotional reactions include wow, haha, care, sad and angry. Among these, like is the oldest reaction button Facebook provides and was introduced on 9 February 2009.8 Other reaction buttons (i.e., the five emotional ones) were only made widely available starting from February 2016. The thumb-up button was originally designed to show users’ agreement to the post. Later as Facebook algorithm became known to users, the like button also started to be used as a way to pay attention. Compared to the reaction buttons, share on Facebook is an optional function the post authors can control. In other words, not all posts are shareable. Comments on Facebook allow users to input texts and emojis, which require a significant amount of effort. Again, based on prior research that tends to confirm the intuitive nature of human moral judgement, we found like to be the most suitable dependent measure because it is easy thus intuitive to use, and its meanings are relatively straightforward (either agreeing or attention-paying). Number of likes to the posts in our dataset follows a typical long-tail distribution. A small number of posts received a large number of likes. 75% of posts only received 55 and lower likes. The standard deviation (5739.33) exceeds the mean (216.46). We thus chose an analytical strategy that suits the data distribution.

Analytical strategy

A negative binomial regression (NBR) was conducted to examine the relative influences of post features and moral foundation values on user likes. NBR is a generalization of the Poisson regression, which is often suitable to analyze count data (Cameron & Trivedi, 2013). Number of likes, the dependent variable in this study, is essentially count data with integrals only. In addition to addressing count data, NBR is particularly useful to analyze count data that are over-dispersed. Over-dispersion means that the conditional variance exceeds the conditional mean, which is the case with the likes count in the dataset as shown above. In addition, we ran a likelihood ratio test to see whether modelling the dispersion parameter significantly improves the model or not. Firstly, we ran a Poisson model which holds the dispersion parameter constant. Secondly, we ran the NBR model which includes the dispersion parameter. Lastly, the Chi-square test (χ2 (1, N = 3,123,863) = 3,233,185,727, p < .001) indicates that it is better to model the dispersion model than fixing it. NBR is a more suitable choice. Post demographics and moral foundation values were entered into the NBR model as predictors and number of likes as the dependent variable. The R package called glm.nb was used to run NBR.

Results

Historical findings on amount of posts

Posts on COVID-19 vaccination changed over time in both volume and sources. Fig. 1 shows that there are roughly three phases of vaccine posts: The fermentation phase, the heated discussion phase, and the cooling down phase. The fermentation phase lasted for about eight months from March to October 2020, during which the volume of posts on COVID-19 vaccination kept at a steadily low level with no more than 60k new posts per month. We see a first jump in November 2020, the month in which vaccine development had significant breakthroughs. On November 18, Pfizer and BioNTech announced that their vaccines are 95% effective; on November 23, AstraZeneca claimed their vaccines to be 90% effective (see Fig. 2).

Fig. 1

Amount of posts over time (march 2020–August 2021).

Fig. 2

Amount of new posts by page categories over time (march 2020–August 2021).

The heated discussion phase started when COVID-19 vaccine posts reached the first peak in December 2020. As shown in Fig. 1, the increase in number of posts in December alone was more than 300k. In the last month of 2020, the U.S. Food and Drug Administration (FDA) approved Pfizer, Moderna and BioNTech; the U.K. government approved AstraZeneca and Oxford. By the end of 2020, about 2.8 million people in the U.S. and 3.3% of the U.K. population have received an initial vaccination. Since December 2020, the increase in vaccine posts has kept at the rate of about 300k per month. The cooling down phase only started to show signals from June 2021, when there was a 70k decrease in new posts. Note that the sharply smaller increase of posts in August 2021 was due to our sampling period ending on August 10. This time series data shows that COVID-19 vaccination posts are still prominent on Facebook and confirms that our analysis is still highly relevant.

Sources-wise, there is clearly a lack of correspondence between number of posts and number of likes. Media pages received most likes (38%) because they created most of the posts (71%). Public figure and NGOs pages, despite of only contributing 9% and 3% of posts, received disproportionally high number of likes (21% and 23%). Government and political entities pages also had bigger shares in likes (14%) than their shares in posts (11%). These contrasts point out the possibility that media pages are not as like-prompting as other pages. Over time, media pages showed large differences in number of posts – other than December 2020, media pages reached a 2nd highest number of posts in March 2021. Although public figures posted much fewer posts, their peaks corresponded to media pages'. Government and political entities' pages didn't follow the two peaks set by media pages. Instead, their posts peaked in May 2021. Posts from business, NGOs, and health pages were stably fewer over time, but their only slight peak fell in May 2021, too.

Descriptive findings of moral foundation values

The amount of moral expression in our data is relatively high, compared to previous studies on other social media content such as Twitter discussions (Matsuo et al., 2021; Mutlu et al., 2020; Van Vliet, 2021). This suggests that talking about COVID-19 vaccine in a moral way is a common practice on Facebook public pages. Fig. 3 shows that as the amount of posts increased, the amount of posts that contain moral content increased. One indicator is that when the total number of new posts increased from 0.12 to 0.33 million in December 2020, the amount of new moral posts also jumped.

Fig. 3

Number of new posts with moral foundation values over time (march 2020–August 2021).

To answer RQ1, our findings show that care has been consistently the moral foundation that has the highest number of posts per month (in the range of 15–18k during the heated debate phase), suggesting that when the posts mentioned any moral values, it was most likely to be the care value. Fairness has been consistently the second-highest moral foundation shown in the posts (in the range of 10k during the heated debate phase). This is understandable because the development and distribution of COVID-19 vaccine is related to fairness. Authority followed fairness in volume (in the range of 7-9k during the heated debate phase), suggesting that decisions made by authorities such as approving and mandating the vaccines and other measures are related to this value. Loyalty has been shown in the range of 6k posts per month during the heated debate phase. Sanctity has been the moral foundation least shown in the range of 3k during the heated debate phase.

The time pattern of all 5 values looks similar. There were two relatively obvious peaks: one in December 2020 and one around March–May 2021. December 2020 was when the first doses were given out. March to May 2021 was the period during which the percentage of people in the world who are partly vaccinated increased from 2% to 11%. By May 2021, both U.K. and U.S. crossed the 50% mark with the majority of their populations partly vaccinated. This was also when controversies over COVID-19 vaccine such as side effects on children and pregnant women as well as racial disparities in COVID-19 mortality surfaced to public attention. These controversies may have led to more posts that focus on how vaccine-related issues violate moral values.

Source differences in moral expression

To answer RQ2, we ran a one-way ANOVA using page category as an independent variable and the 5 moral foundation values as dependent variables. The results indicate that sources demonstrate significant differences (F (6, 3,123,825) = 3482.0 for care; 152.2 for fairness; 273.9 for authority; 371.7 for loyalty; 88.6 for sanctity, p < .001) in all moral foundations. Posthoc tests found that a vast majority of the differences were statistically significant. Fig. 4 shows the two leading sources in expressing each moral value. We summarized our findings as traditional (i.e., media, government and political entities, health) vs. non-traditional information sources (NGOs, business, public figures) in a health context. Media pages were more likely than other pages to repeat the dominant moral value (i.e., care) in the vaccine posts, probably driving the highest probability of the care value considering the largest amount of posts created by media pages. Government pages, without much surprise, focused on talking about authority and loyalty values. Health pages are particularly important in the context of COVID-19 and they stood out in their relatively high rank in sanctity.

Fig. 4

Page categories with top 2 average moral foundation values.

Compared to the traditional information sources on health issues, non-traditional information sources were found to have different emphases on their moral expressions. NGO pages include both international leading entities such as WHO and UNICEF and small community-based NGOs at the local level. These pages had relatively high values in fairness and authority. An interesting finding is that business pages ranked the highest in fairness, loyalty and sanctity, values that were less seen than the care value. Public figure pages were ranked among the top two in expressing the care value, showing a resemblance to media pages.

The influence of moral framing on number of likes

Number of likes per post changed over time, corresponding to the development of the COVID-19 situation in the country. Fig. 5 shows that although the increase in new posts peaked since December 2020, the average number of likes per post in each month peaked in May 2020, 7 months earlier than the COVID-19 vaccine being put into use. In May 2020, the Trump administration and AstraZeneca announced a collaboration to speed up the development of a COVID-19 vaccine called AZD1222, which was one of the first public announcements about a COVID-19 vaccine. This gap between peaks in number of posts vs. number of likes per post suggests that the fermentation phase was already shaping public perceptions because each post created back then elicited almost two times more reactions from Facebook users than the later posts. We speculate that the attention pool harnessed by Facebook public pages is relatively fixed - so the more posts, the lower average likes each post receives. This possibility is partially confirmed when in July 2021, the total increase in posts was lower than the heated months but the average like per post went higher.

Fig. 5

Average number of likes per post over time (march 2020–August 2021).

The NBR result is summarized in Table 1 . NBR models the log of the expected count as a function of the predictor variables. The coefficient in the table means that with 1 unit increase in the predictor, the expected log count of the dependent variable increases by a factor indicated by the coefficient. Incident rate ratios are used to exponentiate the model coefficients. Incident rate ratios can be understood as the ratios at which one unit change in the independent variable will lead to changes in the dependent variable. For instance, an incident rate of 0.99 means 1 unit increase in the independent variable will lead to 1% decrease in the dependent variable. In contrast, an incident rate of 1.01 means 1 unit increase in the independent variable will lead to 1% increase in the dependent variable.

Table 1

Negative binomial regression predicting the logged number of likes to posts.

	Coefficients	Standard Error	Incident Rate Ratio
(Intercept)	3.67∗∗∗	5.02e-03	39.28
Followers at Posting	2.75e-07∗∗∗	1.77e-10	1.00
PostAge(Days)	1.25e-03∗∗∗	9.99e-06	1.00
Page Category (in comparison to Media)
Business	0.69∗∗∗	6.19e-03	2.00
Government and Political Entities	1.11∗∗∗	3.52e-03	3.05
Health	0.77∗∗∗	6.91e-03	2.15
Non-Governmental Organizations	1.76∗∗∗	6.47e-03	5.79
Public Figures	1.65∗∗∗	3.79e-03	5.19
Others	0.87∗∗∗	1.75e-02	2.39
moral_nonmoral_ratio	−1.79e-02∗∗∗	1.08e-03	0.98
care	−0.10∗∗∗	3.10e-02	0.90
fairness	−0.10∗	4.12e-02	0.90
loyalty	0.07	5.44e-02	1.08
authority	0.19∗∗∗	4.55e-02	1.21
sanctity	0.20∗∗∗	5.92e-02	1.22

Note. E refers to exponent. E followed with a number is the common format of scientific notation in software. E.g., e−02 = 10ˆ-2. ∗p < .05 ∗∗p < .01 ∗∗∗p < .001.

The NBR findings show that page features have significantly positive influence on the number of likes one post receives. Number of page followers when the post was created had a positive impact on the number of likes. The longer the page was created, the more likes the page's post received. But the magnitude of both page features' impact was small. To answer RQ3, the set of analyses on page category used “media” as the comparison group and showed bigger impacts. The analyses found that posts from both NGOs and public figure pages received more likes than media pages. The incident rate ratios showed that NGOs and public figure pages' posts on average received about 5–6 times more likes. Compared to media, all other categories including business, health, government and political entities, and others were more likely to receive likes, although the differences were relatively small. The incident rate ratios suggest that these pages' posts on average received about 2–3 times more likes.

Moral-nonmoral ratio was shown to have a significantly negative impact on the log counts of likes. The incident rate ratio suggests that 1 unit increase in the ratio leads to 2% decrease in likes. This finding suggests that posts that contain too much moral content and too little nonmoral content discourage likes.

To test H1, our findings show that after controlling for the moral-nonmoral ratio and page features, 4 out of 5 moral foundations showed significant impacts on the log counts of likes, although the effects had varying directions and magnitudes. Both care and fairness were found to be the moral foundations that had a relatively higher probability to appear in the posts. But mentioning care or fairness showed a negative impact on likes (Incident rate ratio for both = 0.90). One unit increase in care or fairness leads to a 10% decrease in likes. In contrast, although authority and sanctity showed a relatively lower probability to appear in the posts, both showed a positive impact on likes (Incident rate ratio = 1.21–1.22). One unit increase in authority leads to a 21% increase in likes. One unit increase in sanctity leads to a 22% increase in likes. Loyalty was found to have no significant impacts on likes.

The findings are intriguing - how much one moral value can elicit user reactions was not based on how much the moral value was expressed in the posts. The impact of moral values was not equivalent to, even opposite to, the volume of content that contains the moral values. For instance, although the care value was mentioned a lot more than other moral values, posts containing the care value were less likely to get likes. Commonly seen values seemed to work against getting more likes, while relatively novel values brought more likes.

Discussions and conclusions

This study takes advantage of Facebook public page data to discover the historical evolution of COVID-19 vaccine-related posts, the source differences in moral expression, and the influence of moral frames on user likes. Our findings suggest that in order to understand and influence users, we have to consider the social media ecosystem, identify information sources that are morally suitable to the audiences, and morally frame the social media messages.

Consider the social media ecosystem

Our historical data (RQ1) show that although posts have increased dramatically since December 2020, the average number of likes has not changed as much. More posts do not necessarily draw more likes to the topic of COVID-19 vaccine. This implies that Facebook public pages as a social media ecosystem have limits. Within this walled garden, Facebook users can be seen as a fixed attention pool, which does not increase much as the volume of Facebook content increases. In addition, Facebook algorithm may have a role to play here. It is known that Facebook's algorithm only shows the top posts in users' news feed. A Washington Posts article9 suggests that Facebook currently prioritizes user interactions over professionally produced content, which could disadvantage certain pages such as media on Facebook. No matter what the reason is, this finding suggests that Facebook posts need to be highly selective to maintain attention.

Other than treating Facebook public pages as a limited system, our findings on different sources suggest that the ecosystem has its own structure. Media pages are often thought of as a mainstream source for information, pushing the majority of messages to users. However, Facebook public pages witnessed the emergence of other non-media information sources and their impacts cannot be underestimated. Although media pages produce more content, their posts on average are less liked. Two types of pages, NGOs and public figures, emerged to be particularly more influential than media pages. Governments, health, business, even other pages have higher chances of getting likes than media pages, although existing vaccine literatures (Allington et al., 2021; Yaqub et al., 2014) alert us that people's trust in government and health sources is declining.

Understand information sources

More interestingly, different sources (RQ2) presented COVID-19 vaccine-related content with emphases of different moral values. Classic communication studies often understand information sources in terms of their trustworthiness and competency on one hand, while biases and hostility on the other hand. Our findings suggest that information sources can be seen as moral sources, in terms of emphasizing different moral values in their messages that may or may not resonate with audiences' own moral attitudes. In social media ecosystems (e.g., Facebook and Twitter), audiences take the initiative to follow or unfollow a page/account. Through feedback mechanisms such as likes, information sources on social media can cater to their followers' specific moral preferences. Our study points to a new direction in source effects research, which is to define sources' moral stance and audiences’ perception of these information sources being moral sources.

Influence-wise (RQ3), both NGO and public figure pages’ much higher impact in getting likes than media pages suggests they may be becoming an alternative type of information sources that are influential. NGO pages received on average almost 6 times more likes than media pages, and were ranked second in expressing the fairness and authority values. Public figure pages, although getting on average 5 times more likes than media pages, were found to be leading in expressing the care value. This finding suggests that NGO and public figures pages are different in their potential to be moral sources. Although both appearing to generate more user likes than media pages, NGO pages may have achieved that through their moral stance of upholding fairness and authority while public figures could be reacted to for their parasocial relationship with their followers through showing care. Lastly, we should not overlook the morally vocal and nuanced business pages. Our finding that business pages are leading in three moral values is a surprising one. Although business pages were not as influential as NGO and public figures page in getting likes, their activeness in expressing the less seen moral values deserves more careful examination.

Use moral frames

Our moral foundation analysis (H1) shows that Facebook public pages talk about COVID-19 vaccine-related matters in a moral way, more so than a nonmoral way. It makes sense to use moral frames in addressing COVID-19 vaccine controversies. Expressing care and fairness is a common practice, but they may be too common to get likes. Our findings suggest that we should frame the posts using other moral values that are less seen. The most interesting one is sanctity, which focuses on spiritual pursuits and religious beliefs that emphasize the sacredness of the issue. Although sanctity was low across different sources, this moral value is particularly important for vaccine issues because this finding echoes with prior studies on linking religious beliefs to vaccine hesitancy (e.g., Marti et al., 2017). The view that vaccines include “contaminants” (Kata, 2010) often backs up vaccine resistance (Amin et al., 2017). It is worth noting that business, health and NGO pages were the 3 most active ones in talking about sanctity. Our speculation is that these pages tried to directly address sanctity concerns so they repeated these messages in their posts. For example, one top sanctity post stated: “Johnson & Johnson stresses that no fetal tissue is used in the vaccine.”

Another interesting finding regards the value of authority, which shows a strong positive effect on eliciting likes. Discussing the leadership and legitimacy surrounding COVID-19 vaccines may get people's likes. This finding is consistent with a prior study (Heine & Wolters, 2021), which shows that the authority value used in government communication has the largest positive effect on vaccination uptake. The 3 most active sources in mentioning authority value are others, governments, and NGOs. Considering that international NGOs played an authority role in the vaccine posts by defining scope of the crisis and the variants of the virus, their voice and influence are worth further examination. How does this type of global authority function as moral sources? Why can they elicit user likes? How can these sources work with national authorities and health authorities in the global health arena? These questions all need good answers.

Combining the three intervention approaches above, our study suggests that using Facebook public pages to address COVID-19 vaccine-related controversies may have its own limitations in the types of users they can reach and the amount of likes they can attract. The differential effects found from moral foundations suggest that there might be self-selection bias regarding user reactions and some users might be immune to moral framing in such messages. When posting on public pages, Facebook users should incorporate other strategies to combat the lack of effect or negative effects from moral framing. For example, our findings suggest that page categories could be leveraged to get more likes from the audience. Specifically, NGOs or public figure pages could facilitate message distribution and user reactions. Furthermore, our study demonstrates that when combining moral framing and other page related information, social media users should be mindful that the moralization of COVID-19 vaccine-related messages on Facebook is a complex process which can be influenced by user attributes, platform features, or political factors. We thus call for future research to investigate how moral framing may interact with other mechanisms to influence user reactions.

Limitations

As a big data and computational communication study, our research has its unique limitations. Firstly, our 3 million data were made available by CrowdTangle, a Facebook authorized data platform. Although CrowdTangle states10 that it provides almost all data (99.64%) from public pages with more than 25k likes or followers, there is no way for us to cross-check this statement. But we tend to believe that CrowdTangle offers the majority of available data, as their data platform becomes increasingly used by academic research (e.g., Giglietto, et al., 2020). Secondly, our historical data search was an ad-hoc rather than dynamic data retrieval, which means that the permanently deleted posts cannot be obtained. Considering Facebook's tightened measures regarding disinformation, this may lead to omissions of posts that are taken down either by Facebook or the pages. But as our study isn't really a study on disinformation, we are not very concerned about this possibility. Thirdly, statistical analyses of big data have their own challenges, such as the tendency for all effects to be significant. In order to account for that, we have focused on interpreting other statistics such as incident rates and doing so with comparisons among variables. Fourthly, we were only interested in uncovering the mechanism underlying moral framing and likes because we think other outcome variables may entail other mechanisms. Thus, our findings are limited to explaining one social media reaction, likes. Future research is needed to examine the relationship between moral content and other social media reactions such as shares and comments. Lastly, our content measures included mainly moral values measured by eMFD, a tool developed and validated by other researchers prior to our study. Although we are confident with the overall validity of the tool thanks to the tool developers' meticulous documentation, we cannot guarantee that the precision level of eMFD in COVID-19 vaccine-related content is the same as before. Moreover, the posts' content can be measured in other moral-irrelevant ways such as whether the posts contain dis- or misinformation, or whether the posts were pro- or anti-vaccine. Future studies should expand their content measures to such dimensions.

Conclusion

Our work is driven by the Moral Foundation Theory and highlights the fruitfulness of answering theory-driven research questions by utilizing big data that occur naturally in a social media environment. Our research demonstrates that descriptive data (RQ1 and 2) such as amount of posts and user reactions over time can reveal not only the changes that happen in the issue environment (e.g., how COVID-19 vaccination discourse evolved) but also the limits and structure of the social media ecosystem (i.e., Facebook). Future studies should take advantage of big data descritpives to understand the data platform's unique features and their interactions with the issue enviornment. The inferential analyses (RQ3 and H1) in this study contribute theoretically to the moral foundation research by highlighting the human communication of moral values. Moral values might be innate to humans but the expression of these values goes through a social constructivist process. This process demonstrates the more nuanced roles moral framing plays. Our inferential analyses also highlight the importance of sources in addressing controversial issues such as vaccine through moral expressions. Future studies need to revisit our traditional understanding of information sources and start to examine the moral impact of non-traditional sources such as business, public figures and NGOs. Moreover, this study demonstrates that the effects of moral framing and sources are contingent upon each other (e.g., sources choose how to use moral framing in messages). Thus, it offers implications on how to leverage Facebook public pages and specific moral values to drive messages viral when the general public possesses divergent views regarding a particular social issue.

Credit author statement

Weiyu Zhang: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Project administration, Funding acquisition; Rong Wang: Conceptualization, Methodology, Formal analysis, Writing – original draft, Writing – review & editing; Haodong Liu: Formal analysis, Writing – original draft, Writing – review & editing, Project administration

Funding

This work was supported by under Grant R-124-000-112-646.