When do persuasive messages on vaccine safety steer COVID-19 vaccine acceptance and recommendations? Behavioural insights from a randomised controlled experiment in Malaysia.

INTRODUCTION: Vaccine safety is a primary concern among vaccine-hesitant individuals. We examined how seven persuasive messages with different frames, all focusing on vaccine safety, influenced Malaysians to accept the COVID-19 vaccine, and recommend it to individuals with different health and age profiles; that is, healthy adults, the elderly, and people with pre-existing health conditions.
METHODS: A randomised controlled experiment was conducted from 29 April to 7 June 2021, which coincided with the early phases of the national vaccination programme when vaccine uptake data were largely unavailable. 5784 Malaysians were randomly allocated into 14 experimental arms and exposed to one or two messages that promoted COVID-19 vaccination. Interventional messages were applied alone or in combination and compared against a control message. Outcome measures were assessed as intent to both take the vaccine and recommend it to healthy adults, the elderly, and people with pre-existing health conditions, before and after message exposure. Changes in intent were modelled and we estimated the average marginal effects based on changes in the predicted probability of responding with a positive intent for each of the four outcomes.
RESULTS: We found that persuasive communication via several of the experimented messages improved recommendation intentions to people with pre-existing health conditions, with improvements ranging from 4 to 8 percentage points. In contrast, none of the messages neither significantly improved vaccination intentions, nor recommendations to healthy adults and the elderly. Instead, we found evidence suggestive of backfiring among certain outcomes with messages using negative attribute frames, risky choice frames, and priming descriptive norms.
CONCLUSION: Message frames that briefly communicate verbatim facts and stimulate rational thinking regarding vaccine safety may be ineffective at positively influencing vaccine-hesitant individuals. Messages intended to promote recommendations of novel health interventions to people with pre-existing health conditions should incorporate safety dimensions. TRIAL REGISTRATION NUMBER: NCT05244356. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Persuasive messages have been shown to influence COVID-19 vaccination intentions, but evidence from low and middle-income countries is limited.

Little is known regarding the effect of persuasive messages in influencing decisions to recommend the COVID-19 vaccine, especially while considering the health and age profile of the individual receiving the vaccination recommendation.

Persuasive messages that addressed vaccine safety concerns using facts and statistics to stimulate rational thinking did not positively influence Malaysian adults to take the COVID-19 vaccine or recommend it to healthy adults and the elderly.

Addressing vaccine safety concerns via persuasive messages is appealing towards individuals who are being nudged to recommend the COVID-19 vaccine to people with pre-existing health conditions.

In addition to providing verbatim facts and stimulating rational thinking, messages addressing vaccine safety concerns to improve vaccination intent should stimulate emotional reasoning and communicate the gist of the message convincingly.

The decision to take up or recommend a health intervention is based on perceived need that is derived from an internalised and externalised risk-benefit assessment for oneself and others, respectively, while accounting for individual health profiles.

Persuasive messages that are intended to promote uptake of a novel health intervention should incorporate safety dimensions as a form of assurance for others to recommend it to people with pre-existing health conditions, given that they may be perceived as more susceptible to harms from adverse effects due to the intervention.

Introduction

The COVID-19 pandemic has sparked global efforts to develop countermeasures against SARS-CoV-2. One such measure lies with the rapid research and development of effective COVID-19 vaccines1 which are critical to achieve impactful COVID-19 vaccination campaigns.2 Although credible vaccine information from official sources is abundantly available,3–6 vaccine-hesitant individuals risk compromising widespread vaccination7 as they delay or refuse to take a vaccine once it is made available.8

Vaccine safety remains one of the top concerns cited by vaccine-hesitant individuals in Malaysia and abroad.9–12 This is aggravated by misinformation regarding COVID-19 vaccine safety.13 Hence, a question that emerges is how best to effectively communicate vaccine safety information. A potential method stems from applying nudges, which alters the choice architecture or information context to encourage a particular behaviour.14 One such form involves using various frames of persuasive messages to encourage behaviour change.15 16

Multiple studies have experimented with persuasive messages to influence COVID-19 vaccination intentions. The use of goal-framed messages that seek to influence behaviour by highlighting benefits achieved or lost depending on vaccination acceptance has been widely explored.17–24 Dai et al used messages to reduce barriers to schedule a vaccination appointment, provide salient information about vaccine effectiveness, and issue reminders that leveraged on psychological ownership.25 Messages that detail information promoting personal or collective benefits through vaccination have also been explored.26 Some studies have used social norms to motivate individuals to take the COVID-19 vaccine with mixed successes.20 27–29

Amidst this broad coverage of studies, few have found messages that specifically address COVID-19 vaccine safety to be effective among unvaccinated individuals. Positive effects were observed when a risky choice framed message was tested among employees of a healthcare organisation through a personalised email message.29 However, results may not be generalisable to the public as healthcare organisation employees may have higher levels of health awareness. Barnes and Colagiuri also observed positive effects with messages applying attribute framing where vaccine side effect rates were framed positively or negatively.30 However, they investigated booster shot intentions among fully vaccinated individuals.

Based on current available evidence pertaining to vaccine safety, there are several knowledge gaps. The effects of attribute framing have yet to be explored among individuals unvaccinated against COVID-19. Attribute framing manipulates the descriptive valence of an object or event and has been shown to affect the cognitive and evaluation process of decision-making,31–33 which potentially influences health-related behaviour.32 34 35 Risky choice frames are another nudge technique that describes the outcome of potential choices involving differing levels of risk and can be applied to favourably highlight a frame being evidently advantageous when comparing competing frames.36 Although already proven effective at influencing vaccination intentions,29 framing generic health messages that juxtapose vaccine-related death rates or side effects against the COVID-19 disease has not been studied among the general public. Descriptive norm messages have been widely studied in the context of COVID-19 vaccination by communicating that the majority are getting vaccinated, so that individuals become psychologically convinced that vaccinating is a societal norm deemed as effective and hence adopt it.37 However, these messages have not been framed to imply vaccine safety as a motivation for vaccination among the majority. Additionally, using vaccinated health authorities to imply vaccine safety and recommend the vaccine has the potential of leveraging on authority bias.38 For instance, the use of descriptive norms to highlight medical consensus among medical professionals regarding vaccine safety helped reduce risk perceptions and improved attitudes towards the measles, mumps, and rubella (MMR) vaccine.39 However, this effect has not been thoroughly studied in an Asian context. Finally, given that individual decision and behaviour are intrinsically linked to context and culture,40–42 there are reasons to believe that vaccination nudges ought to be adapted to low and middle-income countries (LMIC) such as Malaysia. However, there exists a paucity of information for using such nudges in LMICs, with most published evidence originating from developed countries.16

Furthermore, previous studies have widely investigated nudges to influence personal interests to vaccinate one’s self or own child16 43 44 rather than a person’s decision to recommend vaccination. Although James et al did investigate the effects of persuasive messages in recommending a COVID-19 vaccine to a friend, they did not consider the health or age profile of the person being recommended.21 Having a finer gauge on which group of people have higher likelihoods to be recommended is important especially in Asian communities who pay special attention to advice sought from family and friends with significance when making a health-related decision.45

Therefore, we conducted an experiment in Malaysia using various message frames intended to narrow the current knowledge gaps. Our primary objectives were to investigate whether persuasive messages focusing on vaccine safety influenced the intention to take up the COVID-19 vaccine, and recommend it to healthy adults, the elderly (individuals who are aged 60 and above), and people with pre-existing health conditions. We hypothesise that, compared with a control message, exposure to a single message emphasising vaccine safety can significantly improve intentions among individuals who are initially hesitant to accept or recommend the vaccine. Apart from examining single messages, we investigate the effects of combining messages together to mimic a real-world environment where people are exposed to multiple messages. We hypothesise that, in contrast to the control group, exposure to two persuasive messages will create higher positive shifts in intent among hesitant individuals compared with a single message exposure. Testing this hypothesis allows us to determine if combining messages will improve effectiveness from a higher message dose effect,46 or reduce effectiveness due to message interactions causing a boomerang effect.

Methods

Study design

We conducted a prospective 14-arm randomised controlled experiment with a parallel design. The experiment was conducted using a web-based survey that was launched on a platform belonging to Dynata, an international market research company based in America. The company has an online survey panel composed of Malaysians who have signed up on the survey platform. Participants who complete a survey will receive reward points as per Dynata’s policy.

Study participants and setting

The experimental survey was conducted from 29 April to 7 June 2021 (more details about the COVID-19 situation in Malaysia during participant recruitment can be found in the online supplemental material). The survey was launched during the initial phases of the national COVID-19 vaccination programme which targeted the general adult population. Thus, data involving actual vaccination uptake were largely unavailable as the majority of the population were unvaccinated.47 Participants were recruited from Dynata’s online survey panel. Eligible participants were adult Malaysians aged 18 years and above who could understand either the English or Malay language and had not received any dose of the COVID-19 vaccine. The latter criterion allowed us to investigate the effectiveness of messages in an unvaccinated population, which is an important aspect prior to any novel vaccination roll-out.

Sample size requirement was calculated based on a logistic model to detect a small effect size of 0.1, with the baseline proportion of people who definitely will take the COVID-19 vaccine set at 0.67. This baseline value was chosen based on the reported proportion of Malaysians willing to accept the COVID-19 vaccine in a national survey that was conducted before this study was being planned.12 Sample size was calculated to be 318 respondents per arm, after setting power at 80% and significance level at 0.05. Taking into account a 20% dropout rate in the event of invalid responses, the estimated sample size was 400 participants per arm. Participants were recruited via stratified sampling based on age, sex, ethnicity, and household income to obtain an approximately population-representative sample (more details about the stratified sampling can be found in the online supplemental material).

All participants selected the language of their choice and were then shown a page that described background information about the study. They provided informed consent by clicking on a button indicating agreement to join the experiment.

Randomisation and masking

Enrolled participants were randomly allocated into a particular experimental arm by Dynata through an automated computer randomisation system. This experiment was double blinded whereby participants were unaware of what interventional message was given to them and investigators had no control over treatment assignment as this aspect was completely handled by the market research company.

Data collection and intervention

Sociodemographic variables that screened for inclusion criteria and enabled stratified sampling during experimental arm allocation were first collected from approached participants. General attitude towards vaccines was elicited from recruited participants as this factor has been shown to significantly influence vaccine uptake intent.44 48 Attitude was elicited by measuring the level of agreement (via a five-point Likert scale) with two statements regarding the efficacy of vaccines in protecting against serious diseases, and personal religious or cultural backing for vaccination. Participants were also asked in the remaining sociodemographic section whether they had refused to vaccinate their child in the past. These questions were adapted from locally conducted studies.48 49 Participants were categorised as having a potential negative attitude if they provided responses indicating disagreement, uncertainty, or refusal to any of those questions.

Participants were then asked a series of questions related to their baseline intentions to accept and recommend the COVID-19 vaccine before being randomly assigned to an experimental arm. Participants were exposed to either one or two messages from a selection of eight different types of messages and were instructed to read the message completely before clicking a button to proceed to the next message or section. Each message was calibrated to be on screen for at least 8 seconds before the button becomes active to ensure participants read the message without skipping. Table 1 describes the content of each message and the corresponding nudge technique that the content was incorporated with. The source of the information displayed is stated below the message’s content to provide information credibility.

Table 1

Content of each experimental treatment message used along with the corresponding nudge technique employed

No	Nudge technique	Content	Message code
1	Descriptive norm	Around 70% of Malaysians said that they will get the COVID-19 vaccine.Will you join them?Source:Ministry of Health, Malaysia; 31 December 202012	DN(70%)
2	Descriptive norm	The COVID-19 vaccine was tested with thousands of people, including the elderly, and people with existing health conditions.Now, millions of people worldwide have received it.When it’s your turn, you can be confident that it is safe and effective.Source:Kyriakidis et al. NPJ Vaccines. 2021 February 22;6(1):2892John Hopkins Coronavirus Resource Center, USA93	DN
3	Influence from a government official and health authority, and descriptive norm	Malaysia’s Health Director General, Dr Noor Hisham Abdullah, and 9 out of 10 healthcare workers in Malaysia have received the COVID-19 vaccine.They recommend that you get it too.Source:COVID-19 Immunisation Task Force, Malaysia; 23 February, 29 March 202194 95	HCW
4	Negative attribute framing	Only 4 out of 100 people who received the COVID-19 vaccine experienced side effects.Source:Ministry of Health, Malaysia; 2 April 202196	NF
5	Positive attribute framing	96 out of 100 people who received the COVID-19 vaccine did not experience any side effects.Source:Ministry of Health, Malaysia; 2 April 202196	PF
6	Risky choice framing (safety)	There are 0 deaths caused by the COVID-19 vaccines.On the other hand, over 1400 people have died due to COVID-19 infections.Source:Ministry of Health, Malaysia; 18 March 202197Crisis Preparedness and Response Centre (CPRC), Malaysia; 23 April 202198	RC(S)
7	Risky choice framing (side effects)	Only 4 in 1 million people who received the COVID-19 vaccine experienced blood clots.On the other hand, 200 000 in 1 million people infected with COVID-19 experienced blood clots.Source:Torjesen. The British Medical Journal. 2021;373:n100599Malas et al. EClinicalMedicine. 2020 December; 29:100639.100	RC(SE)
8	Control message	Get the COVID-19 vaccine.It’s safe and effective!	N/A

Each message is assigned a code to ease referencing.

The control message was devoid of any nudge or persuasive element and only displays a slogan that rallies the reader to get the COVID-19 vaccine because it is safe and effective. The other experimental messages began with an opening tagline highlighting the main concern that Malaysians have about the COVID-19 vaccine and serves as a precursor for the following message content which attempts to alleviate that concern. Each message concludes with a rally slogan that is identical with the control message. All messages were validated with at least five people and went through a series of iterations to ensure that the content was interpreted correctly (details about the message design and examples of actual messages can be found in the online supplemental material and figure S1, respectively). Messages were also translated to Malay and similarly validated.

Our experiment presents a total of 14 arms. Participants were exposed to one message in the first eight arms, and two messages in the remaining arms. The control arm was made a common comparator against all other experimental arms. DN(70%) exposure was held constant in arms that applied two experimental messages; that is, arms 9–14. This message was made a constant because it focuses on the Malaysian general public as the reference group, making it the most personally relevant message to our survey’s target participants who are from the Malaysian public. Participants who received two messages were exposed to one message at a time, with the sequence of appearance being random.

After message exposure, participants were asked again regarding their intentions to receive and recommend the COVID-19 vaccine. Participants who were hesitant about taking or recommending the vaccine after exposure were asked about the possible reasons for such responses. Lastly, the remaining sociodemographic variables such as education level and history of contracting COVID-19 were collected.

Outcomes

Intent to accept the COVID-19 vaccine was elicited by asking participants using a four-Likert scale with responses ranging from ‘Definitely no’, ‘Not sure, but probably no’, ‘Not sure, but probably yes’, and ‘Definitely yes’. This scale was used to eliminate subjective ambiguity and allows participants to express their intent in detail which is capably determined as it involves an internalised decision.50

Intent to recommend the COVID-19 vaccine was elicited by asking participants to rate their level of agreement with recommending the vaccine to three groups of family members, namely healthy adults, elderly, and members with any pre-existing health conditions. Family members were chosen as a target character because they are related to respondents, thus they can be interpreted as unbiased responses regarding intent to recommend the vaccine to each of the three studied groups. Participants rated their agreement via a five-Likert scale which provided options of ‘Strongly disagree’, ‘Disagree’, ‘Not sure’, ‘Agree’, and ‘Strongly agree’. This scale was chosen to provide a neutral answer in the form of a ‘Not sure’ option, because the decision to recommend may influence the outcome of another individual, which may be a difficult decision and thus warrant a neutral stance.

Our four study objectives were based on outcomes measured at baseline and post-intervention. Positive intent was defined as responding ‘Definitely yes’ and ‘Agree’ or ‘Strongly agree’ for accepting and recommending the vaccine, respectively. These responses indicated no hesitancy towards the action in question whereas the remaining options reflected uncertainty or refusal.

Statistical analyses

Summary statistics (frequency and percentages, mean and standard deviation) of recruited participants’ demographics, attitude towards vaccines, and intent to accept and recommend the COVID-19 vaccine in each experimental arm were reported. Balance tests were conducted to check if baseline characteristics were significantly different between each experimental arm.

Since the responses for all four outcome measures were ordinal in nature, we applied four separate generalised ordered logistic regressions to estimate how each experimental arm affected the propensity of selecting a particular level of intent. Each regression model was adjusted for general attitude towards vaccines and baseline intent that corresponds to the outcome measure analysed. Generated regression models were subsequently used to compute the average marginal effects of each interventional arm relative to the control arm based on changes in the predicted probability of responding with a positive intent for each of the four outcome measures. This provided an estimate behind the effectiveness and probability change magnitudes exerted by experimented messages. As post hoc analyses, we tested heterogeneous treatment effects of age, sex, and education level to investigate whether our intervention messages impacted certain groups of individuals differently.

All results were reported and presented graphically at the 5% significance level. However, to account for multiple hypothesis testing, we adjusted our p values by applying the sharpened false discovery rate method and reported these together with the marginal effects summary for all outcomes tested.51 All analyses were conducted using Stata V.16. This study was registered on ClinicalTrials.gov (ID number: NCT05244356). An author reflexivity statement was included to address the international partnership that stemmed from this study (the reflexivity statement can be found in the online supplemental file 2).

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

A total of 5784 participants were recruited into the experiment. Each arm was assigned between 410 and 416 participants. Table 2 provides a summary of the sociodemographic characteristics of recruited participants while figure 1 presents the experimental design flow chart. Sampled participants were approximately representative of the Malaysian national population with regard to sex, ethnicity, and household income.52 53 However, given that the experiment was conducted as an online survey, the proportion of participants from the youngest age group (ages 18–39) was 70% higher compared with the national proportion.52 Similarly, our sampled data set was skewed towards more educated participants, with the proportion of samples having education above secondary level surpassing the national proportion by more than threefold. However, all experimental arms were balanced and showed no significant differences with respect to key baseline characteristics. The average baseline proportion of participants with positive intent in each arm to take and recommend the COVID-19 vaccine to healthy adults, the elderly, and people with health conditions was 61.6%, 84.9%, 72.7%, and 51.4%, respectively. Almost all participants did not contract COVID-19 before. Summary statistics of survey participants stratified according to experimental arms can be found in the online supplemental table S1.

Table 2

Sociodemographic characteristics of all recruited participants (n=5784)

	n (%)
Age group (years)
18–39	3635 (62.9)
40–59	1916 (33.1)
60+	233 (4.0)
Sex
Male	2907 (50.3)
Female	2877 (49.7)
Ethnicity
Malay	3399 (58.8)
Chinese	1579 (27.3)
Indians	519 (9.0)
Natives to Malaysian Peninsula or Malaysian Borneo	287 (5.0)
Total household income
T20 group (RM10 960 and above)	815 (14.1)
M40 group (RM4850–RM10 959)	2443 (42.2)
B40 group (below RM4850)	2526 (43.7)
Education
No formal education	26 (0.4)
Primary education (up to standard 6)	69 (1.2)
Secondary education (up to form 5)	1097 (19.0)
Form 6/certificate/diploma/A-level/pre-university course	1727 (29.9)
Tertiary education (degree, master’s, PhD, DrPH)	2865 (49.5)
Contracted COVID-19 before
Yes	60 (1.0)
No	5467 (94.5)
Not sure	257 (4.4)
Intention to accept COVID-19 vaccination
Definitely not	106 (1.8)
Not sure, but probably not	390 (6.7)
Not sure, but probably yes	1724 (29.8)
Definitely yes	3564 (61.6)
Recommend COVID-19 vaccine to healthy adults
Strongly disagree	87 (1.5)
Disagree	185 (3.2)
Not sure	605 (10.5)
Agree	2833 (49.0)
Strongly agree	2074 (35.9)
Recommend COVID-19 vaccine to the elderly
Strongly disagree	168 (2.9)
Disagree	372 (6.4)
Not sure	1040 (18.0)
Agree	2303 (39.8)
Strongly agree	1901 (32.9)
Recommend COVID-19 vaccine to people with pre-existing health conditions
Strongly disagree	292 (5.1)
Disagree	766 (13.2)
Not sure	1752 (30.3)
Agree	1671 (28.9)
Strongly agree	1303 (22.5)

Figure 1

Experimental design flow chart presenting sample size, arm allocations, and item wordings for outcomes.

Figure 2 depicts forest plots that describe the average marginal effects of providing positive responses for each interventional arm relative to the control arm in all outcomes measured. A summary of marginal effects for all levels of responses can be found in the online supplemental tables S2 and S3. In terms of participant’s intent to accept the COVID-19 vaccine or recommending it to healthy adults and the elderly, none of the interventional arms were significantly effective at improving intent compared with the control message. Instead, intent to vaccinate significantly dropped among participants assigned to both the NF message, and its combined exposure with DN(70%). Recommendation intentions towards healthy adults significantly dropped in the DN(70%) and RC(S) arm. Intent to recommend the vaccine to healthy adults in the combination message arm containing DN and RC(S) was also lowered. However, all these findings were not robust after p value adjustments.

Figure 2

Average marginal effects for each interventional arm relative to the control arm based on changes in the predicted probability of responding with a positive intent for each primary outcome measure: (1) intention to vaccinate, (2) recommend to healthy adults (Healthy adults), (3) recommend to the elderly (Elderly), (4) recommend to people with pre-existing health conditions (Health condition). Forest plots present point estimates, 95% CIs, and the line of indifference.

Conversely, five interventional arms were significantly effective at improving recommendation intentions to people with pre-existing health conditions. Both the DN arm and PF arm showed highest significant improvements, with effect sizes measuring about 8 percentage points (95% CI 4.1 to 12.0) and 5.6 percentage points (95% CI 1.7 to 9.5), respectively. These findings were robust after p value adjustments. The remaining arms showing significant improvements were the combination messages containing DN (4.2 percentage points, 95% CI 0.2 to 8.1), HCW (4.7 percentage points, 95% CI 0.8 to 8.6), and RC(S) (4.6 percentage points, 95% CI 0.7 to 8.5) message. However, the significance level of these findings dropped to 10% after p value adjustments.

Being worried about the safety or side effects of the vaccine was the main reason for hesitancy, with 70%–80% participants who were hesitant in each outcome answering as such. A tabulation that reports the proportion of respondents citing reasons for hesitancy for each outcome can be found in the online supplemental figures S2–S5. We found no significant differences between all arms with respect to proportion of respondents citing this top reason (online supplemental table S4).

Figure 3 displays the forest plots with 95% CIs for heterogeneous treatment effects that indicate definite intentions of accepting the COVID-19 vaccine and agreeing to recommend it. A summary of treatment effect values can be found in the online supplemental tables S5–S7. There is evidence showing certain sociodemographic groups are more impacted by our experimented messages.

Figure 3

Sociodemographic determinants of average marginal effects with respect to age, sex, and education level, for each interventional arm relative to the control arm based on changes in the predicted probability of responding with a positive intent for each primary outcome measure: (1) intention to vaccinate, (2) recommend to healthy adults (Healthy adults), (3) recommend to the elderly (Elderly), (4) recommend to people with pre-existing health conditions (Health condition). Forest plots present point estimates, 95% CIs, and the line of indifference.

Subgroup analysis for participants aged below and above 30 years old was conducted. This grouping was selected to investigate if youths, who have much lesser risk for suffering severe consequences from contracting COVID-19 but have their future well-being affected by the pandemic,54 responded differently compared with the older age groups who have a higher risk for serious complications from a COVID-19 infection.55 Although we found that intent to vaccinate among older participants was significantly affected by the NF message both on its own and in combination with DN(70%), this finding was not robust after p value adjustments. Similarly, experimental arms which registered significant drops in recommendation intentions to healthy adults and the elderly became non-significant after p value adjustments. Both age groups responded positively to the DN message for recommendation intentions to people with health conditions, in which youths and older people saw an increase in intent by 6.6 percentage points (95% CI 0.1 to 13.0) and 8.7 percentage points (95% CI 3.7 to 13.8), respectively. Older people also showed an increase in intent to recommend by 5.3 percentage points (95% CI 0.3 to 10.3) when exposed to PF. Intent increased to 7.4 percentage points (95% CI 2.5 to 12.3) when DN(70%) was added. Additionally, older people were more likely to make a recommendation when DN(70%) was combined with HCW (6.5 percentage points, 95% CI 1.5 to 11.5). All messages that significantly influenced older people were robust after p value adjustments.

We find some gender heterogeneity, male respondents were more negatively impacted by the NF message. Vaccination intent further declined when DN(70%) was added. In contrast, females were more negatively influenced by the RC(S) message, causing a reduction in recommendation intentions to healthy adults. However, all these findings were not robust after p value adjustments. We documented a significant increase in recommendation intentions to people with health conditions when they were exposed to the DN message, irrespective of gender. Intent improved by 6.8 percentage points (95% CI 1.2 to 12.3) and 9.4 percentage points (95% CI 3.8 to 15.1) for males and females, respectively. Males were also more positively influenced by both PF and RC(SE) messages. Moreover, males tended to positively respond when DN(70%) was combined with RC(S) while females exhibited a similar response when DN(70%) was combined with HCW. After p value adjustments, only the positive influence of DN message among females remained significant at the 5% level while all other findings except for the influence of RC(SE) on males were significant at the 10% level.

Subgroup analysis was conducted between participants with and without tertiary education to observe any differences in behavioural response to the messages, given that Malaysians with a bachelor’s degree or higher were more likely to accept the COVID-19 vaccine.56 None of the messages significantly impacted vaccination intent among the two groups. However, several messages significantly reduced intent to recommend the vaccine to healthy adults among participants with tertiary education. The DN(70%) arm showed the highest drop in intent (−7.9 percentage points, 95% CI −12.6 to −3.2), followed by the NF arm (−5.7 percentage points, 95% CI −10.1 to −1.2), DN arm (−4.6 percentage points, 95% CI −8.8 to −0.3), and RC(S) arm (−4.5 percentage points, 95% CI −8.8 to −0.1). There were also significant reductions in intent between 5.5 and 6.6 percentage points among tertiary educated participants who were exposed to combination messages containing PF and RC(S). Most of these findings remained significant either at the 5% or 10% significance level after p value adjustments. However, participants without tertiary education revealed significantly lower recommendation intentions to the elderly when exposed to combination messages containing NF (−7.2 percentage points, 95% CI −12.4 to −2.1) and RC(SE) (−5.5 percentage points, 95% CI −10.6 to −0.5), but with only the former result remaining significant at the 10% significance level after p value adjustments. Apart from DN(70%), HCW, and combination messages containing PF and RC(SE), all arms showed significant improvements in intent among those with tertiary education to recommend the vaccine to people with health conditions, ranging from 6.0 to 11.6 percentage points. These findings remained robust after p value adjustments. Participants with lesser than tertiary-level education were also positively influenced by the DN arm, but this finding lost significance after p value adjustments.

Discussion

This study reports the results of one of the first experiments in the Southeast Asian region, and Malaysia specifically, that apply persuasive health messages to influence vaccine uptake and recommendation intentions. Hence, our results may serve as a reference benchmark for expected outcomes when using various types of message frames in a middle-income country. Two single experimental messages, that is, DN and PF, and two message combinations, that is, DN(70%)+HCW and DN(70%)+RC(S), supported the first and second hypotheses, respectively, for only one outcome, which is intent to recommend the COVID-19 vaccine to people with pre-existing health conditions.

None of our experimented messages improved vaccination intentions, with some showing signs of backfiring. Our results concur with other studies that similarly employ messages explaining about COVID-19 vaccine safety. Persuasive messages emphasising vaccine safety either through explaining the rigorous process of drug development and the rarity of side effects, leveraging the authority of a clinician to explain vaccine safety, or highlighting vaccine approval from a regulatory agency, failed to significantly improve vaccination intent.22 28 57 58 Although Diament et al found positive findings with a message explaining the vaccine’s approval process by a regulatory authority to infer vaccine safety, their results were weakly significant.59 There are several possible explanations to our findings. Our experimented messages provided brief verbatim representations that promoted vaccine safety. However, this stimulus did not translate to gist representations that was sufficiently convincing to influence hesitant individuals from a vaccine safety perspective, in accordance with the fuzzy-trace theory.60 Vaccine-hesitant individuals also display a higher reliance on experiential thinking systems,61 62 which poses a formidable challenge when attempting to alter decisions using rational arguments and statistics. Another explanation refers to the limited effect of brief textual messages at capturing attention and sounding convincing. Perhaps delivering messages through an engaging media might have yielded better results.63 For instance, a study reports a positive behavioural change when using a video clip to deliver vaccine safety information.64

Our persuasive messages did not improve recommendation intentions to healthy and older individuals. In contrast, we find significant and sizeable effects of persuasive messages in improving recommendation intentions to people with pre-existing health conditions across several experimental arms. Attribute appeal is a possible reason driving the differences in recommendation intentions between our studied outcomes. More than 80% of our participants agreed to recommend the vaccine to healthy adults at baseline. This observation suggests a general perception that healthy adults are fit enough to take the vaccine without any cause for safety concerns. Hence, addressing vaccine safety may not be a suitable dimension to persuade the hesitant minority who may have deeper qualms about other issues. Conversely, about half of our participants were hesitant at baseline to recommend the vaccine to people with health issues. Such low baseline proportions may be driven by perceptions that vaccines are potentially harmful to individuals with pre-existing health conditions who may have higher susceptibility of being harmed by vaccine adverse effects, given their poorer health state. This presumption is evidenced by the significant improvement in recommendation intentions after exposure to several of our messages promoting vaccine safety, a key attribute that appealed to influenced participants. Similarly, vaccine safety may be a concern among participants who were hesitant to recommend the vaccine to the elderly, given that they are frailer and more fragile to be recommended an intervention perceived as risky. This effect is not driven by ageism, as older people are regarded highly in Asian societies such as Malaysia.65 However, given that our current sample is skewed towards younger individuals, recommending a perceived risky intervention to an elder may seem disrespectful. Therefore, persuasive vaccine safety messages proved insufficient to nudge those hesitant to recommend amidst an additional cultural barrier.

Interestingly, our results suggest that vaccine recommendation intentions to people do not necessarily reflect on one’s own intention to vaccinate. Whilst the decision to vaccinate is based on a personal risk-benefit assessment from getting vaccinated, the decision to recommend the take up of a vaccine refers to an externalised risk-benefit assessment based on another person’s needs. This assessment might be reflective of some overconfidence, or perceived relative risk of disease severity from contracting COVID-19 together with perceived risk tolerance for vaccination, all of which is dependent on an individual’s health profile. For instance, individuals with pre-existing health conditions are presumably at higher risk of being severely ill from COVID-19, while also perceived to bear higher risks of suffering harm from vaccine adverse effects. However, once the latter concern is dispelled, the decision to recommend becomes clearer based on perceived benefits for these individuals.

Our descriptive norm messages are grounded on the perceived sense of safety generated from knowledge that a vast majority are taking or have taken the COVID-19 vaccine, making it a social norm deemed as the right choice. However, such social nudges proved ineffective in significantly raising self-vaccination intent compared with the control message, consistent with other COVID-19 vaccine studies involving norms.20 27 Despite being significant, the norms message performed the poorest in a study by Jensen et al.64 Helfinstein et al also found that descriptive norms had little effect on risk recommendation to others, which reflects our negative observations with respect to vaccine recommendation.66 In contrast, we observe the DN message increase recommendation intentions to people with pre-existing health conditions. Message targeting may have made the DN message relatable to the recommended target group, since it highlights that many people with health conditions have tested and taken the COVID-19 vaccine.67 However, the addition of DN(70%) weakened this effect. Additionally, although insignificant after p value adjustments, there are indications that DN(70%) on its own reduced recommendation intentions to healthy adults. These effects could be specifically due to the reference to "70% of Malaysians", as stated in the DN(70%) message. Such a proportion might be insufficient to be perceived as a convincing norm since mass media widely reports target inoculation rates of 80% by the government through the national immunisation programme.68

Both NF and PF messages induced opposite effects in two separate outcome measures. The PF message improved intentions to recommend the vaccine to people with health conditions. Although insignificant, there were signs that the NF message reduced intent to accept the COVID-19 vaccine and this was similarly observed when the DN(70%) message was added. Generally, studies have shown attribute frames to be more effective when framed positively rather than negatively.31 32 35 69 However, Barnes and Colagiuri found that both positive and negative attribute framed messages increased intentions to accept a booster dose among COVID-19 vaccinated participants if the offered vaccine was unfamiliar and familiar, respectively.30 Their findings differed from our results possibly because our participants have not been vaccinated but were already familiar with the type of COVID-19 vaccine offered that was being widely promoted on mass media, given that our survey coincided with the national immunisation programme.70 71 Inexperience with the vaccine may have heightened negative safety perceptions arising from negative attribute framing while negating positive effects observed with positive attribute framing with respect to vaccination intent. Familiarity with the vaccine’s safety profile may have also attenuated positive attribute framing effects.72 A study involving influenza vaccine similarly found that participants who were exposed to negative framed messages had higher expectations or perceived severity of side effects.73 Interestingly, inexperience did not cloud positive perceptions arising from the PF message to drive improved intentions to recommend the vaccine to people with health conditions. Instead, it appears that preconceived views that such a target population is more susceptible to harms from vaccine adverse effects given their poorer state of health may have been alleviated by this extra boost in safety perception.

Participants exposed to the HCW message did not show any significant changes in intent for all outcome measures examined. There are several possible reasons. The social norm cue used with reference to the majority of healthcare workers already vaccinated was probably ineffective due to participants being unable to identify with the reference population used.27 Furthermore, the message may not have provided the personal touch and physical interaction from a healthcare provider necessary to invoke changes in intent, a condition which is observed among studies reporting raised vaccination intents.43 74–76 This explanation is further supported by findings from Motta et al suggesting that vaccination intent did not differ from the control group when the message encouraging vaccine uptake came from a medical expert.17 Additionally, leveraging the Director General of Health’s influence, who is a government official, may portray him as accomplishing a bureaucratic task driven by political motives.77 The use of a celebrity who is viewed as politically neutral yet popular could prove more efficacious, as shown in a study which found celebrities inducing higher vaccine scepticism reductions compared with government officials or medical experts.77 Interestingly, when both HCW and DN(70%) were combined, recommendation intentions to people with health conditions were significantly raised. This observation is probably borne from positive interactions between a low descriptive norm and a high injunctive norm. Recommendations coming from a convincing proportion of healthcare workers confers the perception that getting vaccinated is a socially desirable action that is expected, which results in a high injunctive norm.37 Habib et al found that willingness to register as an organ donor increased when a low descriptive norm was combined with a high injunctive norm, as opposed to applying the norms individually.78 This interaction arises by stoking a sense of responsibility to act after the incongruent norms highlight salient inconsistencies existing within the group. Although unmeasured, we believe this sense of responsibility to recommend was invoked from this similar interaction. Our finding thus expands knowledge on normative influence by proving such interactions also exist for behaviour recommendation.

Although insignificant, there were signs that recommendation intentions to healthy adults were significantly negatively affected by RC(S). The use of death rates from COVID-19 could be perceived as an irrelevant risk to healthy adults, since most deaths are associated with elderly and people with pre-existing health conditions.79 A mismatch with the target group could have led to drops in intent. Moreover, the number of deaths featured on the message may not be convincing enough to require a need for healthy people to take the vaccine. However, this effect was slightly reduced when DN(70%) was added together, presumably because the higher dosage of pro-vaccination messages counteracted the negative effects of each message when applied individually.46 A similar dose–response interaction may be occurring when DN(70%) was combined with RC(S) to yield a significant increase in intent to recommend the vaccine to people with health conditions. Although RC(S) and RC(SE) addressed safety attributes which are relevant to elderly and people with health conditions, their effects did not differ from the control message when applied alone. A possible reason lies with the message bringing attention to possible health risks associated with the vaccine such as deaths or blood clots. Despite the probability favouring vaccine uptake, the mention of these health risks may have caused hesitant individuals to remain hesitant for fear of recommending something harmful.

Our analysis on heterogeneity treatment effects revealed varied impacts of different messages for each sociodemographic variable. There were indications that intent to vaccinate for both older participants and males was negatively influenced by a negative attribute frame. Studies show older people have higher risk perceptions towards health-related risks.80 This characteristic makes them more susceptible to negatively framed attribute messages as negative frames heighten risk perception. Studies have also shown that men tend to be more optimistic about perceived susceptibility and severity from COVID-19,81 82 rendering males as more likely to take a risk of contracting the virus as compared with taking a vaccine that is perceived unsafe due to the negative attribute framing effect. Our findings highlight the damaging effect such frames can cause among males who generally have higher vaccination intentions compared with females.83

Most of the messages which induced positive recommendation intentions to people with health conditions impacted the older age group, males, and those with a tertiary education. There are several postulations to this pattern of results. Studies show that self-esteem increases with age.84 85 This may confer older people with more confidence to recommend the vaccine if there is information that supports this action. Moreover, our youths may be more hesitant to make recommendations even when nudged as Malaysia practices a collectivist culture.65 People with pre-existing health conditions tend to be older, which makes it more challenging for youths to make recommendations due to social hierarchy barriers. Males having higher intentions to make recommendations are arguably driven by risk acceptance. Recommending a health intervention involves some risk taking since it advocates something that may expose another individual to a certain level of risk. Studies have shown that men exhibit a higher risk-taking behaviour compared to women.86 However, females were also found to similarly respond to the DN message, which underscores the potential of this social norm message in influencing people regardless of gender. On the other hand, behavioural differences to make recommendations based on education level are probably related to cognitive capabilities to synthesise information and perceived vaccine safety. People with tertiary education could have understood and synthesised the health messages better to infer that the vaccine was safe to be used by people with health issues. Being highly educated also increases confidence and imparts a higher sense of social responsibility to recommend.

Individuals with tertiary education were also more impacted by messages which reduced intent to recommend the vaccine to healthy adults. A deeper synthesis of messages by those who have higher education does not necessarily produce positive results and could backfire instead. These people may tend to have more complex interpretations amidst wider information obtained from various sources, resulting in certain messages inducing negative responses. Studies have shown that there is a strong association between education level and extent of COVID-19-related knowledge, both factual and perceived.87 88 Coupled with a lesser perceived severity of the virus by more educated individuals, these messages may have been interpreted with a risk-benefit analysis to suggest healthy individuals not requiring the vaccine.88

Limitations

Our experiment exhibits the following limitations. Study outcomes measured how messages affect intent and do not really indicate whether participants would actually receive or recommend the vaccine in reality. Although actual vaccination behaviour should be the prime outcome of interest, intent has been shown to be a strong predictor for behavioural actions over various contexts, even for actual vaccination uptake.89 However, significant intention–behaviour gaps for vaccination have been shown to exist,90 with a study even concluding that nudges are ineffective at significantly raising actual COVID-19 vaccination rates.22 Previous research has also shown differing results when applying behavioural nudges to promote COVID-19 vaccination under experimental conditions versus in the field.25 These findings underscore the need to field test behavioural interventions that are proven successful in survey experiments to confirm their true effectiveness under real-world conditions.

The extent of misinformation that participants were exposed to prior to our experiment was not measured. Misinformation has been proven to significantly affect vaccination intent.50 Actual vaccination rates declined depending on the theme and quantity of misinformation exposure.13 Therefore, misinformation exposure may be a strong predictor for resisting nudges from health messages. Future studies should find ways of incorporating this measure to further elucidate true effectiveness of messages under various levels of misinformation exposure.

The dynamic nature of the COVID-19 pandemic may have altered attitudes towards the COVID-19 vaccines since our experiment was initiated. This is especially so after the vaccines have been safely rolled out and shown to be effective as time progresses. Hence, the efficacy of these messages may have changed over the course of the pandemic.

Lastly, we did not specify any particular COVID-19 vaccine when asking participants to take up or recommend. During the experimental survey roll-out, vaccines from three different companies were widely mentioned in Malaysia, namely Pfizer-BioNTech, Oxford/AstraZeneca, and Sinovac.71 91 Each of these vaccines was developed using different technologies to yield differing effectiveness and safety profiles. The public may hold differing views about the vaccines based on the familiarity of the technology used to develop them. Hence, we were unsure whether responses obtained were based on a particular vaccine in mind or aggregated in nature.

Further work

Explanations regarding behavioural responses observed were inferred based on past research. More in-depth qualitative research based on theoretical frameworks should be conducted to gain a firmer understanding on how these messages affect individual perceptions that result in provided responses. Additionally, more research should be conducted to understand the science behind individuals recommending healthcare interventions to others, as this aspect of knowledge in the health behavioural field is scarce.

Conclusion

Despite safety being the main concern for COVID-19 vaccine hesitancy, crafting messages that focus solely on this attribute does not significantly improve vaccination intent or vaccine recommendation, except to people with pre-existing health conditions. Our findings highlight the challenges of addressing vaccine safety concerns via frames that present verbatim facts and stimulate rational thinking. Future messages addressing similar concerns should consider adding content that stimulates emotional reasoning and communicates the gist of the message convincingly.

We have documented several examples where combining messages weakened or strengthened intent, thus providing further proof about message interactions between different frames. A deeper understanding of such interactions is needed, especially when conducting health promotion campaigns that use a series of messages together to influence individual decision-making.

On a bigger picture, our study suggests two important findings. First, the decision to take up or recommend a health intervention, such as vaccination, is based on perceived need that is derived from both an internalised and externalised risk-benefit analysis, respectively, which may not necessarily be parallel with one another. Lastly, messages incorporating safety dimensions can update the belief of individuals to advocate an intervention that was previously deemed risky to a vulnerable population. This evidence suggests that persuasive messages should emphasise on safety when promoting recommendations of novel health interventions to individuals with pre-existing health conditions, especially if the intervention is perceived as potentially harmful to them.