Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial in Finland.

BACKGROUND: Vaccination is the most effective means of preventing the spread of infectious diseases. Despite the proven benefits of vaccination, vaccine hesitancy keeps many people from getting vaccinated. METHODS AND
FINDINGS: We conducted a large-scale cluster randomized controlled trial in Finland to test the effectiveness of centralized written reminders (distributed via mail) on influenza vaccination coverage. The study included the entire older adult population (aged 65 years and above) in 2 culturally and geographically distinct regions with historically low (31.8%, n = 7,398, mean age 75.5 years) and high (57.7%, n = 40,727, mean age 74.0 years) influenza vaccination coverage. The study population was randomized into 3 treatments: (i) no reminder (only in the region with low vaccination coverage); (ii) an individual-benefits reminder, informing recipients about the individual benefits of vaccination; and (iii) an individual- and social-benefits reminder, informing recipients about the additional social benefits of vaccination in the form of herd immunity. There was no control treatment group in the region with high vaccination coverage as general reminders had been sent in previous years. The primary endpoint was a record of influenza vaccination in the Finnish National Vaccination Register during a 5-month follow-up period (from October 18, 2018 to March 18, 2019). Vaccination coverage after the intervention in the region with historically low coverage was 41.8% in the individual-benefits treatment, 38.9% in the individual- and social-benefits treatment and 34.0% in the control treatment group. Vaccination coverage after the intervention in the region with historically high coverage was 59.0% in the individual-benefits treatment and 59.2% in the individual- and social-benefits treatment. The effect of receiving any type of reminder letter in comparison to control treatment group (no reminder) was 6.4 percentage points (95% CI: 3.6 to 9.1, p < 0.001). The effect of reminders was particularly large among individuals with no prior influenza vaccination (8.8 pp, 95% CI: 6.5 to 11.1, p < 0.001). There was a substantial positive effect (5.3 pp, 95% CI: 2.8 to 7.8, p < 0.001) among the most consistently unvaccinated individuals who had not received any type of vaccine during the 9 years prior to the study. There was no difference in influenza vaccination coverage between the individual-benefit reminder and the individual- and social-benefit reminder (region with low vaccination coverage: 2.9 pp, 95% CI: -0.4 to 6.1, p = 0.087, region with high vaccination coverage: 0.2 pp, 95% CI: -1.0 to 1.3, p = 0.724). Study limitations included potential contamination between the treatments due to information spillovers and the lack of control treatment group in the region with high vaccination coverage.
CONCLUSIONS: In this study, we found that sending reminders was an effective and scalable intervention strategy to increase vaccination coverage in an older adult population with low vaccination coverage. Communicating the social benefits of vaccinations, in addition to individual benefits, did not enhance vaccination coverage. The effectiveness of letter reminders about the benefits of vaccination to improve influenza vaccination coverage may depend on the prior vaccination history of the population. TRIAL REGISTRATION: AEA RCT registry AEARCTR-0003520 and ClinicalTrials.gov NCT03748160.

Introduction

Vaccinations have contributed enormously to global health. Large-scale vaccination programs continue to reduce morbidity and mortality due to numerous infectious diseases and comprise the backbone of health security strategies around the globe. However, increasing levels of vaccine hesitancy, defined as a delay in the acceptance or refusal of vaccines despite the availability of vaccination services, threatens the progress made in halting vaccine-preventable diseases [1-4]. In 2019, the World Health Organization (WHO) declared vaccine hesitancy to be one of the 10 biggest threats to global health. Understanding how to improve vaccine coverage and overcome different mechanisms underlying vaccine hesitancy is important, not only to improve current vaccination coverage but also to secure high coverage of new vaccines, such as the Coronavirus Disease 2019 (COVID-19) vaccines.

Several factors have been identified as relevant predictors of vaccine hesitancy. These factors include lack of trust in the safety and effectiveness of vaccinations (confidence); lack of appropriate disease-risk perception (complacency); perceived or actual structural and behavioral barriers, such as forgetting or difficulties in access (constraints); engagement in extensive information searching with potential risks of being exposed to misinformation (calculation); and lack of concern for vulnerable others (social responsibility) [4-6]. Despite accumulating evidence about the psychological antecedents of vaccination decisions and the development of validated measures to understand vaccine hesitancy, there is little population-based evidence about the effectiveness of scalable low-cost behavioral interventions that can be used to address specific factors associated with vaccine hesitancy.

Patient reminders and recall interventions via letters, email, or mobile phone messages are shown to be an effective method to increase vaccination coverage in outpatient, community-based, primary care settings [7,8]. Reminders address the psychological barriers that may prevent people from getting vaccinated (e.g., forgetting to make a vaccination appointment and lack of practical information on how to make an appointment). Reminders can also communicate information about other factors related to vaccine hesitancy [9], such as the individual benefits of being vaccinated, which address complacency by providing information about disease risk.

Less attention has been paid to whether enhancing reminders with content that highlights the social benefits of vaccines could further increase vaccination coverage. Vaccinations not only incur individual benefits through direct protective effects but also affect the community at large through indirect effects, which reduce the risk of spreading the disease to others and build up herd immunity [10]. Highlighting these positive behavioral externalities could, in theory, increase the motivation for prosocial vaccination to protect unvaccinated individuals and lead to higher vaccination coverage. In fact, existing empirical research shows that educating individuals about the social benefits of vaccination can increase their social responsibility and vaccination intentions [11-13]. Consequently, sending reminders that provide information about the individual and social benefits of vaccinations could be a way to increase vaccination coverage.

We conducted a large-scale cluster-randomized controlled trial in Finland to test the effectiveness of centralized, one-time, written reminders (distributed via regular mail) that highlighted (i) the individual benefits of vaccinations or (ii) both the individual and social benefits of vaccination in increasing influenza vaccination coverage. The focus was on influenza vaccinations among the older adult population, where the gap between the vaccination target and actual coverage is particularly large [14]. After the intervention, data from comprehensive nationwide health records on influenza vaccination coverage were used to determine the effectiveness of the information treatments.

Methods

Study design

We conducted the trial in 2 geographically and culturally distinct communities in Finland. The trial had 2 active treatment arms. The first treatment highlighted the individual benefits of vaccination. The second treatment highlighted the individual and social benefits of vaccination. In the western region (Fig 1), there was a control treatment group without any intervention. In the southern region, there was no control treatment group because the local health authority had in previous years sent influenza vaccination reminder letters to the entire population aged 65 years and above. Thus, our intervention did not leave anyone without information that they would have otherwise received in the absence of the intervention. The study was conducted in partnership with local health authorities in both regions.

Fig 1

Study regions and randomization scheme.

The map in Fig 1 was created for this article in R software using open source data (CC BY 4.0) from Statistics Finland. The base layer of the map used in Fig 1 is available at Statistics Finland’s map service (https://tilastokeskus-kartta.swgis.fi/?lang=en). The R code and shapefiles to reproduce the map in Fig 1 are available at https://osf.io/v453z/. Control = no reminder, Treatment I = individual-benefit reminder, Treatment I + S = individual- and social-benefit reminder.

The treatments varied the information content of individual reminders (the original letters are available in S1 Appendix). The individual-benefits reminder contained basic information about the severity of influenza symptoms, seasonal influenza vaccination, the availability of vaccinations (locations and dates to receive the vaccine), and instructions about how to book an appointment with the vaccine administration. The individual- and social-benefits reminder provided the same information as the individual-benefits reminder but also contained the following information about herd immunity:

“Your decision to vaccinate does not only protect you but others as well. Your vaccination may protect small children whose immune system is still developing. You will be able to protect your loved ones who are unable to get vaccinated. Your vaccination may prevent the spread of influenza viruses. Thus, the whole society benefits from your decision to vaccinate.”

Study population

The study took place in 2 regions with widely varying baseline vaccination coverage to test the effectiveness of different reminders in 2 different contexts and obtain information about the potential generalizability of findings across populations with differing baseline vaccination coverages and socioeconomic characteristics. The 2 regions represent populations with different socioeconomic characteristics and historical influenza vaccination coverage. The western region on the west coast of Finland is a rural region that contains 5 independent municipalities (Maalahti, Korsnäs, Närpiö, Kaskinen, and Kristiinankaupunki). The region has a single public provider of primary healthcare services that is co-owned by the municipalities. This region has low influenza vaccination coverage among people aged 65 years and older (31.8% during the influenza season of 2017 to 2018) compared to the national average (47.7% during the influenza season of 2017 to 2018). The southern region encompasses the city of Espoo, the second-largest city in Finland. The population in the southern region belongs to the inner urban core of the Helsinki metropolitan area and has one of the highest rates of influenza vaccination coverage among people aged 65 years and above (57.7% during the influenza season of 2017 to 2018).

The study population included everyone born in or before 1953 (aged 65 years and above) residing in the 2 regions on June 1, 2018. However, individuals living in housing units with more than 2 persons (e.g., nursing homes) were excluded from the sample and statistical analyses after the randomization because these units could include private nursing homes that provide seasonal influenza vaccinations to all residents as part of their care plan. Thus, the final analysis sample included only home-dwelling individuals living either in a single- or 2-person household (Fig 1). We excluded from the sample and statistical analysis also all individuals who received influenza vaccination before the beginning of the follow-up time. There were 10 individuals who received influence vaccination in the Western region between June 1, 2018 and October 17, 2018, and 210 individuals who received influenza vaccination in the Southern region between June 1, 2018 and October 17, 2018.

There were no other scientific, ethical, or economic reasons to exclude any individuals who met the specified inclusion criteria. Moreover, since the marginal costs of including additional individuals in these types of information interventions are very low, it was considered worthwhile to maximize the statistical power to detect even potentially small effect sizes. We focused on older adults aged 65 years and above, as they are entitled to free influenza vaccinations in Finland and identifiable from the population register by age. Older adults belong to a risk group with higher morbidity and mortality from influenza viruses than the prime working-age population [15,16].

Randomization and masking

We used the Finnish Population Register to identify the name and postal address of individuals who met the eligibility criteria (age and place of residence). Randomization took place at the household (cluster) level to avoid sending reminders with different contents to the same household members. Randomization at the household level was implemented using unique apartment IDs and a computer-generated randomization code written by the authors (S1 Appendix). We used simple randomization without any blocking factors. The sample in the western region (n = 7,398) was randomized into 3 treatment arms of equal size: (i) no reminder (control treatment group); (ii) individual-benefits reminder; and (iii) individual- and social-benefits reminder. The sample in the southern region (N = 40,727) was randomized into 2 treatment arms of equal size: (i) individual-benefits reminder; and (ii) individual- and social-benefits reminder.

Individuals residing in the study regions and belonging to the target group were unaware of the study. The reminders themselves did not make any reference to any experimental variation in wording. Nurses administering influenza vaccinations during the follow-up period were not aware that different letters were sent to eligible individuals. We had no direct contact with either the recipients of the mailed letters or the nurses administering influenza vaccinations in the target region. We did not obtain informed consents for this study, because we did not recruit any participants and analyzed anonymous administrative data. The study protocol was approved by the Finnish Institute for Health and Welfare’s Institutional Review Board (Decision Number: THL/1444/6.02.01/2018). The study protocol is available as a supporting information (S1 Protocol).

Outcomes

The impact of different letters on vaccination coverage was measured at the individual level using administrative health records. The Finnish National Vaccination Register contains nationwide records of all vaccinations given at public healthcare units in Finland since 2009 [17]. We used lot numbers to identify vaccine types and time stamps to determine when they were administered. The main outcome variable was having (versus not having) received an influenza vaccination during a 5-month follow-up period (from October 18, 2018 to March 18, 2019). We also used prior vaccination history data to study the potential heterogeneity of the average treatment effects.

Procedure

All reminders were sent via regular post to eligible individuals on October 17, 2018. All reminders were double-sided and written in both Finnish and Swedish to consider multilingual study populations. Individual identifiers (social security numbers) from the Finnish Population Register were used to match the received letters with complete vaccination records from the Finnish National Vaccination Register. The final dataset was produced using individual identifiers (encrypted social security numbers) that enabled us to merge population register data with administrative vaccination records. The final dataset did not contain any information that would allow for the direct identification of personal information.

Trial registration

As this study spans multiple disciplines, we preregistered the experimental design and submitted the preanalysis plan to multiple registries: the US National Library of Medicine Registry for clinical trials (clinicaltrial.gov, trial number: 240317), the American Economic Association Registry for randomized controlled trials (trial number: AEARCTR-0003520), and aspredicted.org (trial number: #15682).

Statistical analysis

Our randomized controlled trial included the entire population aged 65 years and above in the study regions. Consequently, we did not perform prospective sample size calculations. However, we report the minimum detectable effect (MDE) size for different treatment comparisons to assess whether potential null findings identify the absence of a true effect or signify a lack of statistical power. Our computations of MDE sizes do not account for potential corrections of multiple hypotheses testing. Taking into account the correlation of outcomes within (2-person) households, randomization at the household level, and a prior baseline vaccination rate of 31.8% in the western region, we computed that the (average) sample size of 2,441 individuals per treatment, divided into 1,740 clusters with an intracluster correlation of 0.7, was sufficient to obtain 80% power for a 5% (two-sided) level test for at least a 4.9 percentage point difference in the probability of receiving an influenza vaccination between any 2 treatments. Combining active treatment arms to estimate the impact of a reminder per se allows for the detection of smaller effect sizes with 80% power (Fig B in S1 Appendix).

The study population in the southern region was divided into 2 equally large treatment groups. Taking into account the prior baseline vaccination rate of 57.7% in the southern region, we computed that a sample size of 40,271 individuals, divided into 2 treatments and 29,395 clusters, was sufficient to obtain 80% power for a 5% (two-sided) level test for at least a 1.5 percentage point difference in the probability of receiving an influenza vaccination between the 2 treatments. More comprehensive power calculations that vary in statistical power and assumed intracluster correlations are available in S1 Appendix).

To determine the impact of reminders per se on influenza vaccination coverage, we estimated the pooled effect of the individual-benefit and the individual- and social-benefit treatments. We estimated statistical models using linear probability estimation, wherein the coefficient of the treatment indicator can be directly interpreted as the impact of the intervention on vaccination coverage. We used linear probability models for simplicity and ease of interpreting coefficient values. Table A in S1 Appendix provides results from logit models and multilevel mixed-effect linear models with an error structure that allows for cluster-level heterogeneity (random effects) at the household level. These alternative regression models provided extremely similar results. For reporting relative risk, we used a Poisson regression with standard errors clustered at the household level.

As preregistered, our primary statistical models did not include any control variables. However, we performed robustness analyses by running complementary linear probability models that controlled for prior vaccination histories and demographics (Table B in S1 Appendix). In addition, we assessed the robustness of our statistical estimates by running balance checks to test whether the random assignment successfully balanced demographics and individual vaccination histories across the treatment groups (Table 1).

Table 1

Summary statistics by study region and treatment (analysis sample).

	Descriptive statistics			Balancing tests—abs. standardized differences and p-values
Panel A: Western region	Control (N = 24,50)	Treatment I (N = 2,445)	Treatment I + S (N = 2,429)	I vs. Control	I + S vs. Control	I vs. I + S
Influenza vaccination, previous season	787 [32.1%]	818 [33.5%]	724 [29.8%]	0.028 (p = 0.401)	−0.050 (p = 0.139)	0.078 (p = 0.020)
Influenza vaccination, any year	1,097 [44.8%]	1,113 [45.5%]	1,033 [42.4%]	0.015 (p = 0.656)	−0.045 (p = 0.178)	0.060 (p = 0.070)
Any vaccination	1,752 [71.5%]	1,809 [74.0%]	1,747 [71.9%]	0.056 (p = 0.082)	0.009 (p = 0.776)	0.04 (p = 0.145)
Age	75.6 (7.86)	75.4 (7.79)	75.3 (7.71)	0.027 (p = 0.413)	−0.044 (p = 0.185)	0.016 (p = 0.615)
Women	1,268 [51.8%]	1,270 [51.9%]	1,256 [51.7%]	0.004 (p = 0.842)	−0.001 (p = 0.961)	0.005 (p = 0.808)
Single households	1,011 [41.3%]	1,027 [42.0%]	1,072 [44.1%]	0.015 (p = 0.659)	0.058 (p = 0.090)	0.043 (p = 0.209)
Joint test				(p = 0.573)	(p = 0.218)	(p = 0.330)
Panel B: Southern region	Control	Treatment I (N = 19,996)	Treatment I + S (N = 20,275)	I vs. Control	I + S vs. Control	I vs. I + S
Influenza vaccination, previous season	-	11,567 [57.8%]	11,683 [57.6%]	-	-	0.005 (p = 0.693)
Influenza vaccination, any year	-	14,280 [71.4%]	14,292 [70.5%]	-	-	0.020 (p = 0.071)
Any vaccination	-	16,243 [81.2%]	16,380 [80.8%]	-	-	0.011 (p = 0.304)
Age	-	74.0 (6.91)	73.9 (7.74)	-	-	0.019 (p = 0.097)
Women	-	11,398 [57.0%]	11,573 [57.1%]	-	-	0.002 (p = 0.816)
Single households	-	9,145 [45.7%]	9,372 [46.0%]	-	-	0.010 (p = 0.414)
Joint test						(p = 0.232)

Note: This table summarizes descriptive characteristics at baseline by region and treatment, and reports results from balancing tests. Reported descriptive statistics are frequencies, except for the variable Age, which shows the average age by region and treatment. Square brackets report proportions (%) and parentheses show standard deviations. Three last columns show results from balancing tests. First row in each cell shows absolute standardized differences in covariates between treatments. Second row in each cell shows p-values based on linear regression models that cluster standard errors at household level. The joint test of orthogonality across all covariates is based on a regression that includes all available (6) covariates and tests the joint hypothesis that β1 = β2 = … β6 = 0. Control = no reminder, Treatment I = individual-benefit reminder, Treatment I + S = individual- and social-benefit reminder.

To determine the impact of the different types of reminders on influenza vaccination coverage, we separately estimated the effects of the individual-benefits reminder and the individual- and social-benefits reminder. These models were estimated using linear probability models. In each model, we used standard errors clustered at the household level. We assessed the robustness of our findings by estimating random effect models that included households as a random intercept (Table A in S1 Appendix). We adhered to the Consolidated Standards of Reporting Trials (CONSORT) checklist (S1 CONSORT Checklist) for conducting and reporting of this trial.

Results

Population and baseline characteristics

Table 1 displays the baseline characteristics across the regions and treatments, showing large differences in the proportion of previously vaccinated individuals between the western and southern regions. Influenza vaccination coverage was 31.8% in the western region and 57.7% in the southern region at the end of the influenza season of 2017 to 2018. Notably, the differences in coverage were not limited to influenza vaccination. The proportion of individuals who had received any vaccination during the 9 years prior to the influenza season of 2018 to 2019 was 72.5% in the western region and 81.0% in the southern region. The average age in our samples was approximately 75 years. Most individuals were women and lived in households with 2 people 65 years and above.

Table 1 also shows results from balancing tests (absolute standardized differences and p-values). Using a critical statistical-significance threshold of p < 0.05, we find one statistically significant difference in covariate balance: Influenza vaccination coverage was higher in the previous influenza season in the individual-benefit treatment than in the individual- and social-benefit treatment in the Western region. This number of statistically significant imbalances is expected to arise by chance alone. To complete the baseline comparisons, we provide results from a joint test of significance across all 6 covariates and find that there are no systematic imbalances between the treatment arms at baseline.

Confirmatory analyses (preregistered)

The primary analysis compared influenza vaccination coverage across the experimental arms in the western and southern regions. We report intention-to-treat results. Thus, individuals in all treatment arms were expected to remain in the initially assigned treatment group. The only potential sources of attrition were emigration or mortality after the postal address was extracted from the population register. There was no reason to expect attrition to be correlated with treatment.

We first report the proportions and differences in proportions of influenza vaccination coverage by treatment arm in the western and southern regions (Fig 2). The statistical analysis adjusts for clustering at the household level. In the western region, we observed the highest rate of vaccination coverage in the individual-benefits treatment (41.8%, 95% CI, 39.5% to 44.1%), the second highest rate in the individual- and social-benefits treatment (38.9%, 95% CI, 36.6% to 41.2%), and the lowest rate in the no reminder (control group) treatment (34.0%, 95% CI, 31.8% to 36.2%). The difference in proportions between the individual-benefits treatment and the control treatment group was 7.8 percentage points (95% CI: 4.6 pp to 11.0 pp, p < 0.001 | Risk ratio: 1.23 (1.13 to 1.34)), 4.9 percentage points (95% CI: 1.7 pp to 8.1 pp, p = 0.002 | Risk ratio: 1.15 (1.05 to 1.25)) between the individual- and social-benefits treatment and the control treatment group, and 2.9 percentage points (95% CI: −0.4 pp to 6.1 pp, p = 0.087 | Risk ratio: 1.07 (0.99 to 1.16) between the individual-benefit treatment and the individual- and social-benefit treatment. Finally, we pooled both reminder treatments (Fig 3A) and found that the effect of receiving any type of reminder versus being in the control treatment group without a reminder was 6.4 percentage points (95% CI: 3.6 pp to 9.1 pp, p < 0.001 | Risk ratio: 1.19 (1.10 to 1.28)).

Fig 2

Vaccination coverage by region and treatment.

Control = No reminder, Treatment I = individual-benefit reminder, and Treatment I + S = individual- and social-benefit reminder. Bar graphs denote influenza vaccination coverage. Error bars denote 95% confidence intervals.

Fig 3

Vaccination coverage by treatment in the western region.

Panel A: Full sample (No reminder vs. Any type of reminder, pooling the I and I + S treatments); Panel B: Vaccination coverage by treatment in the western region stratified by prior vaccination status (No reminder vs. Any type of reminder). Error bars denote 95% confidence intervals.

In the southern region, we observed that vaccination coverage was similar in the individual- and social-benefit treatment (59.2%, 95% CI, 58.5% to 60.0%) and in the individual-benefit treatment (59.0%, 95 CI, 58.3% to 59.8%). Consequently, the difference in proportions of vaccination coverage between the individual-benefit treatment and individual- and social-benefit treatment was small (0.2 percentage points, 95% CI: −1.0% to 1.3%, p = 0.724 | Risk ratio: 1.00 (0.98 to 1.02)), indicating that there was no difference in vaccination coverage between the 2 reminder treatments.

Exploratory analyses (not preregistered)

We explored the effect of reminders conditional on prior vaccination history. Moreover, we estimated possible cross-vaccination spillovers from influenza vaccinations to other common vaccinations among the age group. Only data from the western region were used in the analyses, as only this subdesign included a group of individuals who did not receive either reminder.

We estimated the treatment effect of reminder letters conditional on one of 3 indicators of individual vaccination history: having versus not having received an influenza vaccination during the previous seasonal influenza period (2017 to 2018); having versus not having received an influenza vaccination during the 9 years prior to the influenza season of 2018 to 2019 (from 2009 to 2018); and having versus not having received any vaccination during the 9 years prior to the influenza season of 2018 to 2019 (from 2009 to 2018). The length of the prior vaccination period (9 years) was based on data availability and maximized the available length of individual vaccination histories before the treatment assignment.

Table 2 (columns 1 and 2) shows the joint effect of any type of reminder versus no reminder conditional on influenza vaccination status during the influenza season of 2017 to 2018 (1 year prior to the study). We found that the effect of receiving any type of (versus no) reminder on vaccination coverage was substantially larger among previously unvaccinated individuals (8.8 percentage points higher in the reminder versus no reminder conditions, which corresponded to a relative increase of 82%) than among previously vaccinated individuals (1.9 percentage point increase).

Table 2

The effect of written information letters on influenza vaccination coverage conditional on prior vaccination history in a region with historically low vaccination coverage (Western region).

	Influenza vaccination coverage (Western region)
	Conditional on influenza vaccination 2017–2018		Conditional on influenza vaccination 2011–2018		Conditional on any vaccination 2011–2018
	(1) Vac.	(2) Unvac.	(3) Vac.	(4) Unvac.	(5) Vac.	(6) Unvac.
Regression Coef.: Effect of any reminder (vs. no reminder)	0.019 (.017)	0.088*** (.012)	0.048** (.019)	0.084*** (.011)	0.050*** (.017)	0.053*** (.013)
Risk ratio: Effect of any reminder (vs. no reminder)	1.021 (.018)	1.824*** (.163)	1.071** (.030)	2.340*** (.312)	1.131*** (.041)	2.058*** (.417)
Observations	2196	5128	3243	4081	5308	2016
Coverage in control group (%)	87.3%	10.7%	68.0%	6.3%	45.5%	5.0%

Notes: Reported regression coefficients are estimated using linear probability models. Reported risk ratio coefficients are estimated using Poisson regression. All models are estimated at the individual level. Standard errors in parentheses are clustered at the household level. Indicators for prior vaccination in Models 1 and 2: having vs. not having received influenza immunization during the previous seasonal influenza period (2017–2018); in Models 3 and 4: having vs. not having received any influenza immunization during the 9 years (2009–2018) prior to the influenza season of 2018–2019; Models 5 and 6: having vs. not having received any immunization during the 9 years (2009–2018) prior to the influenza season of 2018–2019.

*** p < 0.01

** p < 0.05, * p < 0.1.

Table 2 (columns 3 and 4) shows the joint effect of any type of reminder versus no reminder conditional on having versus not having received an influenza vaccination during the previous 9 years. We found that receiving versus not receiving a reminder increased vaccination coverage by 8.4 percentage points (relative increase of 134%) among individuals who had not received an influenza vaccination during the previous 9 years. For those who had received at least 1 influenza vaccination during the past 9 years, the increase was 4.8 percentage points (relative increase of 7%).

Table 2 (columns 5 and 6) shows the effects of receiving versus not receiving a reminder conditional on having versus not having received any type of vaccination during the previous 9 years. We found a substantial positive effect (5.3 percentage points) even among the most consistently unvaccinated individuals. As overall influenza vaccination coverage in this unvaccinated group was low (5.0%), the relative effect size of receiving any reminder was very large among the most consistently unvaccinated individuals (106%).

Finally, we examined whether receiving a reminder about the importance of influenza vaccinations increased vaccination coverage for other common vaccinations among the study population. These analyses utilized the fact that our data included comprehensive patient records of all vaccinations received after the implementation of the intervention. We estimated cross-vaccination spillovers separately for the 3 most common types of vaccinations (other than influenza vaccinations), in this age-group: the pneumococcal conjugate vaccine (PCV), the tetanus-diphtheria (TD) vaccine, and the tick-borne encephalitis (TBE) vaccine. Moreover, we estimated the effect of receiving a reminder on the receipt of any other vaccine than influenza vaccine. Our results are reported in Table D in S1 Appendix and strongly indicate that there were no cross-vaccination spillovers. The estimated effects in all models were bounded to a tight interval around zero.

Discussion

The aim of this study was to investigate the effect of 2 different types of centralized written reminders (distributed via regular mail) on influenza vaccination coverage among the older adult population. We observed that a low-cost and scalable intervention relying on individually mailed reminders substantially increased influenza vaccination coverage in a population with low baseline vaccination coverage. However, our results suggest that there was no difference in influenza vaccination coverage between the individual-benefits reminder and the individual- and social-benefits reminder in either the region with historically low influenza vaccination coverage or the region with historically high influenza vaccination coverage.

Comprehensive patient records enabled us to measure the effect of reminders conditional on individuals’ prior vaccination history. The analyses revealed that the effect of reminders was substantially larger among individuals who had not received an influenza vaccination in the previous year. We also observed that even the most consistently unvaccinated individuals, who had not received any vaccination during the previous 9 years, responded positively to written reminders. By contrast, there was no statistically significant effect among previously vaccinated individuals. These findings suggest that the cost-effectiveness of interventions aiming to improve vaccination coverage may depend on the prior vaccination history of the target population.

Our results suggest that a written explanation of the social benefits of vaccinations, in addition to individual benefits, did not increase influenza vaccination coverage. In other words, we found that appealing to social responsibility, in addition to decreasing complacency, did not affect influenza vaccination coverage in our study population. Consequently, we conclude that, at least in the context of influenza vaccination and the reminder intervention used, communicating the social benefits of vaccination in the form of herd immunity leads neither to prosocial vaccination nor free riding on the vaccination efforts of other community members.

Our paper extends the study of behavioral interventions from hypothetical vaccination intentions and small-scale outpatient settings to a large-scale cluster-randomized controlled trial in which vaccination decisions are measured using comprehensive health records that include information about all vaccinations received before and during the follow-up period. The use of data from administrative health records had several key advantages. First, we were not restricted to studying vaccination intentions or self-reported vaccination outcomes but were able to objectively measure whether and when a vaccination occurred. Second, individuals residing in the study regions were not aware that different reminders were sent to eligible individuals. As a result, the generalizability of our results is not limited by the common concern that experimental results based on voluntary participation do not generalize to a population that was not aware of the experiment or that did not volunteer for the experiment when offered the opportunity. Third, the use of data from administrative health records enabled a sample size an order of magnitude larger than in typical randomized controlled trials that require the use of survey instruments to measure outcome variables. Finally, administrative health records of all vaccinations enabled us to measure potential behavioral spillovers to other age-appropriate vaccinations. More generally, this study serves as an example of how a randomized study design can be merged with high-quality administrative data to estimate causal effects in large and representative samples. Using comprehensive and exact administrative information about prior vaccination histories, or statistical variables that predict prior vaccination history in the absence of exact health records, constitutes a promising way to enhance the effectiveness of behavioral interventions aiming to improve vaccination coverage.

Our findings are largely consistent with the literature that has documented the effectiveness of patient reminders and recall interventions on vaccination coverage [7,8,18-20]. However, most of the existing evidence stems from outpatient provider office settings in which there is an active care relationship between the provider and patient. The conclusions from these studies may not necessarily apply to large-scale interventions within the general older adult population. In contrast, our study overcomes these limitations and tests the effectiveness of centralized reminders as an easily scalable and low-cost communication strategy in the general older adult population.

This paper is related to nascent literature that has tested the effectiveness of various communication strategies and behavioral interventions on vaccination coverage across different vaccinations and populations [21-24]. There is increasing evidence that communicating the social benefit of herd immunity using short texts or images without sufficiently explaining the underlying mechanisms (e.g., using interactive simulations [11]) is ineffective at increasing vaccination intentions [25-27]. Hence, the observed null result may partly be due to an ineffective communication format but could also relate to the well-known intention–behavior gap [28]. However, it remains to be studied whether communicating the social benefits of vaccination in the form of herd immunity increases vaccine uptake against more contagious diseases with more exact threshold for herd immunity, such as measles. Overall, our results parallel findings from the literature, which indicate that information materials tailored using behavioral science techniques have, at best, only a modest effect on vaccination coverage. It may also be that behavioral interventions motivate those who plan to vaccinate but does not persuade vaccine-hesitant individuals [29,30]. In contrast, there is some evidence from low- and high-income countries that modest in-kind incentives and direct monetary incentives may increase vaccination coverage [31-33].

We acknowledge that our study has several limitations. First, there could have been some contamination between the treatments if information about the reminders and their contents were shared between individuals (e.g., neighbors, friends, and other individuals in the receiver’s social networks) who belonged to different treatment groups. However, these kinds of information spillovers were minimized by the cluster-randomized design, which guaranteed that the same information would be received by all members of the same household. Second, the effectiveness of reminders may be underestimated, as we report intention-to-treat effects that disregard questions about the effectiveness of reminders among individuals who opened and read the letters. While the postal service in Finland is generally efficient and reliable, we could not obtain information about the proportion of letters that were successfully delivered, opened, and read by the recipients. The fact that the letters were written as centralized reminders (with printed letterheads and signatures by the local chief physicians) in collaboration with the Finnish Institute for Health and Welfare likely minimized recipients’ concerns about their authenticity. Third, we were unable to identify the impacts of reminders per se on influenza vaccination coverage in the southern region, because all the individuals in this region were assigned to either the individual-benefits treatment or the individual- and social-benefits treatment. Thus, we are not able to infer whether the effect of receiving any reminder depends on the aggregate rate of vaccination coverage in the study population.

In conclusion, this large-scale cluster-randomized controlled trial has shown how a behavioral intervention study can be combined with routinely collected high-quality administrative data to estimate causal effects in large and representative samples. We observed that a reminder informing older adults about the benefits of vaccination led to a substantial increase in influenza vaccination coverage in a population with low baseline vaccination coverage. This positive effect on influenza vaccination coverage was observed even among the most consistently unvaccinated individuals. These findings have meaningful implications for the financing of preventive health interventions and public health authorities that implement vaccination communication strategies to enhance vaccine uptake and curb the spread of infectious diseases.

CONSORT Checklist.

CONSORT, Consolidated Standards of Reporting Trials.

(DOC)

Click here for additional data file.

Table A. Average treatment effects estimated using linear probability models, logit models, and generalized mixed effects regression with random effects. Table B. Average treatment effects with and without control variables. Table C. The effect of reminders on influenza vaccine coverage by prior immunization history—Random effects linear model. Table D. Cross-vaccination spillovers to other age-appropriate vaccines. Fig A. Minimal detectable effect sizes (with α = 0.05 and 0.80) for treatment comparisons by intracluster correlation coefficients in the Western region. Fig B. Minimal detectable effect sizes (with α = 0.05 and 0.80) for the joint effect of any type of reminder by intracluster correlation coefficients in the Western region. Fig C. Minimal detectable effect sizes (with α = 0.05 and 0.80) for the treatment comparison by intracluster correlation coefficients in the Southern region.

(DOCX)

Click here for additional data file.

The original protocol/research plan.

(DOCX)

Click here for additional data file.

11 Jun 2020

Dear Dr Sääksvuori,

Thank you for submitting your manuscript entitled "Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external assessment.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Richard Turner, PhD

Senior editor, PLOS Medicine

rturner@plos.org

27 May 2021

Dear Dr. Sääksvuori,

Thank you very much for submitting your manuscript "Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial" (PMEDICINE-D-20-02666R1) for consideration at PLOS Medicine. We do apologize for the long delay in sending you a response.

Your paper was discussed among the editors and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to invite you to submit a revised version that addresses the reviewers' and editors' comments fully. You will appreciate that we cannot make a decision about publication until we have seen the revised manuscript and your response, and we expect to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We hope to receive your revised manuscript by Jun 17 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

Please let me know if you have any questions, and we look forward to receiving your revised manuscript.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

-----------------------------------------------------------

Requests from the editors:

It seems that your trial was registered at clinical trials.gov on November 20, 2018. Please explain how this relates to the expectation that trials are registered prospectively.

Please quote the study dates in your abstract.

Please quote aggregate demographic details for study participants in the abstract.

In the abstract and throughout the paper, please quote p values alongside 95% CI, where available.

Please combine the "Methods" and "Findings" subsections of the abstract.

The final sentence of the new combined subsection should begin "Study limitations include ..." or similar and should quote 2-3 of the study's main limitations.

Please remove the information about funding from the abstract (in the event of publication, this information will appear in the article metadata, via entries in the submission form).

After the abstract, we will need to ask you to add a new and accessible "Author summary" section in non-identical prose. You may find it helpful to consult one or two recent research papers published in PLOS Medicine to get a sense of the preferred style.

The final paragraph of the Introduction would seem more appropriate in the Discussion section, and we ask you either to remove or reposition it within the text.

Please restructure the early part of your Discussion section so that the first paragraph provides a summary of the study findings.

Please use the general style "aged 65 years".

Throughout the text, please style reference call-outs as follows: "... externalities [6,7]." (noting the absence of spaces within the square brackets).

In the reference list, please convert italics into plain text.

Author lists should be formatted as follows (reference 1): Salmon DA, Dudley MZ, Glanz JM, Omer SB.

Noting reference 25, please ensure that all references have full access details.

Again, please remove the information on study funding and competing interests from the end of the main text, as this should appear only in the article metadata.

Please supply a completed CONSORT checklist as a supplementary file, labelled "S1_CONSORT_Checklist" or similar and referred to as such in the Methods section.

In the checklist, please refer to individual items by section (e.g., "Methods") and paragraph numbers, not line or page numbers as these generally change in the event of publication.

Please attach the study protocol as a supplementary file, referred to in the Methods section.

Comments from the reviewers:

*** Reviewer #1:

This is a statistical review of manuscript PMEDICINE-D-20-02666R1. The manuscript is well written and the topic is particularly important nowadays given the current Sars-Cov-2 epidemic. I have a few important comments that require clarification.

Comments:

* Alignment between statistical section and Appendix C: appendix C uses 30% vaccination rate while the text uses 32%. So the text reports a MDE of 3.5% and the appendix 3.7%. The appendix uses 2450 participants per group, but the text uses 2465. I think that it is important to clarify in the text that you're effectively calculating the MDE. In other words it's not a prospective sample size calculation.

* "We estimated also these statistical models using linear probability estimation where the

coefficients represent marginal effects." Please clarify, what do you mean by marginal effects in this instance ?

* Discrepancy between text and abstract: "the effect of receiving any written information letter versus being in the control group without any written information was 6·4 percentage points (95% CI: 3·6 pp to 9·1 pp, p < 0·001)". The CI in the abstract is 4.1 to 8.8 pp. Please verify that all results match between the abstract and the text.

* Table A1: "To take into account the clustered randomization design linear probability models and logit models use standard errors clustered at household level, random effect model includes household as a random intercept". However, in the text, I find "In all regression models, we used standard errors that are clustered at the household level". I think that the text needs to be amended.

Minor comments:

* "the probability of receiving an influenza vaccination between any two treatments". It is actually the between the control group and one of the active arms (either I, or I+S).

* Table A1: p=0.000 does not exist. Please replace by P < 0.000

*** Reviewer #2:

This is a paper describing a cluster randomized trial evaluating the impact of a tailored information letter (about social benefits) vs a standard information letter vs no-letter on improving influenza vaccinations of older adults >65yrs of age in Finland. There appears to be two trials going on simultaneously: in the southern region of Finland (where baseline influenza vaccination rates are around 57%) it was a 2-arm comparative effectiveness trial testing standard letter message vs standard plus social benefit message within a letter. In the Western region (where baseline influenza vaccination rates are 32%), it was a 3-arm trial with control vs standard letter vs standard plus social benefit message in the letter.

The study found no benefit in the southern region suggesting no impact of social benefit messaging, but a benefit of either message over no-letter control in the western region, suggesting some benefit of any letter in a population with very low baseline rates. The study also found larger effect if no prior vaccination (in the western region with very low rates). I believe the intervention involved a single letter, and the clustering involved families.

Study strengths are that the question of testing the added value of a message with social benefit is a good one, and the system that captures vaccinations in Finland is comprehensive and also includes prior vaccinations so this is a true population-based study. It is also nice to see RCTs.

The study has a number of limitations:

Overall

The overall writing is highly complex, difficult to follow, often repeats, and is convoluted. I needed to read it several times to understand the paper. The methods and overall study design are fine, rather it is the writing that makes this paper challenging to follow

Introduction

The introduction of the paper states that the paper is designed to understand behavioral determinants of individual vaccination decisions—yet this study does not do that except that it compares whether adding societal benefit message helps. That is not the same as "understanding behavioral determinants.

The introduction seems to wander back and forth across topics. It does not cite the major Cochrane review of reminder/recall in 2018, but instead cites a JAMA paper (that was linked with a Cochrane review) in 2000. Throughout, it seems naïve about the literature on reminders for vaccinations. This type of reminder is called "centralized reminders" by immunization experts since the reminders were sent from a central group rather than physician practices.

Methods

While the methods are generally fine, the description of the methods is very complex and convoluted. I could not figure out the # letter reminders sent (I assume one),

Results-

The analyses seem written in a complex manner yet they are actually quite simple. Table 1 is confusing in that it is unclear why language was assessed. Table 2 is confusing. The figures are clear.

Discussion

The discussion section again says that the study assesses behavioral determinants of individual vaccination decisions - but it is not clear to this reviewer how it does that. I do see that it tested one behavioral intervention (social benefit). I believe the authors are trying to refer to behavioral economics research. On the other hand, the discussion fails to discuss the issue of very low baseline rates in the western region which may account for the bigger effects.

*** Reviewer #3:

The authors present the results of an interesting pragmatic trial of the effect of letters, including standard information or information about herd immunity/social emphasis) versus no letter across two geographic regions in Finland. A particular strength of this study is the careful selection of two different regions with different prior vaccination coverage to enhance generalizability and explore the effect of letters among patients with different prior exposure to the vaccine. Another strength is the long duration of individuals included in the study (>9 years) and complete coverage of influenza vaccination outcomes through population-level claims. The results largely confirm a large prior trial conducted in the US (along with other trials of mailed letters) that show that mailed letters can increase vaccination rates and that additional tailoring does not provide massive additional better compared with no letter.

Major comments

- Tailored letter: The authors emphasize that the letter is "tailored" about social benefits of vaccination due to herd effect. I posit instead that the letter emphasizes social consequences rather than just informational; it is not exactly tailored to individuals, so the language is inaccurate. It is also inconsistent, as the letters are described differently in the abstract versus the body of the manuscript. The authors should stick with what is written in the manuscript, as that appears more accurate.

- No control group in one region: The authors did not have a control group in the southern region (the region with better vaccination uptake). This means that the control group is likely biased towards being worse than the intervention group already (and therefore makes the overall results less believeable), because the only controls were in the region where there was worse baseline vaccination coverage. The authors recognize this as a limitation but need to be more careful in their interpretation, especially against control. It is understandable why no letter was sent, but the authors need to present results more clearly (including in the abstract) for just the Western region separately to overcome this.

Minor comments

- Abstract (Findings): The authors state that individually mailed letters increased influenza vaccination coverage, but it is not clear which arm (treatment) the results refer to, nor the comparison. This should be clarified. The results for all 3 arms should be presented, rather than subgroups, which is what is being presented now. The results across arms is shown in the last sentence, but the results should be presented independently.

- Statistical analysis (Power): The authors state that they assume an ICC of 0.5; is there citation for this or baseline data to support?

- Table 1: The authors should provide some data (e.g., absolute standardized differences or p-values) to describe balance in the groups. There is some evidence of imbalance across the arms on for example influenza vaccination receipt in prior season within region across arms, which biases towards seeing an effect in the letters (e.g., 33.5% vs. 32.1%).

- Control variables: The authors state that they did not include any control variables in the analysis but should adjust for imbalanced covariates in at least supplementary analyses.

- Figure 2: The scale should be 0-100% to avoid overinterpretation.

Discretionary comments

- Abstract (Methods): The authors do not provide information about the modeling or statistical analysis plan in the abstract.

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

6 Sep 2021

Submitted filename: PLOS_MED_R&R_D-20-02666R1_Ref_3_response_letter.docx

Click here for additional data file.

3 Nov 2021

Dear Dr. Sääksvuori,

Thank you very much for submitting your revised manuscript "Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial" (PMEDICINE-D-20-02666R2) for consideration at PLOS Medicine.

Your paper was discussed among the editors and seen again by two of our reviewers, and by a new statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, we will not be able to accept the manuscript for publication in the journal, but would like to invite you to submit a further revised version that addresses the reviewers' and editors' comments fully. We will be unable to make a decision about publication until we have seen the revised manuscript and your response, and we may seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We hope to receive your revised manuscript by Nov 23 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

Please let me know if you have any questions, and we look forward to receiving your revised manuscript.

Sincerely,

Richard Turner, PhD

Senior editor, PLOS Medicine

rturner@plos.org

-----------------------------------------------------------

Requests from the editors:

Please add "in Finland" or similar to the title.

We suggest adding a few words to the abstract to briefly explain the reason for the difference between the two regions in terms of the control condition.

Please quote the study's primary endpoint early in the "Methods and findings" subsection of your abstract.

Please adapt the "Conclusions" subsection of the abstract to begin "In this study, we found that ... was an effective ..." and adapt the tense of the remaining text in this subsection to match.

Please bullet the individual points in the "Author Summary".

At the start of the "Discussion" section (Main text) we suggest "The aim of this study ...", and make that "... two types of written reminder".

In the reference list, please use the journal name abbreviations "PLoS ONE", "BMJ" and "JAMA", and abbreviate other journal names as appropriate.

Noting reference 6 and others, please list only 6 author names, followed by "et al.".

Noting reference 7, please ensure that all references have full access information.

Please add "U S A" to reference 27.

Comments from the reviewers:

*** Reviewer #2:

This is a revised paper describing a population-level cluster randomized trial evaluating the impact of a tailored information letter (about social benefits) vs a standard information letter vs no-letter on improving influenza vaccinations of older adults >65yrs of age in Finland. There were two trials going on simultaneously: in the southern region of Finland (where baseline influenza vaccination rates are around 57%) it was a 2-arm comparative effectiveness trial testing standard letter message vs standard plus social benefit message within a letter. In the Western region (where baseline influenza vaccination rates are 32%), it was a 3-arm trial with control vs standard letter vs standard plus social benefit message in the letter. The study found no benefit in the southern region suggesting no impact of social benefit messaging, but a benefit of either message over no-letter control in the western region, suggesting some benefit of any letter in a population with very low baseline rates. The study also found larger effect if no prior vaccination (in the western region with very low rates). Perhaps the two largest added values of this paper are: (1) population-level study, and (2) is the sub-analysis by prior vaccination, showing that in this setting the intervention(s) had greater impact among those who did not receive prior influenza vaccinations, although the level of vaccine receipt rose to only about 20% in that subgroup.

The initial submission received generally positive reviews with some critiques, which the authors have addressed. Some of the critiques included unclear writing in numerous spots, which the authors have largely addressed. I have some remining minor critiques:

Abstract:

The Conclusion states this is a "low-cost" intervention yet the abstract does not describe costs, so this should be deleted. The last statement may be an exaggeration since the results shown include findings for both the standard letter plus the behavioral letter (and no difference between standard and behavioral letters) overall. I would suggest writing: "the effectiveness of letter reminders about the benefits of vaccination to improve influenza vaccination coverage may depend on the prior vaccination history of the population." I also did a "caveat" since other studies have not found this finding.

What do these findings mean?

I am confused by the mention of "text-based reminders"—do the authors have another study that involved text message reminders?

Introduction

This is now improved

Methods

I clicked on supplemental materials and I see (in English) the Individual benefit letter but cannot see the social good letter (I + S) that was used in one of the regions. So it is hard for me to judge whether the lack of a difference between the two letters was because including social good is not helpful in this region or whether the wording was not optimal in the social good letter.

Results

The text and figures/tables are mostly fine.

The Cochrane reviews have relied on relative risks (unadjusted and adjusted) and not just percents and regression analyses (with regression coefficients as in Table 2). It would be helpful to display relative risks, without lengthening the already very long Results section.

For Table 2—the subheadings could be better (under 1,2,3,4,5,6) to show what each columns mean (rather than only having that in the footnotes)

As a comment, the literacy level of the individual benefit letter is quite high. It is peculiar that it would have a greater effect than the individual benefit plus social good letter.

Discussion

This section is now good.

One suggestion I have is to also list some large-scale studies that have NOT found an effect of reminders. It turns out that increasingly, simple influenza vaccine reminders such as what was used here, at least in the US, have been found to not be effective in a variety of settings.

*** Reviewer #3:

Major comments:

1) Abstract: The abstract does not provide the referent group for the 6.4pp increase; is this compared with control or another type of letter? The authors state that individually mailed letters increased influenza vaccination coverage, but it is not clear which arm (treatment) the results refer to, nor the comparison. This should be clarified. The results for all 3 arms should be presented, rather than subgroups, which is what is being presented now. The results across arms is shown in the last sentence, but the results should be presented independently. In addition, some aspects of the abstract use words that could be over-interpreted, such as "crucially depends on", which is not supported by the data presented.

Minor comments:

2) Randomization: Was 1:1 simple cluster randomization implemented or did it account for any blocking factors? This is not currently clear.

3) Outcome capture: Is there any chance that individuals could have gotten the flu shot outside of the 5-month follow-up period (particularly before)? How were these individuals treated in the analysis? It would likely bias towards the null but would be important to clarify.

4) Households: Were any households more than 2-person? (e.g., siblings living with married siblings) The manuscript assumes only 2 adult persons/household.

5) Sample size calculations: It is not clear if the authors adjusted for multiple testing in the analysis (e.g., through Bonferroni). This is likely ok given their study question (and the null result) but would be good to clearly specify.

6) Table 1: The authors have not provided a description of balance changes across characteristics (e.g., ASDs or p-values) - this should be in Table 1, not online appendices. This was not sufficiently addressed in the prior revision.

7) Table 2: Please indicate which region these analyses are conducted in in the table. Also please include full titles of information to reduce the need for reviewers to go to footnotes.

8) Abstract: "lack of no reminder treatment": clearer to say "no control"

Discretionary comments:

9) "Text-based reminders" sounds like "text messaging" - recommend re-writing.

10) It is confusing to abbreviate to "Treatment I" and "Treatment I+S" - recommend putting more in a technical appendix but not to use non-standard abbreviations

*** Reviewer #4:

The previous statistical reviewer was not available for the revision. I have checked the changes made in addressing reviewers comments and am satisfied the authors have addressed the statistical comments well.

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

23 Nov 2021

Submitted filename: PLOS_MED_R&R_D-20-02666R2_Reviewer_#3_Nov_23_2021.docx

Click here for additional data file.

22 Dec 2021

Dear Dr. Sääksvuori,

Thank you very much for re-submitting your manuscript "Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial in Finland" (PMEDICINE-D-20-02666R3) for consideration at PLOS Medicine.

I have discussed the paper with editorial colleagues and it was also seen again by one reviewer. I am pleased to tell you that, provided the remaining editorial and production issues are fully dealt with, we expect to be able to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We hope to receive your revised manuscript in the first week of January. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

Please let me know if you have any questions, and we look forward to receiving the revised manuscript.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org

------------------------------------------------------------

Requests from Editors:

In the data statement (submission form) please finalize the statement (noting the current temporary wording "... will be shared ... after acceptance", noting PLOS' data policy, https://journals.plos.org/plosmedicine/s/data-availability.

In the "Methods and findings" subsection of the abstract, please adapt the sentence beginning "There was no control treatment in the region with high vaccination ..." to note that the control treatment was not included as general reminders had been sent in previous years.

In the abstract, please use the style "7398" and "40,727".

Please use the phrasing "... in the control treatment group" or "... with the control treatment", for example, in the abstract and throughout.

In the final subsection of the abstract, please amend the text to "In this study, we found that ... was an effective ...".

In the reference list, noting reference 3 and others, please use the conventional journal name abbreviations, e.g., "N Engl J Med." and "Proc Natl Acad Sci U S A.".

Noting reference 6, please ensure that all citations have full access details.

Please spell out the journal or source name for reference 30.

Comments from Reviewers:

*** Reviewer #3:

The authors have been sufficiently responsive.

***

Any attachments provided with reviews can be seen via the following link:

[LINK]

18 Jan 2022

Dear Dr Sääksvuori,

On behalf of my colleagues and the Academic Editor, Dr Lauffenburger, I am pleased to inform you that we have agreed to publish your manuscript "Information nudges for influenza vaccination: Evidence from a large-scale cluster-randomized controlled trial in Finland" (PMEDICINE-D-20-02666R4) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

Prior to final acceptance, please quote the trial registration number (e.g., at clinical trials.gov) on the title/abstract page.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper.

Sincerely,

Richard Turner, PhD

Senior Editor, PLOS Medicine

rturner@plos.org