'Reducing Delays In Vaccination' (REDIVAC) trial: a protocol for a randomised controlled trial of a web-based, individually tailored, educational intervention to improve timeliness of infant vaccination.

INTRODUCTION: Increasing numbers of children are failing to receive many recommended vaccines, which has led to significant outbreaks of vaccine-preventable diseases in the USA and worldwide. A major driver of undervaccination is parental vaccine hesitance. Prior research demonstrates that mothers are the primary decision maker for infant vaccination, and that their vaccination attitudes form primarily during pregnancy and early in their infant's life. METHODS AND ANALYSIS: This manuscript describes the protocol for an ongoing three-armed randomised controlled trial done at Kaiser Permanente Colorado (KPCO). The trial aims to test the efficacy of provided tailored, individualised information via the Internet to pregnant and new mothers versus untailored information versus usual care on the timeliness of infant vaccination. The primary outcome to be assessed is vaccination status, which is a dichotomous outcome (up to date vs not) assessed at age 200 days, reflecting the time when infants should have completed the first set of vaccine provided (at age 2, 4 and 6 months). Infants with one or more age-appropriate recommended vaccines at least 30 days delayed are categorised as not up to date whereas all other infants are considered up to date. Secondary outcomes include vaccination status at age 489 days, reflecting receipt of recommended vaccines at age 12-15 months, as well as vaccination attitudes, hesitancy and intention. Vaccination data will be derived from the electronic medical record and the state immunisation registry. Other secondary outcomes will be assessed by online surveys. ETHICS AND DISSEMINATION: The study activities were approved by the Institutional Review Boards of the University of Colorado, KPCO and the University of Michigan. Results will be disseminated through peer-reviewed manuscripts and conference presentations. TRIAL REGISTRATION NUMBER: NCT02665013; Pre-results. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

Introduction

Vaccination has been touted as one of the most effective public health interventions ever created.1 Despite this, increasing numbers of parents choose to delay or forgo recommended vaccines for their children because of uncertainty about the vaccines’ safety and necessity and general mistrust of the pharmaceutical industry.2 3 With this, increasing numbers of children are failing to receive many recommended vaccines, which has led to significant outbreaks of vaccine-preventable diseases in the USA and worldwide.4 5

Developing and evaluating interventions to counteract parental vaccine hesitancy and childhood undervaccination is a public health priority.6 While many prior interventions have been tested, the majority have not been effective.7–9 Addressing vaccine hesitancy can be difficult and time consuming because parents’ vaccination decisions are often complex as they are heavily influenced by emotion, past experiences and peers.10–12 Addressing this complexity can be difficult for healthcare providers who attempt to persuade parents to vaccinate their children,13 given that typical paediatric clinical encounters last only 15–20 min. As a result, even when parents only have a few questions that might be easily answered, a provider may feel ‘burnt out’ when having to discuss vaccines with questioning parents.14 In addition, in many cases, the resistance to vaccination is related to psychosocial and political beliefs as much, or even more than, knowledge deficits.

Given this, new approaches to address vaccine hesitancy that are time efficient and address the complex factors influencing vaccine decision-making are needed.15 One promising approach is to use message tailoring to provide parents with information about vaccines that is customised to their own personal needs before their child’s clinical appointments. Message tailoring allows for written information to be individualised to reflect each person’s unique beliefs, experiences, knowledge, attitudes and barriers to action.16 By doing so, the personal relevance of the information increases, which, in turn, improves individuals’ receptiveness to that information—this is especially important in the case of vaccine hesitancy when the new information may not align with a person’s current attitudes or beliefs.16 This approach has been shown to be effective for improving compliance with a number of health behaviours but only minimally applied to vaccination.17 18

This manuscript describes the protocol for a three-armed randomised controlled trial testing the effectiveness of a web-based tailored messaging intervention called ‘Vaccines and Your Baby’ (VAYB) versus an untailored version of the intervention versus usual care for improving timely uptake of recommended childhood vaccines.

Conceptual model

The conceptual model for the intervention is based on a hybrid of the theory of planned behaviour (TPB) and the value–attitude–behaviour hierarchy model (figure 1).19 20 It also incorporates strategies derived from motivational interviewing and self-affirmation.21 22 According to the TPB, behaviour (in this case, following the recommended vaccination schedule) is influenced by intentions (in this case, vaccine hesitancy), which are a result of attitudes towards the behaviour, perceived behavioural control and norms. This intervention primarily focuses on strategies for influencing attitudes—that is, tailored messages addressing individual behavioural beliefs (eg, beliefs that immunity is best achieved through exposure to a pathogen or ‘natural immunity’) framed according to personal values (eg, emphasising the benefits of vaccination for preventing spread of illness among the young and elderly for those who value protecting one’s community). By affirming individual patient values and identity, using non-judgmental and empathetic language, emphasising autonomy (ie, adding tenets from motivational interviewing and self-determination theory)21 23 24 and constructing controlling tones of messages, this can minimise reactance and counterarguments. Individually tailored messages, in general, are known to have greater effects on attitude change than are universal (untailored) messages.20 25 26 According to the value–attitude–behaviour hierarchy model,20 values influence attitudes and behaviour across cultures and domains, including recycling, consumer behaviour and alcohol consumption.26 27 This hybrid approach to establishing the conceptual model allows us to focus the intervention strategies on addressing a select set of known determinants of vaccine hesitancy and behaviour, rather than incorporating the universe of behaviour change techniques into our intervention.

Figure 1

Model of parental vaccine values, vaccine attitudes, hesitancy and behaviour.

Aim and hypothesis

The primary aim of this study is to conduct a three-group randomised, intervention trial to measure the effectiveness of the VAYB intervention versus a similarly constructed but untailored intervention versus usual care on vaccination receipt and timeliness during an infant’s first 15 months of life. Our intervention approach (the VAYB intervention) is novel in that as it combines values framing with message tailoring for vaccination to change parents’ attitudes and behaviour. The primary hypothesis to be tested is that infants of mothers who receive values-framed, individually tailored messages (eg, the VAYB intervention) will have lower levels of vaccine hesitancy and more up-to-date vaccination behaviour than those receiving an untailored version of the intervention or those receiving usual care. A secondary aim of the project is to assess the impact of the intervention on vaccination attitudes and hesitancy level, particularly as these relate to our conceptual model described above.

Methods

A summary of the trial’s specifications is presented in table 1.

Table 1

Trial registration data set summary table

Data category	Information
Registry and trial number	ClinicalTrials.gov—NCT02665013
Data of registration	01 April 2016
Secondary identifying numbers	CO-IRB #: CO-15–2299_07
Financial support	National Institutes of Health
Contact for queries	amanda.dempsey@ucdenver.edu
Title	The REDIVAC study—Reducing Delay in the Vaccination of Children
Countries of recruitment	USA
Health condition studied	Infant vaccination
Interventions	Active comparator—tailored educational website Placebo comparator—untailored educational website Passive comparator—usual care
Key inclusion and exclusion criteria	Inclusion: >18 years, pregnant in third trimester or child <2 months of age, receives care at KPCO health system, able to read English and access to the Internet Exclusion: high-risk maternal or fetal health condition, maternal social issues (such as abuse), fetal or infant death and does not plan to have infant receive care in KPCO health system after birth
Study type	Individually randomised controlled trial
Date of first enrolment	20 April 2016
Target sample size	700
Trial status	Ongoing data collection
Primary outcomes	Average days undervaccinated Up-to-date vaccination status
Key secondary outcomes	Vaccination attitudes Vaccination values Vaccine hesitancy level

KPCO, Kaiser Permanente Colorado.

Study design and registration

This is a three-armed, individually randomised clinical trial with longitudinal follow-up. Study arms include (1) the VAYB (tailored) intervention, (2) an untailored version of the intervention and (3) usual care. The study is registered with ClinicalTrials.gov (see table 1 for details).

Study overview and setting

In the trial, participants are active in the study from the time of enrolment (from late pregnancy or the first 2 months of their infant’s life) until their infant reaches 15 months of age (489 days). The primary vaccination outcome to be assessed is a dichotomous outcome of vaccination status (up to date versus not) that is based on the average number of days undervaccinated for all vaccinations in the recommended vaccine schedule. Assessment of vaccination for the 2-month and 4-month vaccines occurs when the infant is 200 days old (to provide additional time beyond the exact date the vaccine was due). Secondary vaccination outcomes and outcomes related to attitudes and hesitancy are assessed at age 489 days. This time period was chosen to encompass three critical decision-making points associated with the vaccination process: (1) during pregnancy when many vaccination decisions and attitudes are being formed28 29; (2) during the time period that corresponds to the ages when the initial infant series is recommended (generally at ages 2, 4 and 6 months) and (3) during the second stage of the infant vaccination series at age 12–15 months when vaccines different from those offered at the initial stage are introduced. The primary outcome (vaccination behaviour) is assessed using data from the electronic medical record (EMR), augmented with data from the Colorado immunisation registry, Colorado Immunization Information System (CIIS).

The study takes place via the Internet. Participants in the VAYB and untailored arms view educational materials on a web-enabled device or computer of their own and are prompted to view this information again at specific time points during the study (described below). Participants enrolled in the usual care arm receive by mail Vaccine Information Statements (VISs) for all recommended vaccines in the child’s first year of life; VISs are not provided by mail to participants in the VAYB or untailored arms. Participants in all arms complete surveys at baseline, and three additional time points (table 2). Participants are reminded to take the survey at these intervals via a series of emails. Following survey completion, participants are taken automatically to the website that contains either tailored or untailored information, depending on the study arm. The infants of participants in all arms receive care at participating clinics (an eligibility criterion, see below) where VISs are provided to all study groups as part of routine care.

Table 2

Timing and content of study questionnaires

Timing	Rationale	Content
Last trimester of pregnancy or child <2 months of age	Preintervention questionnaire required for study enrolment. Our prior research indicates infant vaccination decisions are actively forming among expectant mothers.	Intention to vaccinate Vaccination values Vaccination attitudes Logistical barriers Vaccine hesitancy status Demographics
At child age 4–6 months	First round of infant vaccines is typically provided at age 2, 4 and 6 months. The same vaccines are given at each visit.	Intention to vaccinate Vaccination attitudes Logistical barriers Vaccine hesitancy (only 3 of 5 Qs)
At child age 10–12 months	The same vaccines are provided at age 2, 4 and 6 months, thus decisions made at the 2-month visit are likely to be followed for 4-month and 6-month vaccines. However, several new vaccines are introduced at the 1-year visit. Vaccine-hesitant parents are likely to need additional, new information for making decisions about the vaccines provided at age 1 year.	Intention to vaccinate Vaccination values Vaccination attitudes Logistical barriers Vaccine hesitancy status
At child age 13–15 months	End of study assessment to track changes over crucial time periods of vaccine decision-making.	Vaccination attitudes Vaccine hesitancy status Satisfaction with website

Study population and inclusion/exclusion criteria

Women in the third trimester of pregnancy enrolled at Kaiser Permanente Colorado (KPCO) between April 2016 and October 2017 are recruited for the trial. KPCO is a non-profit, managed care organisation serving ~667 000 individuals. Each year ~5000 pregnant women and 140 000 children receive healthcare at KPCO clinics. Study participants can enrol from the first recruitment outreach that occurs in the last trimester of pregnancy to when their infant is ≤2 months of age. The infant must be enrolled in the KPCO health plan to continue participation in the study.

A combination of EMR data and study screening questions are used to determine study eligibility. First, the EMR is used to identify English-speaking women, currently enrolled at KPCO, and ≥18 years of age in the last trimester of pregnancy, based on clinically determined expected delivery date. All identified women with a diagnosis International Statisical Classification of Disease, or (ICD10) code in the past 8 months indicating potential abortion, miscarriage, adoption, fetal anomalies, or genetic disorders in the pregnancy, or a high-risk maternal condition (ie, cancer) are flagged for potential exclusion. Medical chart reviews are conducted on these women and they are definitively excluded as potential participants if the EMR indicates that their fetus has a high-risk condition (eg, fatal heart condition, trisomy 18 and anencephaly), or they have a spontaneous or elective abortion, social issues (such as domestic violence) or serious health concerns. Screening questions are delivered online before consent to ensure participants plan to use KPCO medical care for their child, are ≥18 years of age, and are currently pregnant or have a child less than 2 months of age. During the course of the study, participants are removed if they have a fetal demise, infant death, if the infant loses KPCO insurance coverage for greater than 90 days, if they request to be removed from the study or if they die. These data are obtained from a monthly data extraction from the EMR and patient report.

Consent and recruitment

Recruitment occurs via a multistep process. After the EMR is used to screen for initial eligibility, a series of two letters, three emails and one phone call are sent to potential participants 1–2 weeks apart to direct patients to the KPCO study registration website created specifically for this study. On this registration website, identity and eligibility are confirmed, and the participant is consented by signing an online form.

After consent, participants are directed to the study website where they set up login information and are provided with a preintervention questionnaire that assesses their baseline intention to vaccinate, vaccination values, logistical barriers to vaccination, vaccine hesitancy (used for randomisation) and demographics, and reconfirms eligibility. Previously developed and validated measures are used to assess these items.30–32 On completion of this questionnaire, participants are considered to be ‘enrolled’ in the study and are randomised (described below). The screening, consent and enrolment process is repeated monthly until the target sample size is reached.

Assignment of interventions

Participants are randomised on a 1:1:1 basis between the VAYB, untailored and usual care arms. The allocation assignment is generated by back-end software embedded in the study website. Randomisation occurs immediately following enrolment into the study (ie, after completion of the preintervention questionnaire) and remains in place throughout the study. Stratified randomisation along with a permuted block technique is used such that participants are first stratified into either a hesitant or non-hesitant group, based on responses to the preintervention questionnaire. Hesitancy status is assessed using a five-item validated measure developed by Opel (personal communication) and participants are categorised based on the measure’s suggested (but unpublished) cut-offs. Participants from each group are then added to their own set of blocks that each contains six slots. There are two slots available for each of the three study arms. These slots are randomly ordered when the block is created. When all six slots are filled, a new block with six randomly ordered slots is added.

Blinding

Participants are not informed about which study arm they are assigned to, but descriptions of the three potential arms for assignment are provided in the study consent documents. Thus, although they are not told specifically which arm they are in, they are not blinded to their study assignment. The project manager for the study will convert study data to unlabelled arms (ie, arm 1, 2 or 3) allowing for the rest of the study team to be blinded to study arm assignment during the analysis and data interpretation phases of the project. Unblinding will occur when data analysis is complete for the primary study outcome. Clinics, where participants receive care, are not aware of the individuals participating in the study unless brought up by the patient during a clinical encounter.

Sample size calculation

We considered, based on prior studies,3 33 an OR between 2.0 and 3.0 for up-to-date vaccination status between the intervention study arms and usual care to be clinically meaningful. For this, we estimate a needed sample size of 477 (OR=3.0) to 1002 (OR=2.0) participants. This sample size is based on an assumption of 15% of the recruited population being vaccine-hesitant (as has been the case in prior studies in this population)33 and therefore not up to date in their infant’s vaccination, a 1:1:1 randomisation allocation ratio, two-sided tests of statistical significance, 80% statistical power and a 5% type I error rate. Accounting for an attrition of 15%, we need to enrol 561–1179 participants.

Interventions

Tailored intervention

In the VAYB arm, messages were tailored for multiple constructs including intention to vaccinate, personal attitudes about vaccines, vaccination values (table 2), vaccination beliefs and concerns, logistical barriers to vaccination, and child’s name, sex and birthday. Data to inform this initial tailoring come from the preintervention questionnaire. Interim questionnaires are used to refresh the tailored information at three times during the study period. Tailoring occurs based on an embedded algorithm that is part of the VAYB website. An in-depth description of the process used to develop the VAYB intervention, and the resulting content, is described in detail elsewhere but examples are provided in table 3.31 32

Table 3

Examples of VAYB website content for two topics, showing tailoring based on three different values

Vaccines and Your Baby: tailored messages
Value	Topics
	‘Alternative/delayed vaccine schedules’ message	‘Doing your own research on vaccines’ message
Security—disease prevention	Like many parents, your main goal is to keep your child healthy. The last thing you want is for your child to get an illness you could have prevented with a simple vaccine.	You’re the kind of person who will do everything she can to protect her baby from illnesses.
Self-direction	You’re not one to just do what other people tell you to do. You know your child better than anyone, and you have choices to make. You want to do your own research about vaccines. You don’t want him/her to get a disease. But you don’t want to put him/her at risk by getting vaccines.	You’re the kind of person who plays an active role in decisions about her baby’s health.
Security—vaccine risk	That’s a lot of needles (and a lot of tears)! You want to protect your child. But with so many vaccines at once, you’re concerned about exposing him/her to too many unnatural ingredients all at once.	You’re the kind of person who will do everything she can to protect her baby from pain or unnecessary medicines.

VAYB, Vaccines and Your Baby.

On completion of the preintervention questionnaire where initial tailoring information is obtained, participants are automatically directed to the VAYB website, which is individually customised based on their responses. The most highly tailored content is in three ‘Just for You’ tiles that are displayed prominently on the page (figure 2A). These tiles reflect the top three vaccine topics of concern that each participant indicates they want more information about and are further customised to highlight the vaccination values the participant most endorses, and to reflect their most recently reported intention to vaccinate. The remaining content is lightly tailored to reflect participant’s attitudes, concerns, hesitancy and demographics, but is not tailored based on vaccination values. Highlighted text on the home page (figure 2A) is used to further identify additional information that is most relevant to the participant based on their survey answers. The website is refreshed three times during the course of the study based on interim assessments of participants’ attitudes, beliefs, concerns, values and vaccine hesitancy. Specifically, when the infant is 4–6 months of age, participants reanswer all questions excluding the value items and questions used to assess vaccine hesitancy, and the content is refreshed accordingly. Values are reassessed again in a third survey when the infant is 10–12 months of age and the website is refreshed to reflect any new content. Vaccine hesitancy level is reassessed at a fourth survey and the content is again refreshed. Participants receive a gift card after each survey is completed. For all time points, vaccination intention is assessed immediately before and within the hour after viewing the website content (VAYB and untailored arms). A reminder for this vaccination intention assessment is sent to non-responders after 1 day.

Figure 2

(A) ‘Landing page’ of the VAYB website annotated to highlight various types of message tailoring. The right arrow denotes the ‘Just for You’ tiles that represent the most highly tailored content on the VAYB website. The left arrow denotes additional text reflecting lightly tailored material that is particularly salient to the participant based on their survey responses. (B) Landing page of corresponding untailored website that lacks message tailoring. VAYB, Vaccines and Your Baby.

Untailored intervention

To isolate the impact that message tailoring has on mothers’ vaccination attitudes and behaviour, the primary comparator group in the study is a website that is similarly constructed as the VAYB website but lacks tailored elements. Specifically, the untailored intervention has similar text, content and design as the VAYB intervention, but is not linked to survey responses to make the messages individually customised (figure 2B). This means that each participant in this arm receives identical content, messages, text and images. For example, instead of name-tailoring, the text uses generic references, such as ‘you’ and ‘your child’. The recommended vaccine schedule is static in the untailored site compared with the tailored site, which highlights upcoming vaccines based on the child’s age. The order of content displayed is fixed throughout the study period as there is no linkage of the website’s text to participants’ values or attitudes. In addition, the highly tailored ‘Just for You’ tiles are not present in the untailored intervention. The same questionnaires administered to participants in the VAYB arm are administered to participants in the untailored arm, but the material is not used to refresh the website content.

Usual care

After taking the preintervention questionnaire that is used to determine randomisation, participants in the usual care arm are thanked for their information and logged off the study website. They receive an email containing their gift card and are mailed the VISs for the vaccines due in the child’s first year of life. They do not have access to the VAYB or untailored websites used for the other arms of the study but do receive the same interim questionnaires at the same time periods as the VAYB and untailored arms (see table 2). They continue their usual care and their infant’s vaccination status is assessed prospectively when their child turns 200 days old (primary outcome) and again when their child is age 489 days (secondary outcome).

Routine paediatric care is available to infants of all participants in the study. At KPCO, usual care typically consists of a series of paediatric, well-child care visits at age 2 weeks, 2 months, 4 months, 6 months, and 12 months, with an optional visit at age 9 months if desired by the healthcare provider or parent. Visit content is structured based on the Bright Futures programme of the American Academy of Pediatrics, which provides detailed guidelines regarding the content and schedule of paediatric health supervision visits.34 The visit content is intentionally broad, with visits focused on the needs of the child and family that typically last 20 min or less. On the basis of the data in the EMR, a previsit informational sheet lists the vaccines recommended at that visit. Parents are also provided with the VISs relevant to that visit. Providers are often asked about vaccination, and can provide additional information verbally, although the small window of time available for visits can limit discussion.

Outcomes

The primary outcome of the study is a dichotomous categorisation of vaccination status (up to date vs not up to date) that is defined based on a continuous measure of days undervaccinated. This outcome is assessed at age 200 days to cover vaccines in the initial infant vaccination series and to minimise the loss to follow-up. The following six vaccines recommended by the Advisory Committee on Immunization Practices will be assessed: hepatitis B, rotavirus, diphtheria-tetanus-acellular pertussis, Haemophilus influenzae type b, pneumococcal conjugate vaccine and polio. All vaccination data are obtained from KPCO’s EMR and CIIS.

To categorise vaccination status, we will first assess the number of days undervaccinated for the 2-month and 4-month vaccines (combined), by calculating the difference between when a vaccine dose was actually administered and when a vaccine dose should have been administered according to the vaccination schedule recommended by the Advisory Committee on Immunization Practices,35 plus an additional 30-day ‘leeway’ to account for vaccination that did not occur at exactly the minimal interval between doses. For example, the first dose of rotavirus vaccine is due at age 2 months (61 days) but is not considered late until age 92 days. Days undervaccinated for this dose begin accruing on day 93. The number of days undervaccinated is then summed across all doses and vaccines to calculate a total number of days undervaccinated for each infant and can range from 0 to 648 days. Infants with 0 total days undervaccinated (assessed specifically for the 2 and 4 months vaccines) at 200 days will be considered up to date on their vaccination status; those with ≥1 day undervaccinated (representing at least a 30-day delay for at least one vaccine) will be considered not up to date.

A secondary vaccination metric that is assessed is up-to-date status for measles–mumps–rubella and varicella vaccine at 489 days, when delay for the first dose of these vaccines begins. This metric is useful because it incorporates outcomes related to parents’ decision-making about these two vaccines recommended at age 12–15 months that are not offered previously.

The interventions’ impact over time on a variety of additional secondary outcomes that are based on the constructs of our conceptual behavioural model (figure 1) and assessed as part of the baseline and interim questionnaires will also be assessed. These include changes over time in vaccination attitudes and hesitancy, and how these relate to study arm, vaccination values and vaccination status. Vaccination attitudes are assessed using measures previously developed by our team and others,30 values are assessed using a novel vaccination values framework we have developed (manuscript in preparation) and vaccine hesitancy is assessed using a five-item validated measure developed by Opel (personal communication). A variety of covariates and potential moderators will be assessed as part of this analysis including patient age, gender and insurance (some patients have Medicaid KPCO coverage), and mother’s age, race and ethnicity. Also included will be metrics measuring website engagement (VAYB and untailored arms only) including time spent on the website, number of times viewing the website, number and order of pages viewed, and match between stated concerns and website material viewed (VAYB arm only).

Data collection methods

Vaccination data are collected routinely as part of clinical care within the KPCO health system and will be assessed from the KPCO EMR data warehouse at predefined ages (200 days and 489 days). CIIS will be used as a secondary vaccination data source, though internal audits demonstrate that >95% of childhood vaccines given to KPCO patients are captured within the EMR. Survey data are collected on the Internet based on user responses to the online questionnaires.

Participant retention

To assist with retention, participants receive a US$20 gift card incentive for each survey they complete. However, even with this incentive, we expect some drop off in survey participation. Because our primary outcome is vaccination status, mothers who do not participate in all the study surveys are still able to have the primary study outcome assessed, so long as their child maintains coverage and continues to seek care within the KPCO health system. On the basis of past studies, we expect that the proportion of mothers who discontinue KPCO coverage after the birth of their infant to be ~15%, and our study is powered with this attrition in mind.33

Data security and storage

To ensure that the data are protected, several methods are used. Personal identifying data collected on study websites are limited to a participant generated username, email address and child birthdate. The only other data collected on the study websites are vaccine attitudes, beliefs, values and demographics. The study websites use virtualised servers housed at redundant data centres and access is password protected. Virtual servers are backed up automatically onto encrypted tape for recovery and security. Data provided to researchers from the website are encrypted if they are transmitted across the Internet. Data use agreements are in place across all study team member sites.

All medical record data are collected and stored at KPCO behind the firewall in secure password protected files. This data set is linked to a study ID. A limited data set devoid of personal identifying information will be used for data analyses. Data will be shared with study team members through a secure file transfer. Only members of KPCO research project team have access to the personal identifiers linking the study IDs to specific study participants.

Statistical methods

Total days undervaccinated will be analysed primarily as a dichotomous variable (up-to-date vaccination status) and secondarily as a continuous measure. Categorically defined up-to-date vaccination status will be analysed using logistic regression to estimate ORs and associated 95% CIs. For the continuous measure, because total days undervaccinated has a highly skewed distribution, we will use a non-parametric analysis and a rank transformation approach. For both measures, we will conduct analyses stratified by baseline vaccine hesitancy.

For survey measures, descriptive statistics will be assessed and changes in vaccination attitudes and intention over time will be calculated. All measures are assessed using Likert scales and will be analysed as linear measures. Repeated measures analysis of variance (ANOVA) will be used to assess the intervention’s impact on average change by arm for each of these outcomes. Mixed linear models will be used to assess the ‘difference in difference’ over time in these means, by arm, controlling for the covariates described above. Website utilisation data will be measured primarily using linear measures (time spent on the website in minutes, number of times logging in, number of web pages viewed, etc) and may be included in the mixed linear models. ANOVA will be used to assess the association between each of these website utilisation measures and study arm.

Analytic framework

We will use a modified intention-to-treat framework for the analysis of vaccination outcomes. This analytic cohort will include infants of all randomised mothers who maintained KPCO health coverage for the allotted amount of time (200 days for the primary outcome and 489 days for the secondary outcome) with no more than 90 days of no coverage, and thus have vaccination data available for assessment. For survey outcomes, we will use a modified intention-to-treat analysis that includes all participants with data from at least one non-baseline questionnaire.

Missing data

As described above, nearly all vaccines provided to KPCO patients are documented in the EMR, and doses provided outside KPCO are documented in CIIS. Therefore, we expect there to be minimal missing data for vaccination outcomes. To ensure the most complete record, CIIS will be cross-checked for all participants to identify any vaccine doses given to infants outside the KPCO system that are missing from the KPCO EMR. Participants who do not have vaccination data present in either system will be assumed to have not gotten a vaccine dose elsewhere.

For survey data, due to our recruitment strategy, we anticipate no missing data at baseline, as completion of the baseline survey is a criterion for entry into the study. However, there may be missing data for subsequent surveys as these are not required to remain in the study. For missing data in surveys beyond baseline, multiple imputation models will be developed for analyses involving multiple survey points where greater than 10% of subjects would be lost due to missing values.

Subgroup analyses

The main subgroup analysis planned is examining the efficacy of our intervention by vaccine hesitancy status (dichotomous variable), as defined by the five-item Opel measure described above.

Monitoring

KPCO EMR data on participants and their infants will be used to identify any deaths or loss of KPCO insurance coverage, which are subsequently chart reviewed for accuracy. Participants who die or experienced an infant death, or have >90 days loss of insurance coverage, will be removed from the study and will not be included in the modified intention-to-treat analysis. All participants will be monitored weekly for completion of the various surveys in the study and reminder emails will be sent on a pre-set schedule to those who have not completed them. However, failure to complete any surveys beyond the baseline survey will not be the cause for removal from the study.

Assessment of harms and adverse events

Study participants are provided with contact information for the research team and encouraged to contact the team if they experience any adverse events (AEs) related to their participation in the study (eg, being contacted after an infant death). AEs are expected to be very unlikely given the nature of the study and our monitoring procedures. However, should any significant AEs occur, they will be reported to the appropriate institutional authorities.

Ethics and dissemination

Informed consent

All mothers in the study are informed about the study, the risks and benefits, and provide written informed consent via an online registration process prior to participating in the study. As part of the consent process, participants are informed that they may withdraw from the study at any time without impacting their clinical treatment.

Access to data

The data will be accessed only by authorised persons directly involved in the study from the University of Colorado Denver, KPCO and the University of Michigan. Access to a de-identified, aggregated version of the data set and analysis code will be available on request and approval of the study team.

Dissemination plans

Results of the study will be presented at national and international research conferences and through peer-reviewed publications. Any changes to the study protocol will be clearly communicated to journals publishing the study results in a manner that aligns with the journal’s policies for reporting clinical trials. Consolidated Standards of Reporting Trials (CONSORT) guidelines36 will be followed when reporting study outcomes. Study materials, such as questionnaires and screenshots of the intervention websites, will be available to researchers on request from the study principal investigators. If the VAYB intervention proves to be efficacious in reducing delays in the timeliness of infant vaccination, the study team will work with web-developers and community organisations to explore options to make the website available to the general public.

Patient and public involvement

Patients were first involved in this research when designing the intervention, which is informed by the literature, and by the research teams prior to clinical and research experience. The bulk of patient involvement is as research participants. They will not be involved in recruitment or conduct of the study, data analysis or dissemination.