Interventions to increase immunisation coverage among children 12-23 months of age in India through participatory learning and community engagement: pilot study for a cluster randomised trial.

OBJECTIVE: With the aim of conducting a future cluster randomised trial to assess intervention impact on child vaccination coverage, we designed a pilot study to assess feasibility and aid in refining methods for the larger study. TRIAL
DESIGN: Cluster-randomised design with a 1:1 allocation ratio.
METHODS: Clusters were 12 villages in rural Uttar Pradesh. All women residing in a selected village who were mothers of a child 0-23 months of age were eligible; participants were chosen at random. Over 4 months, intervention group (IG) villages received: (1) home visits by volunteers; (2) community mobilisation events to promote immunisation. Control group (CG) villages received community mobilisation to promote nutrition. A toll-free number for immunisation was offered to all IG and CG village residents. Primary outcomes were ex-ante criteria for feasibility of the main study related to processes for recruitment and randomisation (50% of villages would agree to participate and accept randomisation; 30 women could be recruited in 70% of villages), and retention of participants (50% of women retained from baseline to endline). Clusters were assigned to IG or CG using a computer-generated randomisation schedule. Neither participants nor those delivering interventions were blinded, but those assessing outcomes were blinded to group assignment.
RESULTS: All villages contacted agreed to participate and accepted randomisation. 36 women were recruited per village; 432 participants were randomised (IG n=216; CG n=216). No clusters were lost to follow-up. The main analysis included 86% (373/432) of participants, 90% (195/216) from the IG and 82% (178/216) from the CG.
CONCLUSIONS: Criteria related to feasibility were satisfied, giving us confidence that we can successfully conduct a larger cluster randomised trial. Methodological lessons will inform design of the main study. TRIAL REGISTRATION NUMBER: ISRCTN16703097. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

RCT Entities:   Population Interventions Outcomes

This feasibility study for a cluster randomised trial of participatory, community-based interventions to improve child vaccination coverage closely replicated the methods of the planned main study, and enabled us to conclude with confidence that we could successfully carry out the main study.

Field experience enabled us to identify key potential barriers to success of the larger trial related to quality of administrative data on immunisation, baseline covariate imbalances following randomisation, measurement and information biases, and contamination, and to develop strategies to address them in the main study.

This pilot study also considered a range of secondary outcomes using appropriate statistical methods, including cluster-level and individual-level proxy end points indicative of intervention effectiveness.

Analyses revealed an important positive effect of interventions on several proxy end points; however, the relationship of proxy to final end points is unknown, multiple proxy end points were considered, and results were not fully consistent. The number of study clusters was small.

A definitive judgement concerning intervention impact on child vaccine coverage must await the larger study.

Introduction

Background

Immunisation has been instrumental in global progress towards the UN Millennium Development Goal (MDG) to reduce mortality in children under 5 years of age (MDG 4).1 The potential for future impact is even greater, due in part to highly effective new vaccines that can protect against some forms of diarrhoea and pneumonia; the WHO and UNICEF estimate that 29% of deaths among children aged 1–59 months are now vaccine preventable.2

To realise their potential, antigens must reach all children; yet, one in five children worldwide still do not have access to basic vaccines.3 4 In May 2012, the World Health Assembly approved the Global Vaccine Action Plan (GVAP) to ensure that the full benefits of vaccines are extended to all people.4

Interventions to improve vaccination outcomes are commonly categorised either as targeting health services delivery or supply (eg, improving human resources training and supervision, logistics, cold chain maintenance and vaccine storage), or demand for vaccines. Common approaches to increase demand involve offering incentives for vaccination, or knowledge translation and education (KTE), to promote and sustain vaccine uptake.5 Mixed strategies combine features of supply and demand approaches.

Demand-side interventions may be particularly promising as equity-based strategies to reach underserved populations. A recent systematic review and meta-analysis by our group found that demand-side interventions led to substantial gains in child vaccination coverage in diverse developing country settings.6 KTE and incentives strategies were both effective.6

For children <12 months of age, India's Universal Immunisation Programme (UIP) now provides free vaccination against eight vaccine-preventable diseases: tuberculosis, diphtheria, tetanus, pertussis, polio, measles, hepatitis and rotavirus.7 Notwithstanding, in 2012, India accounted for the largest share among all countries (30%) of the world's 22.6 million under-vaccinated (defined as failure to receive three doses of diphtheria-tetanus-pertussis (DTP) vaccine) children, underscoring the need to strengthen uptake of routine immunisation (RI) in this context.8 Only one study has evaluated use of demand-side interventions to strengthen RI in India. Banerjee et al9 tested the use of food incentives to promote immunisation uptake in rural Rajasthan, and found an important positive effect. No published study has as yet evaluated use of KTE to promote vaccine uptake in India. KTE interventions have demonstrated considerable success in increasing vaccination coverage in similar contexts,6 10–14 and may be particularly important where levels of formal education are low.

Our research will make three key contributions to knowledge and practice concerning community-based interventions to increase vaccination coverage. (1) We are designing the first trial of KTE interventions in India. The proposed study location has especially weak health services and poor governance, as well as very low rates of education and health literacy.15 Immunisation coverage is low and there has been little progress over the last two decades. Through formative and evaluative research, we will develop and test a locally appropriate KTE model. (2) The trial will assess scientific issues of global significance related to KTE intervention delivery, including equity impact, cost-effectiveness, and financial and programmatic sustainability over time.6 (3) Through our non-governmental organisation (NGO) partners, it will contribute to development of a scalable implementation model for India.

Objectives and hypotheses

With the ultimate aim of conducting a future cluster randomised trial to assess intervention impact on vaccination coverage, we designed a pilot study to assess feasibility and aid in refining methods for the larger study.

Objectives of the planned future cluster randomised trial to study effectiveness (main study)

Purpose: To evaluate the impact, cost-effectiveness, scalability and sustainability of participatory, community-based KTE interventions to improve coverage of UIP-recommended vaccines among children <24 months of age from underserved communities in India.

Hypothesis: We postulated that participatory, community-based KTE interventions could increase vaccination coverage in these populations.

Primary objective

To evaluate the impact of the KTE interventions on vaccination coverage of children 12–23 months of age.

Secondary objectives

To evaluate the impact of the KTE interventions on equity of vaccination coverage. Analyses will study potential disparities in vaccination coverage among population subgroups reflecting differences in living standards, parental education and religion.

To evaluate the impact of the interventions on RI (as compared to campaign) coverage, overall and among equity strata.

To document the costs of offering the interventions and, if successful, assess the costs, effectiveness and cost-effectiveness of offering the interventions at a larger scale.

To document the process and delivery context to draw lessons for potential scale up and sustainability within the Indian health system.

Objectives of the current pilot study

We conducted a 4-month pilot study to inform development of the main study.16 Pilot study methods closely replicated those of the planned main study, including use of a cluster randomised and controlled design. A cluster design is required as interventions are structured around communities rather than individuals. The pilot study was not designed to determine effectiveness16 as the time period was too short to permit interventions to affect child vaccination status. We studied intervention effect on several proxy outcomes to evaluate proof of concept.

Primary objective

To assess the feasibility of processes key to success of the main study.16 These included ability to recruit the desired number of villages and participants per village, the acceptability of randomisation procedures and interventions, ability to deliver interventions as planned, the value of incorporating a control intervention, and participants’ understanding of intervention materials and data collection tools.

Secondary objectives

To study intervention impact on several proxy indicators of immunisation uptake at cluster and individual levels.

To identify shortcomings and potential barriers to success for the larger trial, and to take steps to allay them.16

We estimated the intracluster correlation for the main trial outcome using a larger, representative sample of 60 villages.15

Methods

Trial design

Consonant with the planned main study, this pilot employed a cluster-randomised design with a 1:1 allocation ratio. Clusters were rural villages of 2000 to 5000 inhabitants in Bawan Block, an administrative division of Hardoi district, Uttar Pradesh (UP). Villages were randomly assigned to either the intervention or control group (CG). A survey to assess knowledge, understanding and practices concerning immunisation, diarrhoea and child health was administered to participants at baseline prior to randomisation, and 4 months later at endline.

Participants

Setting and location

The trial location is rural Hardoi district, UP, India. UP is India's most populous state with more than 200 million inhabitants and the largest share (28% in 2012) among all states, of India's child mortality.17 UP is also home to India's greatest number of unimmunised children.18 With a population of 4 million subdivided into 5 Tehsils (19 blocks),19 Hardoi figures among 81 (of 640) districts accounting for 1/3 of India’s 2012 child mortality,17 and receives development funds targeted to India's most backwards districts. In Hardoi district, estimated mortality in children under 5 years of age was 89.6 per 1000 (UP 74.9; India 57.3) in 2012,20 coverage of DTP3 (a standard measure of RI system performance used by international agencies such as WHO, UNICEF and GAVI21–23) was 41.9%15 (UP 55.9%;20 India 71.5%18) and full immunisation was 49.9% (UP 45.3%; India 61.0%).18 Bawan is one of Hardoi's 19 administrative blocks. Bawan was chosen as the pilot study site for reasons related to logistics and feasibility, and because it consistently performs below district averages on development indicators.

Eligibility criteria for clusters

Villages were eligible for inclusion if they had 2000 to 5000 inhabitants and were located in Bawan Block. Census data showed that 23 of Bawan's 126 villages were candidates for inclusion. We eliminated 1 village with which we had had previous contact. Twelve of the remaining 22 villages were selected to ensure maximum geographical distance between clusters.

Eligibility criteria for participants

All women residing in a selected village, who were mothers of a child 0–2 years of age, were eligible to participate. We excluded those not able to understand and speak Hindi or Urdu, and those cognitively impaired or who did not intend to reside in the village for the study duration (4 months). Eligibility criteria applied only to scientific data collection and inclusion in the main analysis.

The sampling unit was the household. We selected 36 households containing one or more eligible mothers within each village, using sampling procedures designed to provide near random selection of households spread over the community.24

Interventions

Interventions employed a variety of individual, small group and community-wide approaches that can eventually be delivered by volunteers. The target populations reside in rural areas with weak health services. Our focus on community-based KTE is grounded in two findings from our 2013 district survey: (1) social factors structuring access, opportunities and empowerment also shape immunisation coverage. (2) Mothers are considered the primary caregivers for their children but are often not empowered to fulfil this role.15 Study interventions contribute to learning, empowerment and forging of links across social groups, thereby reinforcing the resilience of individuals and communities, and creating conditions for positive change. Interventions aim to increase the effectiveness of local immunisation services without substantial added public investment, and are designed as a support to local health workers. Interventions took place over 4 months and will be standardised in future to facilitate delivery at scale.

India's Integrated Child Development Services (ICDS) scheme offers nationwide nutrition and health promotion. In addition to the regular ICDS services, women residing in intervention group (IG) villages received: (1) six home visits by volunteers (one visit every 3 weeks) using ‘engagement packages’ designed to improve knowledge, awareness and attitudes towards immunisation. Engagement packages consist of visually appealing story cards, discussions, games and activities, delivered one-on-one or in small groups with friends, fathers and other family members. Together, these activities help to cultivate confidence, raise awareness and create a space wherein the mother's capacity to make decisions on immunisation is recognised and supported; (2) three community mobilisation events involving activities, theatre and discussion groups to identify problems related to immunisation in their communities, discuss possible causes and solutions, and give feedback on the project. CG villages received community mobilisation through activities, events and discussion groups to raise awareness on issues related to nutrition, diarrhoea prevention and treatment in their communities, and give feedback on the project. A toll-free number for immunisation enabling anonymous queries and feedback was offered to all residents of IG and CG villages. Table 1 summarises study interventions.

Table 1

Study interventions

Activity	Intervention group (IG)†	Control group (CG)
Baseline survey*
Randomisation*
Home visit 0	Rapport formation; information	Rapport formation; information
Community discussion 1	Project introduction; sharing district report card; skit (vaccination); toll-free number (vaccination)	Project introduction; sharing district report card; skit (ORS); toll-free number (vaccination)
Home visit 1	Story 1 (Key messages) Timely vaccination can save children from life-threatening illnesses (polio) In case of queries, one should ask a local CHW (or use the toll free number) Activity: immunisation schedule
Home visit 2	Story 2 (New messages) Vaccination is free of charge and available at the local Anganwadi centre Activity: protect immunisation card
Home visit 3	Story 3 (New messages) Every vaccine has a specific purpose and all doses are required Challenges occur in daily life, but one should avoid missing a vaccination dose Even if the child has a minor illness, she or he can be vaccinated Activity: immunisation schedule
Community discussion 2	Discussion of barriers to immunisation and local solutions; toll-free number (vaccination); activities: immunisation calendar; immunisation card; skit (vaccination)	Discussion of barriers to early child nutrition and local solutions; toll-free number (vaccination); activities: skit
Home visit 4	Story 4 (New messages) Place the right priority on vaccination—if one's child falls ill, one will incur expense and health risk Activity: immunisation card
Home visit 5	Story 5 (New messages) In case of service delivery problems, tell a responsible person (or use the toll-free number) If a child's vaccine dose is missed, obtain it at the next opportunity Activity: immunisation card
Home visit 6	Story 6 (New messages) Vaccination benefits everyone (herd immunity) Activity: immunisation schedule
Community discussion 3	Feedback on project components (home visits, community discussions, toll-free number); suggestions for improvement	Feedback on project components (home visits, community discussions, toll-free number); suggestions for improvement
Endline survey*

*Identical for IG and CG.

†Key messages were repeated for reinforcement; only new content is described.

CHW, community health worker; ORS, oral rehydration salts.

Interventions addressed individuals, households and communities. While home visits were directed in the first instance to the mothers of young children enrolled in the study, activities were open to friends, neighbours and other members of the household. Community events were open to all.

Outcomes

Primary outcomes

We established ex-ante criteria for feasibility of the main study related to processes for participant recruitment, randomisation and retention. Specifically, we viewed the study as feasible if (1) 50% of villages approached would agree to participate and accept randomisation; (2) 30 women per village could be recruited in 70% of villages; (3) 50% of women were retained from baseline to endline. Below these thresholds, we judged that the study would not be feasible without major modifications.

Secondary outcomes

We studied process indicators related to implementation fidelity (ability to delivery interventions as planned) and two indicators of community response ((1) participation in community events; (2) additional participants who joined the endline survey (as a measure of the indirect effect of interventions)). We also studied three proxy indicators of intervention impact on immunisation uptake. We described differences between IG and CG villages for two cluster-level outcomes: (1) use of the toll-free number and (2) monthly immunisation day footfall. (3) We compared performance of individuals belonging to the IG versus CG on the change from baseline to endline survey on key indicators related to information conveyed through the engagement packages and community events. There were no changes to outcomes after the start of the trial.

Sample size

For a feasibility study, sample size is not established based on power to detect an anticipated intervention impact. We set the number of clusters at 12, as six clusters per study arm is a minimum for cluster comparisons using a t test,25 and for logistical and budgetary reasons. To facilitate statistical testing within clusters, we fixed the number of households per village to be 30, but inflated this to 36 to account for potential non-response and missing values. We therefore sought to recruit 432 individual participants allocated equally between intervention and control villages. In all 12 villages, community discussions and the toll-free number were open to the entire community.

Randomisation

Sequence generation and allocation concealment

Villages were assigned either to intervention or CGs using simple randomisation with a 1:1 allocation following a computer-generated randomisation schedule. The random allocation sequence was generated at the CRCHUM by a professional statistician (M-PS) using the R package blockrand26 and kept in a password-protected computer. The statistician was not involved in study implementation. Prior to release of the randomisation code, only the statistician had access to the allocation sequence. The randomisation code was released all at once, and treatment groups assigned only after completion of all recruitment procedures and baseline measurements.

Implementation

Field team leaders enrolled clusters by contacting village officials in person to explain study aims and activities, and to request consent to participate. Subsequently, in each participating village, field team members directly approached a random sample of households to request their consent to participate in the baseline survey and pilot study. Enrolment occurred prior to randomisation. No advertisements were used for recruitment, and no incentives or rewards were offered for participation. Field team members informed households personally of their study group assignments.

Blinding

Owing to the nature of the interventions, neither participants nor those involved in intervention delivery were blinded to group assignment. Data analysis was not masked. We took two measures to reduce the potential impact of knowledge of group assignment on study outcomes. (1) The study used a control intervention to enhance acceptability of randomisation and to conceal the true study hypothesis. (2) We hired independent surveyors for the endline survey to assess study outcomes. These surveyors were not informed of group assignment. They were told that our goal was to understand how mother's knowledge affects child health. Baseline and endline questionnaires encompassed immunisation, diarrhoea and other aspects of child health.

Statistical methods

We used descriptive statistics (counts, frequencies, proportions) to assess study feasibility and processes. Descriptive statistics were also used to compare the IG and CG on cluster-level outcomes. We compared the IG and CG on individual-level outcomes reflecting change from baseline to endline survey on selected indicators using generalised estimating equations (GEE) regressions adjusted for village-level clustering using an exchangeable correlation structure. Additional analyses considered GEE models adjusted simultaneously for village-level clustering, maternal education and wealth quintile. These analyses included participants with complete data; complementary analyses explored results for participants with incomplete data.

The study will be prospectively registered in an international trial registry before starting the main trial.

Results

Primary outcomes

Village recruitment and the baseline survey occurred between 12 January and 3 February 2014. Four villages were replaced prior to randomisation, one due to failure to meet inclusion criteria, and three due to surveyor error during the baseline survey. A surveyor administered components of the baseline survey in the wrong order. Randomisation assignments were released to the core study team on 4 February 2014 and communicated in person to all 12 villages on 7 and 8 February 2014. The intervention began immediately thereafter and ran for 4 months as planned. The endline survey was initiated on May 27th and completed by 30 June 2014.

Feasibility criteria related to recruitment, randomisation and retention were satisfied (figure 1). All villages contacted (100%, 16/16) agreed to participate and accepted randomisation. Thirty-six women were recruited in 100% (12/12) of villages randomised. No clusters were lost to follow-up. The main analysis included 86% (373/432) of participants, 90% (195/216) from the IG and 82% (178/216) from the CG.

Figure 1

Flow diagram of progress of clusters and individuals through the study.

Baseline data

Table 2 presents characteristics of the study sample. There were clear baseline imbalances between intervention and CGs, with the CG having higher living standards, maternal education and proportions of children vaccinated. The 12 study villages had better access to electricity and health services, and somewhat higher living standards than Hardoi district as a whole. Notwithstanding, the IG had lower proportions of children vaccinated as compared to the CG and to district averages.

Table 2

Characteristics of 12 villages and 432 households, mothers and children included in the pilot study

	Characteristics of participating villages						District*
	Controln=6		Interventionn=6		TotalN=12		N=60
	n	(%)	n	(%)	n	(%)	n	(%)
Village electrification
None	0	(0)	0	(0)	0	(0)	8	(13)
Less than 6 h	0	(0)	2	(33)	2	(17)	30	(50)
More than 6 h	6	(100)	4	(67)	10	(83)	22	(37)
Number of health centres
0	0	(0)	0	(0)	0	(0)	2	(3)
1	3	(50)	5	(83)	8	(67)	25	(42)
2	2	(33)	1	(17)	3	(25)	16	(27)
3	0	(0)	0	(0)	0	(0)	13	(22)
4	1	(17)	0	(0)	1	(8)	4	(7)
Proportion in poorest 20%†
Mean (SD) in %	14 (1)		26 (10)		20(11)		24	(15)

*Data for Hardoi district are from a representative sample of 60 villages (1192 households) collected in 2013 and included for purposes of comparison.15

†This is the proportion in the poorest wealth quintile. Wealth quintiles for this 12 village pilot study were calculated using items and methods similar to those for India's major national surveys.15

‡Dwelling characteristics were assessed by observation using definitions drawn from India's major national surveys.15

§‘Full immunisation’ among children 12–23 months is defined as 1 dose of BCG vaccine, 3 doses of polio vaccine, 3 doses of diphtheria-pertussis-tetanus (DPT) vaccine, and 1 dose of measles vaccine.

Secondary outcomes

Intervention fidelity: Of the six planned immunisation KTE home visits, an average of 5.9 visits were delivered to the 216 IG participants. Three rounds of community discussions were held as planned in all 12 villages.

Community response: (1) Community discussions—community discussion 1 included 1210 adult participants (752 from IG villages and 458 from CG villages). Community discussion 2 involved 1140 participants (593 IG and 547 CG). Community discussion 3 involved 946 participants (604 IG and 342 CG) (see online supplementary table S1). (2) Additional participants in the endline survey: 139 additional community members (96 (69%) IG; 43 (31%) CG) volunteered to take the endline survey (figure 1).

Toll-free number

IG and CG villages differed in use of the toll-free number, with more calls originating from IG villages (n=11) versus CG villages (n=3). Monthly call volume was lower than anticipated; there were 14 calls from the 12 villages over 4 months.

Immunisation day footfall

Inspection of monthly immunisation day footfall from April 2013 to May 2014 revealed no clear differences between IG and CG villages (see online supplementary figure S1). Quality of administrative data was poor. One of six IG villages and one of six CG villages had no immunisation records. Four months of data were missing for the IG including 2 months during the intervention period. Two months of data were missing for the CG, but none in the intervention period. Records were incomplete and often related to only one immunisation centre rather than to the entire village. The precise number of children eligible for vaccination in the catchment area was not known in any village.

Analysis of individual responses to baseline and endline surveys

Table 3 describes proportions of correct responses given by the 373 mothers who completed both baseline and endline surveys for selected indicators of knowledge and understanding about vaccination. Results were heterogeneous; regression analyses modelled the change from baseline to endline to clarify patterns of variation (table 4, online supplementary table S2). General knowledge about vaccination improved in the IG (OR 2.38 (95% CI 1.60 to 3.58), p value <0.001) but not in the CG (OR 1.29 (95% CI 0.56 to 2.99), p value 0.545). Ability to interpret the child's immunisation card did not improve in either study group. Knowledge of the vaccination calendar increased markedly in the IG but not in the CG, for all doses except the birth dose (table 4). For these indicators, ORs for the intervention effect ranged from 7.55 to 18.25; adjustment for maternal education and wealth quintile resulted in larger IG effect sizes (table 4). Maternal education (ever having attended school) confounded the intervention effect in several analyses. Wealth quintile was not an important confounder in any analysis (table 4, online supplementary table S2).

Table 3

Proportions of correct responses given by 373 participating mothers on selected indicators of knowledge and understanding about vaccination, by study group (n (%))

Indicator	Control (N=178)		Intervention (N=195)
	Baselinen=178	Endlinen=178	Baselinen=195	Endlinen=195
General knowledge of vaccination*
Vaccination protects against how many diseases?	102 (57)	113 (63)	90 (46)	131 (67)
Questions related to the immunisation card†
What is the child's date of birth?	42 (23)	47 (27)	32 (16)	36 (18)
Why does the immunisation card have multiple boxes?	114 (64)	72 (40)	85 (43)	86 (44)
How many doses has the child received?	31 (17)	27 (15)	30 (15)	16 (8)
Which vaccine has the child received?	34 (19)	30 (17)	26 (13)	21 (11)
Questions related to the immunisation schedule‡§
Do you know the immunisation schedule of your child? (yes)	20 (11)	111 (62)	21 (11)	182 (93)
At birth	11 (6)	93 (52)	15 (8)	170 (87)
Dose at 1.5 months	4 (2)	31 (17)	7 (4)	148 (76)
Dose at 2.5 months	4 (2)	22 (12)	5 (3)	156 (80)
Dose at 3.5 months	5 (3)	15 (8)	3 (2)	148 (76)
Dose at 9 months	11 (6)	47 (26)	12 (6)	172 (88)

*Surveyors informed participants of the correct answer directly prior to asking the question.15

†A sample filled vaccination card was used to elicit responses to these four questions.15

‡These are spontaneous responses to the question ‘Do you know the immunisation schedule of your child? If yes, please name all the doses’.15

§We recorded as correct all responses recommended in the Indian immunisation schedule; however, this table presents only doses for children <12 months of age.

Table 4

Univariate regression models describing knowledge of the vaccination schedule at endline versus baseline for 373 participating mothers, by dose and study group

Vaccine dose*	Model 1†			Model 2‡
	OR	(95% CI)	p Value	OR	(95% CI)	p Value
Birth
IG	5.23	(0.57 to 48.29)	0.145	5.52	(0.53 to 57.38)	0.152
CG	4.17	(1.51 to 11.57)	0.006	4.34	(1.62 to 11.61)	0.003
1.5 months
IG	7.55	(3.25 to 17.29)	0.000	10.01	(3.51 to 28.58)	0.000
CG	1.45	(0.49 to 4.37)	0.500	1.35	(0.42 to 4.41)	0.614
2.5 months
IG	14.11	(5.90 to 33.78)	0.000	18.56	(8.08 to 42.65)	0.000
CG	0.76	(0.27 to 2.11)	0.598	0.70	(0.21 to 2.29)	0.558
3.5 months
IG	18.25	(7.43 to 44.83)	0.000	25.04	(8.79 to 71.3)	0.000
CG	0.43	(0.13 to 1.44)	0.170	0.38	(0.08 to 1.73)	0.209
9 months
IG	11.30	(3.42 to 37.31)	0.000	15.37	(5.12 to 46.17)	0.000
CG	0.60	(0.34 to 1.05)	0.073	0.53	(0.27 to 1.03)	0.061

*These are spontaneous responses to the question ‘Do you know the immunisation schedule of your child? If yes, please name all the doses’.

†Models adjusted for village-level clustering.

‡Models adjusted for village-level clustering, mother's education (mother attended school yes/no) and wealth quintile.

CG, control group; IG, intervention group.

Analysis of the 59 mothers lost to follow-up revealed no significant differences with respect to the 373 mothers remaining in the study in terms of household characteristics, personal characteristics or baseline survey responses (see online supplementary tables S3–5). The proportion of correct responses was generally higher among the 139 additional community participants who volunteered to take the endline survey, as compared to the 373 participating mothers. The child's father had the highest proportion of correct responses among all categories of participants (see online supplementary tables S6–8 and figure S2).

Discussion

Key findings from the pilot study

We highlight six findings from the pilot study: (1) pre-established criteria for feasibility of the planned main study related to recruitment, randomisation and retention were fully satisfied for clusters and individuals. (2) Intervention fidelity was excellent. (3) Community participation was high but weaker in the CG. (4) As hypothesised, use of a toll-free number was higher in the IG than in the CG. Notwithstanding, call volumes were low overall. (5) No clear conclusions could be drawn concerning trends in immunisation centre footfall. (6) Multiple analyses revealed a very strong effect of the immunisation intervention on individual participant learning, but results were variable across indicators.

Methodological lessons for the planned main study

Reflection on pilot findings and experiences suggests several lessons for the future study.

Study groups were unbalanced on characteristics likely to be related to study participation and child vaccination status, with IG participants systematically disadvantaged as compared to controls. (i) During the pilot, we adopted two strategies to enhance community partnership, and to ensure participation of households and individuals. First, we learned to cultivate support from influential members of the community and to seek their help in motivating others to join the study. Second, we diversified our field teams to include under-represented groups. We view these strategies as essential to successful recruitment, randomisation and retention. (ii) Randomisation failed to achieve balance between study arms due to the small number of clusters involved in this pilot study. The future study should consider design options to improve balance, and to increase study power and precision, such as inclusion of a larger number of clusters, stratification25 and restricted randomisation using balancing criteria.27 (iii) Substantively, pilot results show that the intervention was successful on many measures in a very disadvantaged population.

Although intervention fidelity was excellent, the value of a parallel control intervention was unclear. Through discussion with our study teams, we learned that active presence of field staff in control villages led to opportunities for contamination as villagers requested field team members to share information and advice on child health themes raised by the baseline survey. All content for the main study should likely be delivered through an intervention arm.

The first round of community discussions was delivered as one large gathering per village. In the CG, the first community discussion coincided with local religious festivals in two of the six control villages, while in one control village, people were reluctant to meet together due to caste and class divisions. The second and third community discussions were delivered in rounds of three to six smaller meetings per village, and groups were more homogeneous with respect to age, sex and status. This strategy was more effective.

Use of a toll-free number was lower than anticipated. To our surprise, in endline discussions with the community we learned that the toll-free service was extremely valued. However, active presence of our field teams meant that questions were addressed preferentially to them. Toll-free and KTE intervention components should not be evaluated together.

Administrative data on immunisation were inadequate for scientific use. The Government of India is investing to improve vital statistics using biometric and digital technologies. Until this process reaches maturity, the future main study should consider undertaking a census of participating villages at baseline to determine target population denominators.

Analysis of individual participant survey responses showed a compelling effect for some but not all components of the main vaccine intervention on proxy outcomes. (i) These results provide an evidence base to improve aspects of the main intervention prior to fielding the larger study. Different communication methods were used for the different components. (ii) Results also encouraged us to consider the adequacy of our questionnaires. We used a previously developed survey with only marginal modifications.15 Questions were only distantly related to the study interventions to ensure that both study groups had a reasonable chance to reply correctly. Questions related to the immunisation card may have been difficult to interpret. The main end point of the future study will be immunisation status rather than knowledge or health literacy, avoiding some of these difficulties. (iii) The intervention had an indirect effect. In our sample, community participants had higher scores than the mothers who were the target participants, with fathers having the highest scores on average. This likely reflects how gender biases shape ability to learn over the life course in this context. The community sample was not randomly selected.

Conclusions

This pilot study provided rich lessons to inform design of a future trial and to refine interventions. Criteria related to feasibility of the main study were satisfied. Based on our extensive knowledge of the district,15 28 we believe that evidence of feasibility can be generalised to other settings in Hardoi, UP. In addition, cluster-randomised trials of participatory learning interventions have been conducted in similar locations,29 30 lending support to our assessment that the study is feasible.

This pilot study is part of a series of targeted research efforts encompassing formative and evaluative dimensions. Prior to the main trial, we will continue to refine our interventions to ensure that they are effective, responsive to policy context, scalable and sustainable. In December 2014, the Government of India announced a new strategy for periodic intensification of RI in 201 high priority districts including Hardoi.31 Week-long immunisation sessions will be offered four times per year to improve coverage levels. Our NGO partner, Pratham Education Foundation, has achieved considerable success in improving child education in India often through literacy campaigns.32 33 Pratham has been able to sustain large cadres of community volunteers through non-financial rewards enabling low cost operation at national scale. With their guidance, we may simplify and streamline our interventions to be deliverable in campaign mode.

Lessons from this pilot study give us confidence that we can successfully design and conduct a cluster randomised trial to assess the effectiveness of KTE interventions to improve vaccination coverage among children <24 months of age in rural northern India.