Does exposure to health information through mobile phones increase immunisation knowledge, completeness and timeliness in rural India?

INTRODUCTION: Immunisation plays a vital role in reducing child mortality and morbidity against preventable diseases. As part of a randomised controlled trial in rural Madhya Pradesh, India to assess the impact of Kilkari, a maternal messaging programme, we explored determinants of parental immunisation knowledge and immunisation practice (completeness and timeliness) for children 0-12 months of age from four districts in Madhya Pradesh.
METHODS: Data were drawn from a cross-sectional survey of women (n=4423) with access to a mobile phone and their spouses (n=3781). Parental knowledge about immunisation and their child's receipt of vaccines, including timeliness and completeness, was assessed using self-reports and vaccination cards. Ordered logistic regressions were used to analyse the factors associated with parental immunisation knowledge. A Heckman two-stage probit model was used to analyse completeness and timeliness of immunisation after correcting for selection bias from being able to produce the immunisation card.
RESULTS: One-third (33%) of women and men knew the timing for the start of vaccinations, diseases linked to immunisations and the benefits of Vitamin-A. Less than half of children had received the basic package of 8 vaccines (47%) and the comprehensive package of 19 vaccines (44%). Wealth was the most significant determinant of men's knowledge and of the child receiving complete and timely immunisation for both basic and comprehensive packages. Exposure to Kilkari content on immunisation was significantly associated with an increase in men's knowledge (but not women's) about child immunisation (OR: 1.23, 95% CI 1.02 to1.48) and an increase in the timeliness of the child receiving vaccination at birth (Probit coefficient: 0.08, 95% CI 0.08 to 0.24).
CONCLUSION: Gaps in complete and timely immunisation for infants persist in rural India. Mobile messaging programmes, supported by mass media messages, may provide one important source for bolstering awareness, uptake and timeliness of immunisation services. TRIAL REGISTRATION NUMBER: NCT03576157. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Sending mobile messages to caregivers to remind them of vaccination dates have emerged as a cost-effective way to improve parental awareness and immunisation coverage in low-income and middle-income countries, though there is mixed evidence on its impact on vaccination coverage and timeliness.

Exposure to Kilkari mobile messages on immunisation was positively associated with father’s knowledge and receiving timely vaccination at birth.

Wealth was the most significant determinant of men’s knowledge and of the child receiving complete and timely immunisation for both basic and comprehensive packages.

Mobile messaging programmes, along with other mass communication methods, may provide one important source for bolstering information about, and the uptake of, immunisation services.

Introduction

Immunisation is one of the most efficient and effective public health interventions to reduce child mortality and morbidity.1 Every year, an estimated 2–3 million lives are saved by immunisation.2 However, close to 20 million children under 1 year of age do not receive basic vaccinations. Incomplete vaccination, late vaccination and ‘zero dose’ children are concentrated in conflict-affected regions, low-income countries and socially marginalised populations in middle-income countries.2

In India, the percentage of fully immunised children 12–23 months of age increased from 44% in 2005–2006 to 62% in 2015–2016.3 However, despite this increase, inequities in coverage persist across states and by sociodemographic characteristics, including wealth and education.3 Regionally, coverage of critical vaccines, including Diphtheria Pertussis and Tetanus (DPT) and measles, among children 12–23 months old is much lower in India (87% each for DPT and measles) compared with neighbouring countries of Bangladesh (96% for DPT and 97% for measles) and Sri Lanka (99% for both).4

While timely and complete childhood immunisation is closely linked to health service provision, parental knowledge and practice also plays a role. Parents’ level of knowledge about vaccination needs, time availability, travel capacity to bring the child to vaccination sites and attitudes towards vaccination are key influencers on childhood vaccination. Positive association with full immunisation5 has been found with children whose parents had higher immunisation knowledge in many settings.6–8 Higher parental knowledge about child immunisation is associated with higher parental age, more years of education, being employed, having a greater number of children and higher household income.9–11 Full immunisation has also been found to be more prevalent among children whose parents owned mobile phones, after controlling for other socioeconomic factors in Mozambique.12 While India has one of the highest rates of vaccine confidence in the world, with over 80% of the population considering vaccines important, safe and effective,13 14 low awareness and poor access have been identified as drivers of some hesitancy.15

Mobile health (mHealth) interventions have been used to provide health information on vaccines, as well as send alerts and reminders to beneficiaries to improve immunisation timeliness.16 Programmes in Zimbabwe,17 Kenya,18 Nigeria19 and Bangladesh,20 which have used vaccination appointment reminders and immunisation-related prerecorded messages, have been found to positively influence immunisation uptake. But, in rural Haryana, India, a text message service, which sent reminders to care-givers and used Global Positioning Satellites to track children who had missed appointments was found to have no significant impact on full and timely vaccination of children.21 However, when these services were combined with a financial incentive, the intervention showed a positive impact.21 These mixed findings highlight the need for further evaluations of mHealth services in India, particularly, low cost direct-to-beneficiary mHealth programmes like Kilkari.

Kilkari is India and the world’s largest direct-to-beneficiary mobile communication programmes. Since its inception in 2012, Kilkari has reached over 10 million new and expectant mothers and their families with up to 72 weekly stage-based audio health information messages. Kilkari calls span from the 12th week of pregnancy up until the child’s first birthday and includes content on a range of health areas: family planning, child immunisation, nutrition, infant feeding, pregnancy care, entitlements and postnatal care. Immunisation messages are provided over 11 calls, which focus on the benefits of vaccination and diseases prevented by it, along with a reminder to the parents to take the child for next vaccination (online supplemental Annexure-1). In this study, we aim to assess the determinants of parental knowledge of immunisation, full immunisation and timely immunisation in the context of the Kilkari programme. We draw from cross-sectional survey data captured as part of an individually randomised controlled trial (RCT) of the Kilkari programme in four districts of Madhya Pradesh (MP), India.22

Methods

Study setting and context

The study was conducted in rural MP, a state in central India with one of the lowest per capita state domestic products.23 There remains a significant gender gap in women’s literacy and access to mobile phones within and across MP and throughout much of India. Findings from the 2015–2016 National Family Health Survey (NFHS) suggest that across MP, literacy rates were 59% and 82% for women and men, respectively.24 The gender gap in women versus household access to mobile phones is estimated to be 60% in rural areas and 44% in urban areas in 2015.24 MP falls below national averages for most health indicators, including immunisations. Among children 12–23 months of age, 54% were fully immunised (Bacille Calmette-Guerin (BCG), measles and three doses each of polio and DPT) in MP as compared with 62% nationally.24

Study findings draw from data captured as part of the RCT in four districts (Rewa, Hoshangabad, Mandsaur and Rajgarh) of MP, described in detail elsewhere.22 In brief, women 4–7 months pregnant (5095) with access to mobile phones were randomised to either a comparison arm of no calls or an intervention arm where they were eligible to receive Kilkari calls from the 18th week of pregnancy up until the child’s first birthday. The RCT assessed the impact of Kilkari on various reproductive maternal newborn and child health (RMNCH) behaviours including full and timely immunisations among children 0–12 months of age.22

Data collection

The present analysis used the postpartum survey data from 4423 postpartum women (87%, out of 5095 total enrolled in Kilkari) with a live birth during the index pregnancy and their husbands (3781 men). Data could not be collected from 642 husbands primarily due to migration for work. The interviews were conducted face-to-face in Hindi and included questions on socioeconomic and demographic factors, access to mobile phones and parental knowledge about immunisation. Information on what vaccines the child received from birth to the time of postpartum survey (approximately 12 months after birth) was recorded from the government-issued vaccination card called the Mother and Child Protection (MCP) card.

After adjusting for twins and number of children born since the baseline, information was available for 3891 children, for whom interviews of both the parents could be conducted. Out of these 3891 children, the MCP card was available for 3230 children. Posthoc power analysis indicated that the final sample of 3230 children was powered to detect a change of up to 4% in immunisation coverage for children 0–12 months old, with a power of 80% and a type-I error of 0.05.

Outcome variables

Parental knowledge on immunisation

During the postpartum survey, women (mothers) and their husbands (fathers) were asked 11 questions to assess their knowledge about childhood immunisation. Questions covered timing of first vaccination, diseases against which a child needs to be vaccinated within the first year of their birth and benefits of Vitamin-A. A score of 1 was assigned for each correct response and 0 otherwise, with equal weights assigned to each response, resulting in a composite knowledge score for each respondent ranging from 0 to 11.

Child’s immunisation status

Full immunisation (comprehensive) was if the child received all 19 vaccines listed in online supplemental Annexure-2 (following the guidelines of India’s National Immunisation Guideline25), to which a score of 1 was assigned, else they were assigned 0.

Full immunisation (basic) was if the child received all eight basic vaccines: BCG, three doses of DPT, three doses of Polio and measles, to which a score of 1 was assigned, else they were assigned 0 (following the NFHS24).

Timely immunisation (comprehensive) was assessed by comparing the expected vaccination date to the actual vaccination date. Each vaccine’s recommended vaccination date was estimated by calculating when the vaccine should have been administered based on the child’s date of birth as listed in their MCP card and according to Government of India vaccine schedule guidelines.25 The actual vaccination date was recorded from the child’s MCP card. For each vaccine, we assigned 1 if it was administered within 28 days of the date the vaccine becomes due and a 0 if not. Each child thus had a total timely immunisation score that varied from 0 to 19.

Timely immunisation (basic) was defined as the administration of BCG, three doses of DPT, three doses of polio and the measles vaccine at the 6–10 weeks, 10–14 weeks, 14–18 weeks and 9–10 months, respectively. For each vaccine, a score of 1 was assigned if it was administered within time (as described for the comprehensive package above), else 0, with a total score varied from 0 to 8.

Data analysis

Frequencies and proportions were used to understand the characteristics of the sample, exposure to Kilkari immunisation calls, parental knowledge on immunisation and immunisation status among the children.

Immunisation knowledge

We used ordered logistic regression, separately for mothers and fathers, to assess the factors associated with parental knowledge of immunisation. To assess the correlates of parent’s immunisation knowledge, the following model specification was used:

I-KNOW=f(KILKARI, SES, M-INDIV, M-ACCESS, IMMU-INFO, DIST) (1),

where I-KNOW is the immunisation knowledge score, on a scale varying from 0 to 11.

KILKARI is a binary variable, where 1 indicates if the cumulative time of content listening under the Kilkari programme was ≥50% (at least 441 econd of total 881 second of cumulative content on immunisation were played), 0 otherwise. The information was collected from the call records data.

SES is a vector of socioeconomic characteristics of the mother which includes dummies for caste (other backward class (OBC), scheduled caste/scheduled tribe (SC/ST) and others) and wealth quintiles (computed by principal component analysis based on various assets ownership and availability of amenities in household). M-INDIV represents a mother’s total years of completed formal education, number of children she has, dummy variables to represent her employment status (1 if employed), and if she plays any role in taking decision of daily purchases of the household and her pregnancy. A mother’s mobile phone access is proxied by M-ACCESS which comprises dummies representing whether she owns a personal mobile phone, if she has access to the mobile phone for more than 12 hours a day and if the phone is often at zero balance.

Sources of immunisation information are represented by the vector IMMU-INFO which includes dummies on whether the mother has asked for immunisation information from any health worker, whether she has received the same from any health worker and if she reported to receive information from (i) television or (ii) print (poster/newspaper/handout). Finally, vector DIST represents district dummies.

For the analysis of factors associated with father’s immunisation knowledge, the explanatory variables remain the same as equation (1) (KILKARI, SES, father’s individual characteristics, father’s access to mobile phone and district dummies) except the vector on sources of immunisation information (asked and received information from health worker, received information from television or print media), which was not collected and is not part of the analysis.

Immunisation practice

Out of the total sample of 3891 children, the MCP card was available for 3230 (83% of sample) children. There could be systemic differences between mothers who could produce the MCP card and those who could not. To address this, we applied Heckman two-stage sample selection model to correct for selection bias.26 In the first step, the model estimates the probability of having the MCP card, which can be expressed as:

Prob (MCP=1 | Z)=f(Z (KILKARI, SES, M-INDIV, M-ACCESS, IMMU-INFO, ASHA, DIST)) (2),

where MCP=1 if the mother could produce the MCP card, 0 otherwise; Z is a vector of explanatory variables representing a mother’s characteristics explained in equation (1) and the caste of ASHA, the frontline worker playing a crucial role in RMNCH related issues, as a proxy to capture the capability of her.

In the second stage, after incorporating the predicted probability of having the MCP card as an explanatory variable, the model to predict the practice of immunisation can be represented as:

P-IMMU* = β*X + ε (3),

where P-IMMU, probability of child to be immunised, is not observed if the mother could not produce the MCP card. The conditional expectation of immunisation practice can be expressed as:

E(P-IMMU | X, MCP=1) = β*X+E(ε|X, MCP=1) (4),

where X is a vector of characteristics expressed in equation (1), additionally including (i) parental immunisation knowledge scores, (ii) child’s gender (to assess any gender-based bias) and (iii) if the child received most vaccines at the Anganwadi centre, the nutrition and childcare centre at the village and a common place for immunisation.

The analysis of determinants of full immunisation and timely immunisation of basic and comprehensive package, vaccination at birth, at 6–10 weeks, 10–14 weeks, 14–18 weeks and 9–10 months were done using Heckman probit model.

Ethical clearance

Ethical clearance for this study was obtained from the Independent Ethics Review Boards of Johns Hopkins School of Public Health Institution in Baltimore, Maryland and Sigma Research in India. Verbal informed consent was obtained from the study participants, for participation in the RCT and in the baseline and postpartum survey.

Results

Study population

Table 1 outlines the characteristics of the women and men included within the study population. Women had completed an average of 7.7 years (SD=4.0) of formal education and men had completed 8.8 years on average (SD=3.9). While 33% of the women reported that they asked health workers for immunisation information, 89% of total women reported receiving the same from health workers. Thirty-two per cent and 24% of the women respondents reported receiving immunisation information from television and print sources, respectively.

Table 1

Percentage distribution of key characteristics of mothers and fathers

Variable	Mothers	Fathers
Sample subscribed to Kilkari
Randomised to receive Kilkari calls (sample size)	2348	1998
Exposure to Kilkari (% of Kilkari subscribers listened to ≥50% of cumulative total of Kilkari immunisation calls)	20.1	19.3
Full sample
Sample size	4423	3781
Caste
Others	22.4	22.6
Other backward class	47.1	48.6
Scheduled caste/scheduled tribe	30.5	28.8
Wealth quintiles
Poorest	19.1	16.3
Poor	19.6	18.7
Middle	20.0	20.4
Rich	20.6	21.6
Richest	20.7	23.0
Completed years of formal education (Mean)*	7.7 (4.0)	8.8 (3.9)
Employed	36.3	99.3
Role in decision making
In daily purchase	21.3	49.1
In pregnancy	20.4
Number of children (Mean)*	1.0 (1.1)	1.0 (1.1)
Mobile phone ownership and access
Owns personal mobile	63.8	94.8
Has access to mobile for >12 hours a day	79.4	89.5
Phone often with no balance	9.5	67.1
Other sources of immunisation information
Asked health worker	33.2	–
Received information from health worker	88.7	–
Television	31.9	–
Newspaper/poster/handouts	24.3	–
District
Mandsaur	16.7	19.3
Hoshangabad	8.3	9.6
Rewa	56.1	48.5
Rajgarh	18.9	22.8

*SD in parentheses.

Exposure to Kilkari immunisation messages

There were 11 Kilkari calls on immunisations spanning from the 18th week of pregnancy up until the child’s 1st birthday (online supplemental Annexure-1). Figure 1 depicts the week-wise proportion of immunisation calls answered by Kilkari subscribers (light blue bars) randomised to the intervention arm and the average percentage of content listened to (dark blue line) for each immunisation call. On pregnancy week-18, 39% of the Kilkari calls were answered by the subscribers, which went up to 55% in the next immunisation call (week-28, when the child is 2–3 months old), and then gradually decreased over the time. Regarding content listening, on average 51% of the content of the Kilkari pregnancy week-18 call was listened to, as compared with 22% of the content of the call received in the 12th month postpartum (week-71). Overall, 20% of mobile numbers subscribed to Kilkari listened to ≥50% of the cumulative total of the 11 Kilkari immunisation calls (table 1) in the intervention arm of the RCT. Declines in exposure across the RCT sample were influenced by a range of factors, including increases in call non-delivery rates and SIM change.

Figure 1

Exposure to Kilkari immunisation messages among subscribers surveyed during postpartum data collection who were randomised to receive Kilkari calls in four districts of MP from 2018 to 2020.

Immunisation knowledge

Overall immunisation knowledge scores, out of a possible 11 points, averaged 3.64 (SD=1.95) for men and 3.62 (SD=1.79) for women (a 33% of total score for both) (table 2 and online supplemental Annexure-3). Among the study population, 83% of women and 53% of men were aware that a child’s first vaccination should be administered at birth. Most parents were aware that vaccination protects children from poliomyelitis (74% of mothers and 87% of fathers). When asked to name the diseases that can be prevented through vaccination, less than 10% of the parents cited whooping cough as an illness that vaccination protects children from. Around 17% of mothers and 23% of fathers knew that Vitamin-A improves eyesight.

Table 2

Percentage of correct responses by parents on immunisation knowledge

Questions	Mother	Father
	(N=4423)	(N=3781)
Knows that a child should receive the first vaccination at birth	82.6%	52.7%
Could name the diseases a child needs to be immunised against, during 0–12 months
Tuberculosis	15.0%	17.4%
Poliomyelitis	73.9%	86.6%
Hepatitis B	13.4%	16.2%
Diphtheria	11.4%	4.5%
Whooping cough	8.5%	9.4%
Tetanus	22.1%	25.2%
Measles	38.6%	35.4%
Diarrhoea	11.1%	18.1%
Knows the benefits of Vitamin-A
Good for eyesight	16.5%	23.1%
Helps make baby strong	68.7%	75.1%
Mean composite score of overall knowledge (out of a possible total score of 11)	3.62 (1.79)*	3.64 (1.95)*

*SD in parentheses.

Ordered logistic regression models exploring determinants of immunisation knowledge among women and men were used (table 3). Exposure to Kilkari immunisation calls was associated with higher mean immunisation knowledge among men (OR=1.23, 95% CI 1.02 to 1.48) but not women. Factors associated with higher overall knowledge scores among women were: more years of formal education (1.07, 1.05 to 1.09), a greater number of children (1.07, 1.02 to 1.13), asking for immunisation information from a health worker (1.29, 1.15 to 1.44) and receiving immunisation information from television (1.46, 1.30 to 1.65) or print (posters/newspapers/handouts) (1.97, 1.72 to 2.24). In contrast, mean knowledge scores were lower for women in the more disadvantaged OBC (0.85, 0.74 to 0.98) and SC/ST castes (0.83, 0.70 to 0.98) and for those who reported frequently having a zero-credit balance on their mobile phones (0.76, 0.63 to 0.91). Among men, apart from Kilkari exposure, knowledge scores were significantly higher with increasing wealth, years of formal education (1.15, 1.13 to 1.17) and a greater number of children (1.08, 1.03 to 1.14). For both men and women, significant differences in knowledge scores were observed across districts.

Table 3

Ordered logistic regression of the determinants of mothers’ and fathers’ immunisation knowledge

	Mother's immunisation knowledge (N=4423)			Father's immunisation knowledge (N=3781)
	OR	95% CI	P value	OR	95% CI	P value
Kilkari exposure
Listening to ≥50% immunisation calls	1.14	0.96 to 1.36	0.13	1.23	1.02 to 1.48	0.03
Caste (base: others)
OBC	0.85	0.74 to 0.98	0.02	0.86	0.74 to 1.00	0.05
SC/ST	0.83	0.70 to 0.98	0.03	0.78	0.66 to 0.93	0.01
Wealth (base: poorest)
Poor	1.08	0.91 to 1.29	0.38	1.13	0.93 to 1.38	0.21
Middle	1.10	0.91 to 1.32	0.33	1.26	1.02 to 1.55	0.03
Richer	0.98	0.81 to 1.20	0.87	1.33	1.07 to 1.66	0.01
Richest	1.18	0.94 to 1.47	0.15	1.47	1.16 to 1.87	<0.01
Education
Years of education	1.07	1.05 to 1.09	<0.01	1.15	1.13 to 1.17	<0.01
Employed	0.95	0.84 to 1.06	0.35	0.81	0.40 to 1.64	0.56
Role in decision making
Daily purchase	1.00	0.80 to 1.23	0.97	1.02	0.81 to 1.29	0.88
Pregnancy	1.07	0.87 to 1.33	0.51	1.04	0.83 to 1.32	0.71
Number of children	1.07	1.02 to 1.13	0.01	1.08	1.03 to 1.14	<0.01
Mobile access
Owns personal mobile	1.14	0.99 to 1.32	0.08	1.28	0.95 to 1.73	0.11
Phone with >12 hours	0.91	0.77 to 1.07	0.26	1.19	0.96 to 1.49	0.12
Often with 0 balance	0.76	0.63 to 0.91	<0.01	1.09	0.97 to 1.23	0.15
Other source of immunisation information
Asked advice from HW	1.29	1.15 to 1.44	<0.01
Received information from health worker	1.16	0.98 to 1.38	0.08
Television	1.46	1.30 to 1.65	<0.01
Poster/newspaper/handouts	1.97	1.72 to 2.24	<0.01
District (base: Mandsaur)
Hoshangabad	0.96	0.77 to 1.20	0.72	1.18	0.94 to 1.48	0.16
Rewa	3.14	2.65 to 3.71	0.00	1.16	0.98 to 1.38	0.08
Rajgarh	1.19	0.99 to 1.42	0.06	0.83	0.69 to 0.99	0.04

AWW, Anganwadi Centre; HW, health worker; LMIC, Low- and middle income countries; MR1, Measles & Rubella; OBC, other backward class; PCV, Pneumococcal Conjugate Vaccine; SC, scheduled caste; ST, scheduled tribe.

The prevalence for completeness and timeliness of immunisation practices for children 0–12 months old for BCG, 3 doses each of DPT and polio, and measles (basic package) and all 19 vaccines (comprehensive package) is provided in online supplemental Annexure-4. While 47% of children 0–12 months old had been fully vaccinated for the basic package at 12 months of age, around 44% had received the full comprehensive package. The overall timeliness of immunisations received to date was 6% and 1% for basic and comprehensive packages, respectively. Receipt of vaccines under the basic package was higher (84%–95%) than that of the comprehensive vaccines (including Rotavirus and Pneumococcal Conjugate Vaccine (PCV), 33%–77%) (online supplemental Annexure-4). For timeliness, while 84%–89% of the children received the birth vaccines on time, except for Measles & Rubella vaccine (MR1) (36%), the corresponding percentage figures for 6–10 weeks (35%–52%), 10–14 weeks (22%–28%), 14–18 weeks (9%–16%), 9–10 months (0%–16%) show that timely administration of vaccines reduced as the child’s age progressed.

Table 4 presents data on the determinants of immunisation completeness and timeliness for basic and comprehensive package. Table 5 presents data on the determinants of timely immunisation by vaccination schedule.

Table 4

Results of Heckman two-stage analysis of determinants of full immunisation and timely immunisation (birth–12 months)

Indicators	Full immunisation (probit)						Timely immunisation (ordered probit)
	Basic 8 vaccines			All 19 vaccines			Basic 8 vaccines			All 19 vaccines
	Coefficient	95% CI	P value	Coefficient	95% CI	P value	Coefficient	95% CI	P value	Coefficient	95% CI	P value
N=3230
Kilkari exposure
Listening to ≥50% immunisation calls	0.03	−0.12 to 0.17	0.74	0.04	−0.10 to 0.19	0.56	0.09	−0.03 to 0.21	0.12	0.09	−0.02 to 0.21	0.11
Immunisation knowledge
Father	0.00	−0.02 to 0.03	0.91	0.00	−0.03 to 0.02	0.85	0.01	−0.01 to 0.03	0.49	0.01	−0.01 to 0.03	0.38
Mother	−0.02	−0.04 to 0.01	0.27	0.00	−0.03 to 0.02	0.84	−0.01	−0.03 to 0.01	0.04	−0.01	−0.03 to 0.01	0.45
Gender of child—male (base: female)	−0.03	−0.17 to 0.11	0.66	−0.14	−0.24 to 0.03	0.01	−0.07	−0.15 to 0.02	0.13	−0.05	−0.14 to 0.03	0.21
Caste (base: others)
OBC	0.01	−0.11 to 0.13	0.82	−0.01	−0.13 to 0.11	0.91	0.00	−0.09 to 0.10	0.93	−0.01	−0.10 to 0.09	0.91
SC/ST	0.13	−0.02 to 0.18	0.02	−0.04	−0.18 to 0.10	0.61	−0.10	−0.22 to 0.01	0.04	−0.10	−0.21 to 0.01	0.08
Wealth (base: poorest)
Poor	0.26	0.11 to 0.42	<0.01	0.23	0.00 to 0.07	<0.01	0.21	0.08 to 0.34	<0.01	0.22	0.10 to 0.35	<0.01
Middle	0.38	0.21 to 0.56	<0.01	0.36	0.00 to 0.19	<0.01	0.30	0.17 to 0.44	<0.01	0.28	0.15 to 0.42	<0.01
Richer	0.30	0.13 to 0.48	<0.01	0.28	0.00 to 0.10	<0.01	0.33	0.19 to 0.47	<0.01	0.30	0.16 to 0.44	<0.01
Richest	0.35	0.16 to 0.54	<0.01	0.28	0.00 to 0.09	<0.01	0.33	0.18 to 0.48	<0.01	0.33	0.18 to 0.48	<0.01
Education
Years of education	−0.01	−0.03 to 0.01	0.18	−0.01	−0.03 to 0.01	0.26	0.01	0.00 to 0.02	0.05	0.01	0.00 to 0.02	0.07
Employed	−0.28	−0.38 to 0.18	<0.01	−0.26	−0.36 to 0.10	<0.01	−0.08	−0.16 to 0.00	0.04	−0.09	−0.17 to 0.02	0.03
Household decision making
Daily purchase	0.00	−0.19 to 0.19	0.96	−0.03	−0.22 to 0.16	0.74	0.02	−0.13 to 0.17	0.78	−0.02	−0.17 to 0.13	0.76
Pregnancy	−0.16	−0.35 to 0.04	0.11	−0.10	−0.30 to 0.09	0.28	−0.01	−0.16 to 0.14	0.89	0.03	−0.11 to 0.18	0.65
Number of children	0.00	−0.05 to 0.05	0.95	0.00	−0.05 to 0.05	0.94	−0.06	−0.10 to 0.02	0.00	−0.06	−0.10 to 0.01	<0.01
Mobile access
Owns personal mobile	0.01	−0.12 to 0.13	0.02	0.02	−0.10 to 0.14	0.73	0.05	−0.05 to 0.14	0.36	0.04	−0.05 to 0.14	0.36
Phone with >12 hours	−0.11	−0.25 to 0.03	0.12	−0.09	−0.23 to 0.04	0.18	−0.03	−0.14 to 0.09	0.65	−0.03	−0.14 to 0.08	0.59
Often with 0 balance	0.05	−0.12 to 0.22	0.56	0.05	−0.12 to 0.22	0.55	−0.02	−0.16 to 0.11	0.73	0.01	−0.12 to 0.14	0.90
Place of most of the vaccinations—AWW	−0.04	−0.21 to 0.13	0.05	−0.03	−0.20 to 0.14	0.72	−0.10	−0.24 to 0.03	0.13	−0.10	−0.23 to 0.04	0.15
Other sources of immunisation information
Asked advice from health worker	−0.08	−0.18 to 0.03	0.14	−0.05	−0.16 to 0.05	0.30	−0.14	−0.22 to 0.06	<0.01	−0.12	−0.20 to 0.04	<0.01
Received information from health worker	0.13	−0.02 to 0.27	0.10	0.11	−0.04 to 0.26	0.15	0.05	−0.07 to 0.17	0.38	0.04	−0.08 to 0.15	0.55
Television	−0.10	−0.21 to 0.01	0.06	−0.10	−0.20 to 0.01	0.08	−0.09	−0.18 to 0.01	0.03	−0.12	−0.20 to 0.05	0.01
Poster/newspaper/handouts	0.08	−0.04 to 0.19	0.04	0.08	−0.03 to 0.19	0.16	0.01	−0.08 to 0.10	0.90	0.02	−0.07 to 0.11	0.65
District (base: Mandsaur)
Hoshangabad	0.17	−0.02 to 0.37	0.08	0.12	−0.07 to 0.32	0.22	−0.18	−0.34 to 0.03	0.02	−0.06	−0.21 to 0.09	0.41
Rewa	−0.43	−0.57 to 0.29	<0.01	−0.52	−0.67 to 0.3	<0.01	−0.70	−0.81 to 0.58	<0.01	−0.68	−0.79 to 0.5	<0.01
Rajgarh	−0.17	−0.32 to 0.01	0.03	−0.21	−0.36 to 0.02	0.01	−0.47	−0.60 to 0.35	<0.01	−0.39	−0.51 to 0.2	<0.01

OBC, other backward class; SC, scheduled caste; ST, scheduled tribe.

Table 5

Results of Heckman two-stage analysis of determinants of timely immunization (by vaccination time-groups)

	Birth			6-10 weeks			10-14 weeks			14-18 weeks			9-10 months
	Odds Ratio	95% CI	P value	Odds Ratio	95% CI	P value	Odds Ratio	95% CI	P value	Odds Ratio	95% CI	P value	Odds Ratio	95% CI	P value
	(N=3230)
Kilkari exposure
Listening to >=50% immunisation calls	0.08	−0.08 to 0.24	0.03	0.01	−0.14 to 0.16	0.87	0.15	0.00 to 0.29	0.05	0.13	−0.03 to 0.30	0.12	0.08	−0.07 to 0.22	0.32
Immunization knowledge
Father	0.01	−0.02 to 0.03	0.65	0.01	−0.02 to 0.03	0.59	0.01	−0.02 to 0.03	0.53	0.02	−0.01 to 0.05	0.11	−0.02	−0.04 to 0.01	0.17
Mother	0.05	0.02 to 0.08	<0.01	−0.03	−0.05 to 0.00	0.06	−0.01	−0.04 to 0.01	0.33	−0.02	−0.05 to 0.01	0.26	−0.01	−0.04 to 0.02	0.41
Gender of child − Male (Base: Female)	0.07	−0.04 to 0.18	0.23	0.00	−0.10 to 0.11	0.94	−0.06	−0.17 to 0.05	0.28	−0.14	−0.27 to −0.0	0.04	−0.21	−0.32 to −0.1	<0.01
Caste (Base: Others)
OBC	0.04	−0.09 to 0.16	0.58	0.04	−0.08 to 0.16	0.51	−0.06	−0.18 to 0.06	0.32	−0.06	−0.20 to 0.07	0.36	0.03	−0.09 to 0.15	0.62
SC/ST	0.11	−0.04 to 0.25	0.17	−0.13	−0.27 to 0.01	0.08	−0.09	−0.24 to 0.06	0.23	−0.14	−0.31 to 0.03	0.11	−0.09	−0.24 to 0.05	0.21
Wealth (Base: Poorest)
Poor	−0.19	−0.36 to −0.0	0.02	0.33	0.17 to 0.49	<0.01	0.28	0.11 to 0.45	<0.01	0.34	0.11 to 0.56	<0.01	0.15	−0.01 to 0.32	0.07
Middle	−0.06	−0.24 to 0.12	0.52	0.32	0.15 to 0.49	<0.01	0.23	0.05 to 0.41	0.01	0.35	0.11 to 0.59	<0.01	0.30	0.12 to 0.47	<0.01
Richer	−0.10	−0.29 to 0.09	0.32	0.37	0.19 to 0.55	<0.01	0.24	0.05 to 0.44	0.01	0.39	0.15 to 0.63	<0.01	0.31	0.13 to 0.50	<0.01
Richest	−0.34	−0.55 to −0.10	<0.01	0.38	0.19 to 0.57	<0.01	0.36	0.16 to 0.57	<0.01	0.50	0.25 to 0.75	<0.01	0.36	0.16 to 0.55	<0.01
Education
Years of education	0.01	−0.01 to 0.03	0.33	0.00	−0.01 to 0.02	0.68	0.00	−0.02 to 0.01	0.87	0.01	−0.01 to 0.03	0.21	0.02	0.00 to 0.03	0.06
Employed	0.09	−0.02 to 0.19	0.12	−0.11	−0.22 to −0.01	0.03	−0.12	−0.22 to −0.01	0.03	−0.03	−0.15 to 0.10	0.68	−0.07	−0.18 to 0.03	0.16
Household decision making
Daily purchase	−0.03	−0.22 to 0.17	0.79	−0.05	−0.25 to 0.14	0.59	−0.06	−0.26 to 0.14	0.56	−0.04	−0.28 to 0.20	0.76	0.05	−0.14 to 0.25	0.58
Pregnancy	0.09	−0.11 to 0.29	0.37	0.05	−0.14 to 0.24	0.59	0.10	−0.10 to 0.30	0.31	0.08	−0.15 to 0.32	0.5	−0.10	−0.29 to 0.09	0.32
Number of children	−0.03	−0.08 to 0.02	0.28	−0.07	−0.12 to −0.01	<0.01	−0.08	−0.13 to −0.01	<0.01	−0.07	−0.13 to −0.01	0.03	−0.02	−0.07 to 0.03	0.53
Mobile access
Owns personal mobile	0.05	−0.08 to 0.19	0.42	0.07	−0.06 to 0.19	0.29	0.08	−0.05 to 0.21	0.21	0.08	−0.06 to 0.22	0.28	−0.12	−0.24 to 0.01	0.07
Phone with >12 hours	0.06	−0.09 to 0.20	0.45	−0.06	−0.20 to 0.08	0.39	−0.03	−0.18 to 0.11	0.65	0.00	−0.17 to 0.17	1.00	0.04	−0.11 to 0.18	0.62
Often with 0 balance	0.10	−0.08 to 0.28	0.28	−0.09	−0.27 to 0.08	0.31	−0.03	−0.20 to 0.15	0.77	−0.07	−0.28 to 0.13	0.49	0.02	−0.15 to 0.20	0.81
Place of most of the vaccinations − AWW	−0.03	−0.20 to 0.15	0.77	−0.03	−0.20 to 0.14	0.70	−0.09	−0.26 to 0.08	0.30	−0.14	−0.33 to 0.05	0.15	−0.26	−0.43 to −0.01	<0.01
Other sources of immunization information
Asked advice from HW	−0.01	−0.11 to 0.10	0.88	−0.16	−0.26 to −0.01	<0.01	−0.13	−0.23 to −0.01	0.02	−0.11	−0.23 to 0.02	0.10	−0.05	−0.15 to 0.05	0.33
Received information from HW	0.01	−0.15 to 0.16	0.95	0.02	−0.13 to 0.18	0.76	−0.01	−0.17 to 0.14	0.89	−0.07	−0.25 to 0.11	0.47	0.11	−0.05 to 0.26	0.18
Television	0.13	0.02 to 0.25	0.02	−0.16	−0.27 to −0.0	<0.01	−0.10	−0.21 to 0.01	0.08	−0.07	−0.20 to 0.06	0.27	−0.07	−0.18 to 0.04	0.19
Poster / Newspaper / Handouts	−0.07	−0.19 to 0.05	0.25	0.07	−0.05 to 0.18	0.26	0.01	−0.12 to 0.11	0.97	0.01	−0.12 to 0.14	0.91	0.02	−0.10 to 0.13	0.76
District (Base: Mandsaur)
Hoshangabad	−0.17	−0.37 to 0.04	0.11	−0.13	−0.34 to 0.08	0.22	−0.20	−0.41 to 0.00	0.05	−0.22	−0.46 to 0.02	0.07	−0.06	−0.26 to 0.14	0.57
Rewa	−0.51	−0.71 to −0.30	<0.01	−0.67	−0.82 to −0.50	<0.01	−0.74	−0.89 to −0.50	<0.01	−0.77	−0.93 to −0.60	<0.01	−0.53	−0.68 to −0.30	<0.01
Rajgarh	−0.24	−0.40 to −0.01	0.01	−0.44	−0.60 to −0.20	<0.01	−0.34	−0.50 to −0.10	<0.01	−0.39	−0.57 to −0.20	<0.01	−0.28	−0.43 to −0.10	<0.01

OBC, other backward class; SC, scheduled caste; ST, scheduled tribe.

Higher probability of complete immunisation for the basic package was associated with being a member of the most disadvantaged SC/ST caste (coefficient=0.13, 95% CI 0.02 to 0.18), in the higher wealth quintiles, families where women reported owning a personal mobile phone (0.01, –0.12 to 0.13) and where women received immunisation information from poster/newspaper/handouts (0.08, –0.04 to 0.19) and lower probability for children with employed mothers (−0.28, –0.38 to −0.18) (Table 4). The probability of receiving the comprehensive package was lower among male children (−0.14, –0.24 to −0.00), those in the poorest socioeconomic strata and those with an employed mother (−0.26, –0.36 to −0.10).

The probability of timely receipt of the basic package was lower among those in the SC/ST caste (−0.10, –0.22 to 0.01), poorest socioeconomic strata, with employed mothers (−0.08, –0.16 to 0.00), whose mothers had greater immunisation knowledge scores (−0.01, –0.03 to 0.01), who were in families with more children (−0.06, –0.10 to −0.02), whose mothers reported receiving immunisation information from television (−0.19, –0.18 to −0.01) and who reported asking for or receiving advice on immunisations from health workers (−0.14, –0.22 to −0.06) (Table 4). Timely receipt of comprehensive package was lower among the poorest, families with a greater number of children (−0.06, –0.10 to 0.01), with an employed mother (−0.09, –0.17 to 0.02), who reported receiving immunisation information from television (−0.12, –0.20 to 0.05) and who reported asking advice on immunisations from health workers (−0.12, –0.20 to 0.04).

Improved timeliness of vaccination by visit schedule was assessed at birth, 6–10 weeks, 10–14 weeks, 14–18 weeks and 9–10 months (Table 5). Timely immunisation at birth was significantly associated with listening to ≥50% of the cumulative content of Kilkari immunisation calls (0.08, –0.08 to 0.24) as well as higher women’s knowledge scores (0.05, 0.02 to 0.08). Improvements in the timeliness of other visits was significantly associated wealth.

Discussion

Kilkari is the largest direct-to-beneficiary mHealth service in the world. This analysis aims to provide evidence on the determinants of parental immunisation knowledge and immunisation practice (completeness and timeliness) for children 0–12 months of age from four districts in rural MP, India. Study findings suggest that only one-thirds of women and men knew the timing for the start of vaccinations, diseases linked to immunisations and the benefits of Vitamin-A. Exposure to Kilkari calls was associated with higher knowledge among fathers but not mothers. It was also associated with higher likelihood of children 0–12 months receiving birth vaccines on time but had no association with overall full and timely immunisation for the basic or comprehensive immunisation package.

Determinants of immunisation knowledge

The finding that exposure to Kilkari increased men’s but not women’s immunisation knowledge may be linked to men’s higher access to mobile phones in India.27 While the Kilkari backend data show whether a Kilkari call was answered and whether it was allowed to play, it cannot reveal who picked up the call and who listened to the message. Qualitative research suggests that Kilkari calls made to mobile phones shared by spouses tend to be answered by husbands, not wives.28 About 25% of the women who were subscribed to the RCT reported that they had access to a shared household phone. The remaining 75% of women reported that they owned their own phone, but 25% of these women reported that they shared these phones with others. Even among phones owned by women that were not reportedly shared, husbands and other household members are likely to have answered and listened to a portion of the calls. While other mobile-based information dissemination programmes have improved immunisation knowledge and vaccination practice,16 it also improved immunisation practice among caregivers with shared mobile phones in Kenya.18

Beyond Kilkari exposure, higher immunisation knowledge for both mothers and fathers was associated with being higher caste, having higher education attainment and having had more children. Greater wealth was associated with higher knowledge for fathers but not mothers. Mothers who asked health workers about immunisation and received immunisation information through TV and print had higher knowledge than those who did not. This could indicate that women were more dependent on these sources for immunisation information than on information received through a mobile phone. The association between formal education and greater knowledge about immunisation echoes findings from other studies conducted in India and Iraq.9 29 These findings on health information seeking and sources resonate with research in other LMIC settings that emphasise the value of primary level healthcare workers30 31 and mass media to increase immunisation awareness and practice.32 33

There was negative relationship between mothers’ immunisation knowledge and mothers whose phone is often at zero balance (in India, many mobile network operators do not allow phones with zero balance to receive calls). Since our sample consists of women with access to a mobile phone and our model controlled for family wealth, this finding suggests that having an operational phone creates opportunities for knowledge improvement. At the same time, since women rely on men for mobile phone top-ups,34 it appears that women in families where the men delay or avoid topping up their phones are disempowered in terms of seeking or retaining immunisation knowledge as well.

Determinants of immunisation coverage

Elsewhere, we report findings on the impact of Kilkari exposure on immunisation coverage for children of 0–12 months age, among other RMNCH outcomes using intention to treat and instrumental variable analysis.35 Findings from that analyses suggest a significant improvement in immunisation coverage at 10 weeks. We have sought to build on that work here, using a novel methodology, borrowed from economic literature, to model the selection biases when outcome assessments (availability of vaccination records) are correlated with probability of outcomes (vaccination status). Our analysis looks at possible pathways for explaining the differences between the exposed and unexposed. Here, we sought to focus on the mediator of parental knowledge of immunisation for completeness and timeliness of certain key vaccines. Since we restrict our sample to those with documentation of vaccination (rather than reported), we expect our analysis to have less measurement error and a more accurate measure of vaccination indicators. However, differences in the samples make further comparisons with the findings reported elsewhere challenging.

The poor coverage of full immunisation (44%) and timely immunisation among our sample in this analysis echoes findings from other northern and central Indian states, showcasing a widespread shortcoming of the Indian health system.36 We found that timeliness was highest for birth vaccines—around four out of five children were administered these on time, likely because these are administered at the facility before the mother and child are discharged postpartum. However, the percentage of children receiving timely vaccinations decreased as they grew.

Kilkari exposure was not associated with overall improvements of full and timely immunisation coverage but did increase timely immunisation at birth. This finding may be linked to the timing of Kilkari calls and parental listening patterns. The first Kilkari call on child immunisation coincides approximately with the time of delivery (week-18) and was the most listened to immunisation message; all subsequent messages went out over the first year after the child was born and showed decreasing listenership. Parents may have listened to the week-18 message with heightened interest and greater focus right before childbirth due to their anticipation of a major life event and without the distraction of a new baby. Parental knowledge of birth vaccination was higher than knowledge of later vaccines as well. However, additional qualitative research may provide the insights to understand the relation between father’s knowledge and birth vaccination.

While mobile-based information dissemination and reminders to parents have increased immunisation coverage in Kenya, Nigeria, Pakistan, Bangladesh and other LMICs,18–20 37 38 these interventions focused specifically on immunisation (whereas Kilkari covered a wide range of maternal, reproductive and child health topics), involving tailored vaccine reminders timed to occur prior to a scheduled visit (whereas Kilkari messages were standardised and not only linked to scheduled appointments). The intervention in Kenya also included financial incentives for timely vaccination (whereas Kilkari did not).18

One of the most significant and positive determinants for both full and timely immunisation was the economic status of the household, echoing findings elsewhere in India39 and in China, Ethiopia and Iraq.9 32 40 Children were more likely to have their full and timely immunisation if their mothers did not work and if they had fewer or no siblings, similar to the findings from Iraq and Nigeria.9 41 Families with more financial resources and whose mothers had more time in the home may find it easier to access health facilities on time for immunisation as compared with poorer families, families where the mother worked and families with many children.

Parental knowledge and immunisation coverage

While an association between parental knowledge about immunisation and child immunisation coverage has been found in Mozambique12 and Nigeria,7 the link is not always clear. In Ghana, there was no association30 and our study found mixed evidence, where mother’s immunisation knowledge was associated with a higher likelihood of timely birth immunisation but lower likelihood of timely vaccination with the basic package. The major role of health systems factors may at times override the link between parental factors and child vaccination, such as when a strong health system enables providers to vaccinate children without parental advocacy, or when a weak health system leaves even knowledgeable parents without access to timely and complete vaccination for their children.

Surprisingly, timely vaccination was negatively associated with mothers asking about immunisation from health workers. In Ghana, interaction with health workers during antenatal care visits was a positive predictor for full immunisation for children.30 It may be that the mothers who spoke to health workers about immunisation were those who had already missed appointments; health worker engagement may have enabled them to get their child’s vaccination back on track.

Conclusion

Maternal mobile messaging programmes have been identified as a cost-effective mechanism to increase parental awareness and send reminders for vaccination in order to achieve complete and timely child immunisation.16 This analysis showcases the potential and the limitations of Kilkari, which is one of the world’s largest maternal mobile messaging programme. While this intervention increased father’s knowledge and improved the timeliness of immunisation at birth, it did not increase mother’s knowledge, nor did it improve overall child (0–12 months) immunisation completeness and timeliness. In a context with overall low parental knowledge about immunisation and poor child immunisation coverage, improvement demands expanded and intensified outreach coupled with health system strengthening. Parental engagement through a programme such as Kilkari can be an important component of a broader immunisation improvement strategy. While in its current form, Kilkari covers a wide range of maternal, reproductive and child health topics, of which immunisation is a small part, intensified immunisation content, and a mechanism to enrol both spouses whenever possible, may achieve deeper impact.

Further efforts to use mobile-based communication to raise awareness about the benefits of immunisation, (ideally) supported by mass media communication, is even more crucial with the backdrop of COVID-19, with limited information dissemination through face-to-face interactions, the possible diversion of focus from routine immunisation, and pockets of scepticism about the COVID-19 vaccine. At the same time, there was bias of immunisation practice towards wealthier and higher caste group. Structured and focused approach needs to be adopted to reach socially and economically vulnerable in remote areas, specifically for immunisation practice.

Patient and public involvement

Patients were first engaged on identification in their households as part of a household listing carried out in mid/late 2018. Those meeting eligibility criteria were interviewed as part of the baseline survey and ultimately randomised to the intervention and control arms. Prior to the administration of the baseline, a small number of patients were involved in the refinement of survey tools through qualitative interviews, including cognitive interviews, which were carried out to optimise survey questions, including the language and translation used. Finalised tools were administered to patients at baseline and endline, and for a subsample of the study population, additional interviews carried out over the phone and via qualitative interviews between the baseline and endline surveys. Unfortunately, because of COVID-19, patients and associated travel restrictions could not be involved in the dissemination of study findings.