Small mobile conditional cash transfers (mCCTs) of different amounts, schedules and design to improve routine childhood immunization coverage and timeliness of children aged 0-23 months in Pakistan: An open label multi-arm randomized controlled trial.

Background: Cost-effective demand-side interventions are needed to increase childhood immunization. Multiple studies find tying income support programs (≥USD 50 per year) to immunization raises coverage. Research on maximizing impact from small mobile-based conditional cash transfers (mCCTs) (≤USD 15 per fully immunized child) delivered in lower-income settings remains sparse.
Methods: Participants in Karachi, Pakistan, were individually randomized into a seven arm, factorial open label study with five mCCT arms, one reminder (SMS) only arm, and one control arm. The mCCT arms varied by amount (high ∼USD 15 per fully immunized child versus low ∼USD 5 per fully immunized child), schedule (flat versus rising payments over the schedule), design (certain versus lottery payments), and payment method (airtime or mobile money). Children were enrolled at BCG, pentavalent-1 (penta-1) or pentavalent-2 (penta-2) vaccination and followed until at least 18 months of age. A serosurvey in 15% sub-sample validated reported study coverage. The full immunization coverage (FIC) at 12 months (primary outcome) was analyzed using logit regression. ClinicalTrials.gov (NCT03355989), 3ie registry (58f6ee7725fc1), and AEA RCT Registry (AEARCTR-0001953). Findings: Between November 6, 2017, and October 10, 2018, a total of 11,197 caregiver-child pairs were enrolled, with 1598-1600 caregiver-child pairs per arm. FIC at 12 months was statistically significantly higher for any mCCT versus SMS (OR:1.18, 95% CI: 1.05-1.33; p = 0.005). Within the mCCT arms, FIC was statistically significantly higher for high versus low amount (OR: 1.16, 95% CI: 1.04-1.29; p = 0.007), certain versus lottery payment (OR: 1.30, 95% CI: 1.17-1.45; p < 0.001) and airtime versus mobile money (OR: 1.17, 95% CI:1.01-1.36; p = 0.043). There was no statistically significant difference between a flat and increasing schedule (OR: 1.03, 95% CI: 0.93-1.15; p = 0.550). SMS had a marginally statistically significant impact on FIC versus control (OR: 1.16, 95% CI: 1.00-1.35; p = 0.046). Findings were similar for up-to-date coverage of penta-3, measles-1 and measles-2 at 18 months. Interpretation: Small mCCTs (USD 0.8-2.4 per immunization visit) can increase FIC at 12 months and up-to-date coverage at 18 months at USD 23 per additional fully immunized child, in resource-constrained settings like Pakistan. Design details (certainty, schedule and delivery method of mCCTs) matter as much as the size of payments. Funding: Global Innovation Fund, GiveWell.

Evidence before this study

We searched PubMed and Cochrane Library without date restrictions for evidence on small Conditional Cash Transfers (CCTs) to increase uptake of immunization using the search terms “conditional cash transfers”, “immunization”, “vaccination”, “monetary incentives”, “lottery-based payments” and “immunization coverage” in February 2021. We found several rigorous assessments of large (≥USD 50 per child per year) CCT payments, primarily from Latin America aimed at increasing income of the poor with positive impacts on immunization. Experimental evidence on small CCTs (≤ USD 15 per fully immunized child) to promote immunization was scarce, limited to three randomized control trials from India, one from Kenya and one from Pakistan. All studies found statistically significant increases in immunization coverage as a result of small CCTs. However, not all these studies explored the effect of different CCT designs on immunization coverage, and those that varied design aspects (i.e. CCT size, structure), reported inconclusive results.

Added value of this study

This study tests a range of practical questions raised by the potential introduction of large-scale programs linking small airtime payments (that can only be used for mobile talk time, SMS or data), to immunization in LMICs like Pakistan. Specifically, it tests: how large the transfer should be; whether using lotteries to incentivize immunization is preferable to small certain payments; whether payments should be higher for later immunizations where take-up is lower; and whether mobile money or airtime provide a more effective incentivization mechanism. Our study finds that the design of CCTs impacts immunization coverage rates and provides evidence for leveraging ubiquitous airtime payments in CCT programs (as opposed to mobile-money payments with limited take-up and in-kind transfers with logistic and leakage challenges).

Implications of all the available evidence

Our findings, combined with others, make a strong case for implementing small mCCT-based demand-side interventions for increasing immunization coverage and timeliness, and provide a practical road map for scale-up, especially in Pakistan. Programs should explore introducing small mCCTs in populations not currently covered by income support programs, or make a small part of existing income support programs conditional on immunization.

Alt-text: Unlabelled box

Introduction

Routine childhood immunization is a proven intervention for increasing child survival in developing countries. Yet despite the availability of free-of-cost vaccines, in 2020 alone, 23 million infants failed to receive age-appropriate basic immunizations globally, and over 1.5 million children died from vaccine-preventable diseases. Approximately 60% of unvaccinated children live in 10 countries, including Pakistan, where full immunization coverage (FIC) for basic vaccines is 66%. Sub-optimal immunization coverage can be attributed to both supply-side constraints including poor health service delivery infrastructure, vaccine and health worker shortages, lack of accountability and monitoring, and demand-side barriers such as lack of parental awareness regarding importance of vaccination, competing priorities for caregivers and vaccine hesitancy. Until recently, Pakistan focused on improving immunization supply while demand-side interventions were limited to social mobilization, education, and communication. However, high take-up of early vaccines (88% of children in Pakistan receive BCG (Bacille Calmette Guérin; the first vaccine in the schedule) means the vast majority of households can access vaccines, are not deeply opposed to vaccinations, and might therefore respond to demand-side interventions designed to act as nudges to increase uptake and address small financial and nonfinancial barriers to immunization.

Historically, middle-income countries (MICs), especially in Latin America, leveraged widespread income support programs to promote immunization by making cash transfers conditional on immunization, regular clinic visits, and school enrollment. CCTs increased the use of preventive health services, including immunization,, and improved health status in Mexico, Honduras and India. A recent meta-analysis from LMICs on the effect of CCTs on neglected tropical diseases (NTDs) also found CCTs to be associated with improved NTD outcomes. Varying the timing of CCT payments had large impacts on outcomes. However, as the transfers are primarily designed to increase incomes of the poor, these CCTs are large (typically over ≥USD 50 per child) and determining income eligibility and compliance with conditions is expensive. CCTs are therefore relatively rare in low- and lower-middle-income countries (LICs and LMICs), where immunization coverage remains low. Yet more affordable CCTs (USD ≤15 per child delivery/HIV test/fully immunized child) have been shown to increase institutional deliveries among pregnant women, improve patient HIV test acceptance, and raise immunization uptake. Conditional cash transfers of

While these studies serve as proof of concept that small mCCTs can promote immunization, they do not determine the most effective way to structure small mCCTs in terms of amount, schedule, and design, nor do they test a scalable platform for delivering the small mCCT. Existing evidence on the size of mCCTs is inconclusive. In Kenya and India, a larger mCCT (USD 2.4 versus USD 0.9 in Kenya, USD 1.25 versus 0.70 in India) yielded higher but not significantly different FIC rates.,

People will often adopt a behavior that is good for them but takes effort, but then fail to persist. Immunization follows this pattern, with immunization rates in Pakistan falling from 88% for BCG to 73% for measles. Weighting payments towards the end of the schedule would focus mCCTs where they are needed most. Banerjee et al. found higher payments towards the end of the schedule were more effective than equal payments across the schedule. Lotteries to promote immunization and other health behavior are increasingly used on the assumption that some people are risk-loving and/or overestimate the chance of winning. No studies (to our knowledge) compare lottery and certain payments. While earlier small CCT programs have relied on in-kind transfers (with logistics and leakage challenges) and mobile money (which is not widely used in many countries), airtime (which was used in India) is a promising alternative with comparatively simple logistics and widespread use.

Recent meta-analyses of SMS reminders in LMICs found reminders on their own significantly improve immunization coverage and timeliness. They provide both a benchmark against which to measure mCCTs, and have the potential to enhance the effectiveness of mCCTs by reminding caregivers of their next payment.

We measured the relative effectiveness of different types of small mCCT structures (the amount, progressivity, certainty, and payment method) on immunization coverage rates and timeliness. We also tested the impact of SMS reminders (with and without mCCTs) on immunization coverage and timeliness.

Methods

Study design and participants

RCT: We conducted a seven-arm factorial, individually randomized, open label, controlled trial where participants were evenly allocated among five mCCT arms, one reminder (SMS) only arm, and one control. We randomized mCCTs on three dimensions: high versus low, flat versus rising payments over the schedule, and certain versus lottery payments in a factorial design. Out of the five mCCT arms, four used airtime, while one used mobile money. The study protocol envisaged pooling arms to answer key design questions with precision (Figure 1). Each mCCT arm received SMS reminders. The conduct, analysis, and reporting of results followed the Consolidated Standards of Reporting Trials (CONSORT) multi-arm guidelines.

Figure 1

Trial design: children visiting a study immunization clinic for BCG, penta-1, or penta-2 vaccine were enrolled and followed up until at least 18 months of age in a seven-arm study comprising five mCCT arms with varying amounts (high or low), schedules (sharp or flat progressivity), design (certainty of payment), method of payment (mobile money or airtime top-ups) an SMS only arm and a control arm.

*We used an exchange rate of 1 USD=137 PKR (average exchange rate for the study duration) for the costs effectiveness analysis. Easypaisa is a trademark of Telenor Microfinance Bank.

The study was conducted in Korangi town, located in Karachi city in Sindh province of Pakistan, which has FIC rates below the national average (48.8% of 12-23 month olds). Korangi has an ethnically diverse population of over 1 million. Participants were recruited from all ten government immunization clinics in Korangi, a high-volume private immunization clinic, and a private birthing center. Vaccination services in Korangi are provided at fixed immunization clinics and during outreach by vaccinators. All caregiver-child pairs visiting study clinics were screened for eligibility by study staff. Inclusion criteria for participation included the child being under 2 years, visiting to receive the BCG, pentavalent-1 (penta-1) or pentavalent-2 (penta-2) vaccine, and the ability of the caregiver to provide a cell phone number where they could be reached. Exclusion criteria included multi-birth children (twins or triplets), or plans to migrate from Korangi within three years. National identity card (NIC) was not a requirement for enrollment.

Serosurvey: To ensure quality control and validate the study coverage estimates, biomarkers for measles and tetanus toxoid (TT) were measured in a 15% random sub-sample of enrolled children. Antibodies for TT served as a proxy for vaccines given at 6-, 10-, and 14-week visits while measles antibodies proxied measles visits. Children were eligible for the serosurvey if they were between 18–24 months when the blood collection visit occurred and had not received the pentavalent or measles vaccine within 4 weeks. Exclusion criteria included child being unwell or a child's death.

Vaccination schedule

In 2017, Pakistan's routine EPI immunization schedule included BCG at 0-6 weeks of age, three doses of pentavalent (penta; containing DTP, HepB, Hib) vaccine, two doses of pneumococcal vaccine (PCV) and three doses of OPV at 6, 10 and 14 weeks of age, and two doses of measles at 9 and 15 months.

Randomization and masking

RCT: Each enrolled caregiver-child pair was randomly assigned to one of the study arms by study staff. Stratified block randomization was used with a block size of 56 and six strata based on enrollment vaccine cohort (BCG, penta-1, and penta-2) and sex. The randomization lists were generated by Dr. Rachel Glennerster and her team using the statistical software Stata. The allocation sequence was concealed from the study staff responsible for screening and enrolling participants through real-time phone-based access to the randomized sequence (see supplemental material). The final treatment assignment was not masked from the study staff responsible for enrollment or the participants.

Serosurvey: The sub-sample was stratified on sex, enrollment vaccine cohort, study arm, and lottery status. Households of selected children were approached for serosurveys in sequential order. If the child was ineligible or consent was refused, a replacement ID was selected from the randomized list, and the process was continued until the target sample size was achieved.

Study procedures and intervention

RCT: All screening, enrollment, and follow-ups were conducted by study staff at the study immunization clinics via the Government of Sindh's Zindagi Mehfooz electronic immunization registry (EIR) with added study-specific functionalities. Participants providing verbal consent were enrolled and assigned a unique study ID and Quick response (QR) code pasted on the government-issued EPI card. The QR code was scanned through the EIR to record the child's biodata, demographic information, and immunization history. Participant data was submitted to an electronic server in real-time, and the caregiver-child pair was allocated to a study arm via the pre-programmed randomized sequence.

Airtime payments to caregivers’ registered cellphone number were automatically generated by the EIR, approved by a dedicated study team member, and could be used instantly by participants. Mobile money payments were sent to the vendor (Easypaisa), who sent a payout notification and passcode to the registered caregiver via SMS. Caregivers could redeem the cash by presenting their NIC and SMS passcode at any Easypaisa franchise following biometric verification. Caregiver-child pairs in mCCT and SMS arms also received up to 3 automatic SMS reminders: a day before, on the day of, and (if the appointment was missed) six days after the scheduled immunization date (see supplemental material for detail). Immunization dates were automatically calculated by the EIR using Pakistan's EPI schedule and the child's date of birth reported at enrollment. SMS reminders in the mCCT arms also specified the mCCT amount caregiver-child pairs would receive post vaccination on their next visit. All caregiver-child pairs were followed until the child was at least 18 months.

Serosurvey: Children selected for the serosurvey were approached by study staff at their residence. Up to three visit attempts were made, after which the next listed child was approached. Following written consent, a finger prick sample of at least 0.2 ml was collected using aseptic techniques. A maximum of three pricks were made, after which the next eligible child was approached. All blood samples were transported via cold box and reached the laboratory within two hours.

Database and data handling procedures

Data for the study was directly captured within the Government's Electronic Immunization Registry. The primary study data collected for the immunization was the same as the routinely collected data as per Department of Health requirements. Additional demographic data was collected for children enrolled in the study. For mCCT transfer via mobile money, the NIC number was also collected which is standard practice for Government programs disbursing funds to individuals. Data on SMS reminder status (receipt or failure) was automatically populated in the database through the EIR.

The phones used for data collection by field staff had password locks with additional protection through software “sign-on” passwords. The data was transferred from the device to a server in real-time where possible. In case of data connectivity disruption, paper-based forms were filled. Access to data on the server was via a password protected web-dashboard interface. The data was shared only with authorized program personnel responsible for data entry and analysis. The de-identified data set was available for the Research team responsible for analysis (further details in supplemental material).

Ethical review

The protocol received ethical approvals from the Institutional Review Board at Interactive Research and Development (IRB-IRD) and the Committee on the Use of Humans as Experimental Subjects (COUHES) at the Massachusetts Institute of Technology (registration number IRB00000522). IRB-IRD is registered with the US Department of Health and Human Services (DHHS) Office for Human Research Protections (OHRP) with registration number IRB 00005148. This trial is registered with ClinicalTrials.gov (NCT03355989), 3ie registry (58f6ee7725fc1), and AEA RCT Registry (AEARCTR-0001953).

Outcomes

RCT: The primary outcome measure was FIC at 12 months, defined as receiving one dose of BCG, three doses of Penta, PCV and OPV, and one dose of measles vaccines. We examined the proportion of children receiving timely doses of each antigen (receiving the antigen within 28 days of the recommended age), and up-to-date immunization coverage at 18 months of age (proportion of vaccinated children at 18 months) for the third dose of pentavalent (penta-3), the first dose of measles (measles-1), and the second dose of measles (measles-2) vaccines by study arm, mCCT amount, schedule and design. Children's immunization data for analyzing study outcomes was collected at the study immunization clinics by the study staff during enrollment and follow-up through verifying vaccination administration by the vaccinator and follow-up phone calls if required. For 4.0% (446/11,197) of children, vaccination dates for at least one vaccine came from outreach data reported by the study site vaccinator.

Serosurvey: Serum was extracted by centrifuge at 6000g for 3 minutes and stored at −20°C until they were tested for measles immunoglobulin G (anti-measles IgG) and tetanus immunoglobulin G (anti-TT IgG) antibodies with enzyme-linked immunosorbent assays (ELISAs) (Eurroimmun anti-measles IgG ELISA and Euroimmun anti-TT IgG ELISA). Reported sensitivity and specificity were 100.0% each for anti-measles IgG ELISA and 98.0 and 100% for anti-TT IgG ELISA, respectively. Serum samples for anti-measles IgG were classified as positive, borderline, or negative, with all borderline samples retested once. Serum samples for anti-TT IgG were classified as positive (sufficient immunity) and negative (insufficient immunity) based on manufacturer reference for the Euroimmun anti-TT IgG test.

Sample size

As per the protocol, the study was powered to detect a minimum detectable effect size (MDE) of an absolute 5ppt change in FIC (binary outcome) rates at 12 months based on a judgment that this was the minimum increase needed to cause a change in policy. A priori sample size calculations were carried out in Stata (version 14.2) and assumed a baseline FIC of 51.5% at 12 months, alpha of 0.05, and power (1-beta) of 0.80 resulting in an equal sample size of 1559 per arm. This was rounded up to 1600 reflecting uncertainty in coverage rate in the control arm. The MDE for high versus low payments was 3.5%. No adjustment was made in the power analysis for multiplicity or attrition as drop-out was an outcome of interest.

Statistical analyses

The analysis was performed by the original assigned group (Intention to Treat). Children with missing vaccination dates (321/11,197, 2.9%) were included in coverage analysis but excluded from (secondary) timeliness analysis. Means and standard deviations of baseline data across mCCT, SMS, and control arms were described.

Unadjusted and adjusted odds ratios were calculated for receiving SMS only versus control and any mCCT versus SMS for our primary and secondary outcomes. Adjusted odds ratios and 95% confidence intervals were calculated using logit regression and adjusted for risk variables selected from all baseline characteristics using one step lasso. We then estimated the unadjusted and adjusted odds ratios for the four key design choices: high versus low, flat versus sharp, certain versus lottery, and airtime versus mobile money reporting p-values adjusting for multiplicity (using the Romano and Wolf approach). We also reported FIC at 12 months for all 7 arms (SMS only versus control and any mCCT versus SMS). The supplemental analysis calculated unadjusted and adjusted odds ratios for receiving any mCCT and SMS versus control to determine the impact of the full program (mCCTs and SMS) and also reported FIC estimates for all 12 study sub arms. These results should be considered exploratory. Our analysis differed slightly from our original protocol to reflect emerging best practice: we compared all mCCTs to SMS only (rather than control) as SMS is now the standard of care (at least in the study setting), and risk variables were selected using emerging best practice machine learning techniques. Supplemental Material describes these deviations in detail.

Analyses were performed using R, version 4.1, and Stata, version 15.1.

Role of the funding source

The funding source for the study had no role in the study design, data collection, data analysis, data interpretation, or manuscript writing. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Results

Between November 6, 2017, and October 10, 2018, we enrolled 11,197 caregiver-child pairs into the study (Figure 2). We enrolled 1598-1600 caregiver-child pairs per mCCT arm,1600 in the reminders (SMS) only arm and 1599 in the control arm. The primary analytic sample included all 11,197 caregiver-child pairs, followed until 18 months of age at the study immunization clinics.

Figure 2

Participant flow diagram.

Participant characteristics and sociodemographic distribution were similar across arms (Table 1). The proportion of male children (51.3%; 5740/11,197) enrolled was slightly higher than females (48.7%; 5457/11,197). The average age at enrollment was 61.4 days, with more than half (61.8% (6915/11,197) enrolled at BCG. A total of 91.4% (10,185/11,197) of fathers owned a personal cell phone compared to 49.5% (5538/11,197) of mothers. Only 51.7% (5784/11,197) of participants provided NICs (necessary to receive mobile money payments), although the proportion was significantly higher in the mCCT arm relative to control (54.2%; 4337/7998).

Table 1

Demographics and baseline characteristics of children enrolled by study arms (n = 11,197).

	Any mCCT(n = 7998)		SMS(n = 1600)		Control(n = 1599)		Total(n = 11,197)
	n	%	n	%	n	%	n	%
Female	3898	48.7	777	48.6	782	48.9	5457	48.7
Enrolment vaccine
BCG	4938	61.7	994	62.1	983	61.5	6915	61.8
Penta-1	2043	25.5	411	25.7	413	25.8	2867	25.6
Penta-2	1017	12.7	195	12.2	203	12.7	1415	12.6
Enrolment Age in weeks mean (sd)	7.1	(8.9)	7.0	(8.4)	7.0	(8.7)	7.1	(8.8)
Previously used Mobile Money	2579	32.3	554	34.7	512	32.1	3645	32.6
NIC provided against child record	4337	54.2	738	46.1	709	44.3	5784	51.7
Father has cell phone^	7274	91.4	1445	90.8	1,466	92.4	10,185	91.4
Father has NIC^^	7407	92.8	1463	91.6	1476	92.5	10,346	92.6
Father's Education (years) mean (sd)	7.6	(4.7)	7.6	(4.7)	7.5	(4.7)	7.6	(4.7)
Mother has cell phoneᵻ	3920	49.1	828	51.8	790	49.5	5538	49.5
Mother has NICᵻᵻ	3708	46.5	791	49.5	733	46.0	5232	46.9
Mother's Education (years) mean (sd)	7.5	(4.6)	7.6	(4.6)	7.6	(4.6)	7.6	(4.6)
Number of children delivered by mother mean (sd)	2.6	(1.6)	2.6	(1.6)	2.7	(1.6)	2.6	(1.6)
Father's Occupation‡
Employed	7837	98.6	1567	98.7	1571	99.0	10,975	98.7
Unemployed	112	1.4	21	1.3	16	1.0	149	1.3
Mother Occupation‡‡
Employed	88	1.1	9	0.6	6	0.4	103	0.9
Unemployed	7893	98.9	1589	99.4	1591	99.6	11,073	99.1
Ethnicity
Muhajir	5131	64.2	1052	65.8	1009	63.1	7192	64.2
Punjabi	850	10.6	167	10.4	186	11.6	1203	10.7
Sindhi	425	5.3	80	5.0	83	5.2	588	5.2
Pashtun	432	5.4	74	4.6	75	4.7	581	5.2
Other	1160	14.5	227	14.2	246	15.4	1633	14.6
Mode of Transport to Clinic
Taxi/Rickshaw	1794	22.4	347	21.7	354	22.1	2495	22.3
Personal Vehicle	3563	44.5	731	45.7	712	44.5	5006	44.7
On foot	2369	29.6	468	29.2	482	30.1	3319	29.6
Other	272	3.4	54	3.4	51	3.2	377	3.4
Transport time (minutes) mean (sd)	11.2	(6.9)	11.3	(6.6)	11.1	(6.5)	11.2	(6.8)
Transport cost (PKR) mean (sd)	17.3	(39.9)	17.1	(39.2)	17.8	(42.2)	17.4	(40.1)
Cell Phone owner's Relationship to Child
Mother	1563	19.5	283	17.7	264	16.5	2110	18.8
Father	5762	72.0	1172	73.2	1196	74.8	8130	72.6
Sibling	39	0.5	7	0.4	8	0.5	54	0.5
Grand Parent	299	3.7	57	3.6	62	3.9	418	3.7
Aunt/Uncle	333	4.2	80	5.0	69	4.3	482	4.3
Other	2	0.0	1	0.1	-	0	3	0.0
NIC owner's Relationship to Child
Mother	285	6.6	38	5.1	33	4.7	356	6.2
Father	3648	84.1	649	87.9	618	87.2	4915	85.0
Sibling	12	0.3	4	0.5	3	0.4	19	0.3
Grand Parent	207	4.8	27	3.7	30	4.2	264	4.6
Aunt/Uncle	179	4.1	19	2.6	22	3.1	220	3.8
Other	6	0.1	1	0.1	3	0.4	10	0.2

Sd: Standard Deviation, 137 Pakistani rupees (PKR)=1USD,^ 57 missing values,^^ 20 missing values, ᵻ 19 missing values, ᵻᵻ 39 missing values, ‡ 73 missing values, ‡‡ 21 missing values.

Data from electronic records showed that the program was implemented with fidelity. We put processes in place to monitor if the interventions (SMS reminders and mCCTs) were successfully delivered and received by caregivers. A total of 85,587 SMS reminders, 16,490 airtime transfers and 3291 mobile money payments were successfully made during the study. Out of the unique caregivers eligible for airtime and mobile money transfers, 0.3% (16/5066) could not be incentivized in the airtime arms due to incompatible numbers and 14.4% (184/1275) could not be incentivized in the mobile money arm due to the unavailability of NIC. Of all eligible caregivers, 83.9% (8050/9598) reported receiving at least 1 SMS reminder, 78.4% (855/1091) reported receiving at least one mobile money payment and 82.9% (4185/5050) reported receiving at least one airtime payment. Only 77% of those receiving mobile money encashed it by the end of the study (for further detail see supplemental material).

FIC was 62.3% (4980/7998) for participants receiving any mCCT, compared to 58.4% (934/1600) for the SMS arm (adjusted odds ratio [OR]:1.18, 95% CI: 1.05-1.33, p = 0.005) (Table 2). Effects are similar for up-to-date coverage of penta-3 (OR: 1.17, 95% CI: 1.02-1.33; p = 0.022), measles-1 (OR: 1.19, 95% CI: 1.06-1.34; p = 0.003) and measles-2 (OR: 1.25, 95% CI: 1.12-1.40; p < 0.001). However, mCCTs did not have a statistically significant impact on timeliness of penta-3 (OR:1.13, 95% CI: 0.99-1.30; p = 0.073), measles-1 (OR: 1.06, 95% CI: 0.90-1.25; p = 0.463) and measles-2 (OR: 0.96, 95% CI: 0.81-1.14; p = 0.625) compared to SMS arm.

Table 2

Full Immunization Coverage (FIC) at 12 months, timeliness, and up-to-date immunization coverage for children at 18 months in (a) SMS versus control arm, and (b) any mCCT versus SMS arm adjusting for risk variables using one step lasso (n = 11,197).

Outcome	Control (n = 1599)			SMS (n = 1600)						Any mCCT (n = 7998)
	Vaccinated/n	Coverage (%)	Odds	Vaccinated/n	Coverage (%)	Odds	Unadjusted Odds Ratio	Adjusted Odds Ratio (95% CI)*	p-value	Vaccinated/n	Coverage (%)	Odds	Unadjusted Odds Ratio	Adjusted Odds Ratio (95% CI)*	p-value
FIC (12 months)	887/1599	55.5	1.25	934/1600	58.4	1.40	1.13	1.16 (1.00-1.35)	0.046	4980/7998	62.3	1.65	1.18	1.18 (1.05-1.33)	0.005
Pentavalent-3 received (18 months)	1146/1599	71.7	2.53	1207/1600	75.4	3.07	1.21	1.26 (1.07-1.49)	0.006	6270/7998	78.4	3.63	1.18	1.17 (1.02-1.33)	0.022
Measles-1 received (18 months)	971/1599	60.7	1.55	1014/1600	63.4	1.73	1.12	1.14 (0.98-1.32)	0.080	5389/7998	67.4	2.07	1.19	1.19 (1.06-1.34)	0.003
Measles-2 received (18 months)	630/1599	39.4	0.65	686/1600	42.9	0.75	1.15	1.19 (1.03-1.38)	0.020	3877/7998	48.5	0.94	1.25	1.25 (1.12-1.40)	<0.001
Pentavalent-3 received timely	632/1149	55.0	1.22	676/1209	55.9	1.27	1.04	1.14 (0.95-1.36)	0.160	3763/6301	59.7	1.48	1.17	1.13 (0.99-1.30)	0.073
Measles-1 received timely	733/992	73.9	2.83	802/1028	78.0	3.55	1.25	1.29 (1.05-1.59)	0.017	4323/5461	79.2	3.80	1.07	1.06 (0.90-1.25)	0.463
Measles-2 received timely	431/723	59.6	1.48	521/758	68.7	2.20	1.49	1.53 (1.23-1.90)	<0.001	2894/4245	68.2	2.14	0.97	0.96 (0.81-1.14)	0.625

Adjusted odds ratio calculated from a logit regression.

Table 3 shows size and certainty of payment mattered for FIC and timeliness. Participants in the high payment arm had higher FIC than participants with low payment (OR:1.16, 95% CI: 1.04-1.29; p = 0.007. Those in the certain payment and airtime arms had higher FIC compared to those in the lottery (OR: 1.30, 95% CI: 1.17-1.45; p < 0.001) and mobile money arm (OR: 1.17, 95% CI: 1.01-1.36; p = 0.043) respectively. Comparison of FIC across payment schedules (sharp versus flat) did not show statistically significant differences (OR:1.03, 95% CI: 0.93-1.15; p = 0.550). The effects were similar for up-to-date penta-3, measles-1, and measles-2 coverage at 18 months. There was no statistically significant difference in the timeliness of penta-3, measles-1, and measles-2 between the mCCT arms.

Table 3

Full Immunization Coverage (FIC) at 12 months, timeliness, and up-to-date immunization coverage for children at 18 months by mCCT schedule (sharp or flat progressivity), amount (high or low), design (certainty of payment) and mode of mCCT (airtime or mobile money) adjusting for risk variables using one step lasso (n = 11,197).

	Flat			Sharp
	n = 3200			n = 3198
Outcome	#Vaccinated/n	Coverage (%)	Odds	#Vaccinated/n	Coverage (%)	Odds	Adjusted Odds Ratio (95% CI)*	p-value	p-value Adjusted for Multiplicity
FIC (12 months)	1989/3200	62.2	1.64	2018/3198	63.1	1.71	1.03(0.93-1.15)	0.550	0.528
Pentavalent-3 received (18 months)	2495/3200	78.0	3.54	2541/3198	79.5	3.87	1.09(0.96-1.24)	0.160	-
Measles-1 received (18 months)	2159/3200	67.5	2.07	2165/3198	67.7	2.10	1.01(0.90-1.12)	0.936	-
Measles-2 received (18 months)	1581/3200	49.4	0.98	1573/3198	49.2	0.97	0.99(0.89-1.10)	0.853	-
Pentavalent-3 received timely	1513/2512	60.2	1.52	1500/2550	58.8	1.43	0.94(0.84-1.07)	0.359	-
Measles-1 received timely	1710/2191	78.1	3.56	1749/2192	79.8	3.95	1.11(0.96-1.27)	0.166	-
Measles-2 received timely	1162/1728	67.3	2.05	1191/1721	69.2	2.25	1.10(0.95-1.28)	0.188	-

Adjusted odds ratio calculated from a logit regression.

Adjusting for multiplicity in mCCT design approaches tested increases p-values marginally but results remain broadly similar (airtime is no longer statistically significantly different from mobile money at the 5% level with a p-value of 0.067, OR: 1.17, 95% CI: 1.01-1.36).

Comparing FIC across the 7 study arms (Figure 3) shows the highest coverage rates are found in the high payment, flat rate arm, 64.2% (1027/1600) with an odds ratio of 1.30, (95% CI: 1.12-1.52; p = 0.001) and the high payment, sharp rate arm, 63.6% (1017/1598) with an odds ratio of 1.28 (95% CI: 1.10-1.48; p = 0.002)

Figure 3

Full Immunization Coverage (FIC) at 12 months by (a) SMS versus control arm, and (b) 5 mCCT arms versus SMS arm adjusting for risk variables using one step lasso (n = 11,197).

Serosurvey: we collected 96.1% (1615/1680) of our target sample between November 15, 2018, and March 21, 2020, while the rest could not be collected due to the nationwide COVID-19 lockdown imposed on March 23, 2020. Seropositivity results showed that the range of differences (between the study coverage estimates and seropositivity results) in the control and any mCCT arm were similar (Supplementary Table 1). The study coverage estimates and seropositivity rates differed by 1.1 ppt in the control arm and 2.3 ppt in any mCCT arm for measles-1. For penta-3, this difference was 13.5 ppt and 10.3 ppt in the control and any mCCT arm respectively. The serosurvey was not powered to test individual study hypotheses.

Cost per additional immunization

Administrative costs of USD 0.05-0.08 per transfer are much lower than traditional CCTs. The cost to the program administrator is USD 30 (in 2020 USD) per additional fully immunized child in the most effective arm (low, sharp, certain) based on cost analysis. The largest component of this cost is a transfer and thus a benefit to participants. Including participants and government costs and benefits, the cost per additional fully immunized child falls to USD 22, most of which is the cost of additional vaccine administration. If, as the Pakistan Government claims, they already supply enough vaccines and vaccinators to immunize 100% of every birth cohort, then the marginal cost to the government of higher vaccination demand is zero and the cost per fully immunized child is just USD 8. Program costs per additional immunization are higher for early vaccines (highest being USD 29 for penta-1) because most payments go to those who would be immunized without the mCCT and this proportion is lowest for the second dose of measles (USD 3). If we include the benefits and costs of participants, the cost per additional immunization is similar for low (USD 23) vs high (USD 24) payments (details in supplemental material).

Discussion

Our results show a small mCCT (USD 0.60-1.80 per immunization visit) delivered through a platform that can easily be scaled in low resource settings (like Pakistan) with low administrative costs can increase immunization uptake by as much as 6 ppt. However, the design details matter. Adopting the most effective delivery method (airtime payments) and structure (certain payment) increases FIC as much as or more than shifting from a low (∼USD 4) to a high amount (∼USD 11) i.e. nearly tripling the size of payment.

Lotteries have become an increasingly popular way to encourage immunization and other health behaviors. Across virtually all payment amounts and schedules, we find small certain payments have a larger impact on FIC and are more effective at cost per additional immunization than the chance to win a bigger payment (a result consistent with prospect theory and surveys of attitudes towards incentives for health behavior in high-income countries). The magnitude is large: on average, lottery payments reduce take-up by 5.5 ppt (OR: 1.30, 95% CI:1.17-1.45; p < 0.001) compared to certain payments of the same expected value.

Airtime has a 3.4 ppt larger impact on FIC than mobile money in our study (although the result is not robust to multiplicity adjustment and should be considered exploratory), even though mobile money is widely accepted across Pakistan, is the primary mechanism for delivering government cash transfers, and is more flexible than airtime (which can only be used for talk time, SMS and data). We postulate that the real-time receipt of airtime increases its value compared to mobile money that has an additional burden of verification (biometric verification and presentation of NIC).

On average, larger payments (USD 1.80 per visit) led to higher FIC than lower payments (USD 0.6 per visit). Still, the difference was relatively modest (2.6ppt) and similar to the difference between airtime and mobile money. The finding supports the hypotheses of diminishing returns to payment size. The low mCCT arms had lower program cost per additional fully immunized child than higher mCCT arms. The best performing low mCCT arm cost USD 30 per additional fully immunized child versus USD 127 for the best performing high mCCT arm. Once government and beneficiary costs and benefits are included, the cost-per additional immunization is almost identical (USD 23 versus USD 24). An implementer who values participants' income might choose a higher mCCT amount with a resulting higher vaccination rate. In contrast, resource-constrained implementers might choose a small transfer and cover more children. Our findings are also in line with the results of a prepublication parallel study conducted in India, which also concluded that low mCCT amounts and increasing payments over the immunization schedule were the most cost-effective combination and the most effective at increasing coverage rates (when combined with social networking interventions).

Further research is needed to understand why weighting payments towards the end of the schedule has differential effects depending on the size of the mCCT, but reviewing our and the parallel study's results suggest merit in pursuing sloped payments. Further research may also be needed to investigate whether vaccine hesitancy or limited access to health care could explain why not all participants responded to mCCTs. The impact of immunization mCCTs on other health seeking behavior also needs further research. By driving additional visits to clinics, mCCTs for immunization could encourage use of other clinic services. Alternatively, mCCTs could reduce utilization of other services if caregivers end up prioritizing immunization over other health activities.

The effect of SMS reminders alone on improving the timeliness of vaccines in our study is consistent with the existing literature which highlights the utility of reminders for later vaccines administered when there are larger gaps between scheduled visits. The low cost of SMS reminders means they are cost-effective even if they induce small (and thus hard to detect) changes in behavior.

There is an ethical debate about whether tying immunization to cash rewards is coercive and that conditional cash transfers designed to reduce poverty risk excluding the most marginalized (who are unable to meet the conditions). Small mCCTs helps address both points: participants are unlikely to take action they strongly oppose for a small mCCT and small mCCTs do not impose prohibitive conditionalities. For instance, at least in the local Pakistani context, for large CCTs, receipt of hard cash is linked to valid NICs which are not available to the most vulnerable population segments in the country. In contrast, small mCCTs in the form of airtime reach a much higher proportion of the most vulnerable (only 7.5% of those screened did not have access to a mobile phone while 48.3% of those enrolled in the study did not have a valid NIC needed to access mobile money).

The Government of Pakistan launched an unconditional cash transfer program, the Benazir Income Support Program (BISP), in 2008 to provide a financial cushion to women below a poverty threshold. Now called Ehsaas, the Program has expanded as an umbrella initiative to address poverty and inequality, focusing on human capital formation. Our findings suggest tying a small part of the payment to childhood immunizations would boost immunization while ensuring the marginalized who fail to meet conditions continue to receive some transfer. For countries or populations, including many in Pakistan, not covered by income support programs, our findings suggest small mCCTs can substantially increase immunization at low cost.

Our study also demonstrates how EIRs (which are increasingly popular even in LMICs) can be leveraged to automatically deliver small mCCTs at a large scale with little additional administrative burden. A provincial Government EIR is in use throughout Sindh, and pilot projects in other provinces have generated interest in a nationwide EIR, which could provide a platform for large scale implementation of small mCCTs.

Our study has limitations. Firstly, as we used individual randomization among those attending immunization clinics, we could not advertise the existence of mCCTs to those not reached by immunization services or test the impact of mCCTs on this group. Other studies have found community-level information on mCCTs increases their impact, and thus the program might have a larger impact at scale with broader communication. Secondly, out of those eligible for mCCTs, 14.4% could not be incentivized in the mobile money arm due to the unavailability of NICs, and 0.3% could not be incentivized in the airtime arms due to incompatible mobile subscriptions (only pre-paid mobile connections could be sent airtime). Lastly, some children may have been vaccinated outside of the study province or through door-to-door campaigns which do not use the EIR. Resultantly, data on these vaccinations may not be part of study coverage estimates. Our analysis suggests mCCTs had a large impact even with these limitations. Our eligibility criteria of cell phone access, enrollment of children at immunization clinics as well as reliance on an EIR to administer small mCCTs mean our results will generalize best to LIC and MIC settings with similar conditions. In Pakistan, 94% of households have access to a cell phone, 96% of children receive at least one vaccine and EIRs are rapidly being rolled out. In LMICs, mobile phone penetration has exceeded 90% in recent years, 85% of children receive at least one vaccine and the use of EIRs is rapidly expanding (they are now present in 50 countries at varying scales).

Small mCCTs (USD 0.6-1.8 per immunization visit) improve both immunization coverage and timeliness with a cost as low as USD 23 per additional fully immunized child, in LMICs like Pakistan. We find design details matter more than the size of mCCT, with certain payments and airtime substantially outperforming lottery payments and mobile money transfers. From a policy perspective, programs should explore strategies to introduce small mCCTs for health, or make part of existing cash transfers in LMICs conditional on immunization, as an effective policy tool to improve immunization and overall health outcomes for children.

Contributors

SC, AJK and RG conceptualized the study. SC and RG designed the study with input from ED. SC, AJK and RG acquired funding. SC, RG, and DAS developed the methodology. SC supervised the changes in EIR software. RG generated the randomization sample. DAS and SA implemented the project and curated the data under supervision of SC. ED and RG conducted the statistical analyses with input from SC and DAS. SC, DAS, and RG interpreted the data and wrote the original draft. All authors contributed to and reviewed the final submitted manuscript. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Data sharing statement

Deidentified participant data and data dictionary are available to any researcher under reasonable request. To facilitate the data access process, please contact mch@ird.global.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.