Vaccination timeliness and delay in low- and middle-income countries: a systematic review of the literature, 2007-2017.

Background: Traditional measurements of vaccine coverage at specific ages can mask poor vaccine timeliness. However, optimal measurement of timing is unclear due to variations in countries' recommended vaccination schedules and lack of a commonly accepted standard for "timeliness". We conducted a systematic review of literature on vaccine timeliness and delay in low- and middle-income countries from 2007 to 2017.
Methods: A search of articles published between January 1 2007 and December 31 2017, was performed in PubMed, EBSCOhost, and Embase.
Results: 67 papers were included, of which 83% used a categorical measure of delay and 41% evaluated continuous delay. The most common age at assessment was 1 month, with earlier age benchmarks typically used with birth doses.Conclusions: Categorical definitions of vaccination timing vary widely, with benchmarks of delay varying from days to weeks to months. Use of a continuous measure of vaccine delay may be more informative and comparable.

Introduction

Vaccines have proven to be one of the most effective preventive interventions and remain one of public health’s most successful means for controlling and eradicating serious and sometimes fatal diseases.[1] Since 1974, the World Health Organization (WHO) has promoted vaccines through the Expanded Program on Immunization (EPI), which includes recommendations for countries to publicly fund select vaccines.[2] The initial EPI-recommended pediatric vaccines were Bacillus Calmette-Guérin (BCG), oral polio (OPV), diphtheria-tetanus-pertussis (DTP), and measles vaccine, although the list has since expanded to include hepatitis B, Haemophilus influenzae type b (Hib), rubella, pneumococcal conjugate vaccine (PCV), and rotavirus vaccine.[2] Most countries have a National Immunization Technical Advisory Group (NITAG) which provides country-specific recommendations, such as which vaccines are offered, whether combination vaccines are used, and when vaccines should be administered.[3]

Widespread use of these vaccines has had a lasting impact on the occurrence of vaccine-preventable diseases. The incidence of most vaccine-preventable diseases in the US has declined >99% over the twentieth century.[4] Worldwide, we are on the verge of polio and measles eradication, due in part to the widespread adoption of EPI programs.

Vaccination program performance has typically been assessed through measurements of vaccine coverage: the proportion of individuals who have received a vaccine by a benchmark age (such as 5 years), regardless of the timing of administration.[5] However, vaccination coverage and timeliness of vaccine administration are related but separate issues, and high levels of coverage can sometimes mask low levels of timeliness. For example, an analysis of the Vaccine Safety Datalink, a consortium of 10 healthcare delivery organizations in the United States, shows that although vaccination coverage is high (>90%),[6,7] less than half of the children had received all vaccine doses on time.[6] Untimely vaccination could relate to access or affordability, or could instead be the result of the parent’s not accepting the recommended vaccine schedule and being vaccine hesitant.[8,9]

Studies measuring vaccine timeliness have become more frequent in the literature in recent years. In 1998, Bolton et al. explored issues with vaccination coverage estimates within Baltimore, Maryland, and recommended the use of age-appropriate indicators,[10] and a 2007 article[11] and 2009 editorial[12] both encouraged more regular measurement of vaccine timeliness. Subsequently, additional studies have been conducted focusing on low- and middle-income countries (LMICs). LMICs have a disproportionately high burden of vaccine-preventable diseases, which are often more serious in younger infants so that administration of vaccines at an early age (as scheduled) is especially important.[13] Additionally, children in LMICs are more likely to be non- and under-vaccinated and encounter significant barriers to vaccination access, magnifying the challenges faced in vaccination receipt and delivery. However, the optimal measure of timeliness remains unclear, resulting from both differences in schedules among different countries, but more so by confusion as to what constitutes “timeliness” relative to vaccination date recommendations. There currently is no agreed-upon definition of timeliness across international bodies.

This systematic review presents a summary of the literature on vaccine timeliness and delay in low- and middle-income countries between the years of 2007 and 2017, with a focus on timeliness definition, age at assessment, countries/setting of research, and analytic approach, among others. It is the first paper, to the author’s knowledge, which seeks to evaluate variable ways of measuring and quantifying vaccine timing in LMICs. This manuscript serves to describe the findings of previously conducted studies and to provide recommendations for future studies measuring vaccination timing.

Methods

This systematic review followed guidelines from the Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA).[14]

Search strategies

Searches were performed in 3 different electronic databases: PubMed/MEDLINE, EBSCOhost, and Embase. The search terms used were (“vaccination” or “immunization” or “immunisation” or “vaccine” or “EPI”) and (“timely administration” or “timeliness” or “on time”). The searches were limited to papers published between January 1, 2007 and December 31, 2017.

The titles of all papers returned through the use of the search terms were initially screened for relevance. The abstracts of all remaining papers were then assessed with specific inclusion and exclusion criteria in mind. Manuscripts were included if they met the following inclusion criteria: (1) studies were conducted on data from LMICs, as defined by the World Bank;[15] (2) studies examined pediatric vaccines that were a part of the WHO EPI[2] – studies only featuring adolescent and adult vaccines (e.g. human papillomavirus vaccine (HPV)) or vaccines not on the EPI (e.g. influenza vaccine) were excluded; (3) studies had to calculate some measure of timeliness – broadly defined as a vaccination measure which was age-specific; (4) the study had to provide some analysis about vaccination timeliness – studies presenting only descriptive statistics were excluded; comparisons between groups with or without statistical testing was required and, (5) the study had to be a non-randomized-controlled trial exploring vaccination outcomes, as RCTs test an intervention to change vaccine administration rather than assess vaccination timeliness. Finally, non-English studies were excluded.

Vaccination timing definitions

We use the following definitions: vaccine uptake refers to any vaccine measurement, vaccine coverage is the vaccine uptake measured without regard to any timing measure, and vaccination timing is vaccine uptake with some measurement of time or date. Vaccination timing could refer to one of the three conditions: (1) up-to-date vaccination means an individual received a vaccine by a certain threshold age, (2) vaccination delay is the continuous measurement of vaccination – either in the age of vaccination, or the time elapsed since the vaccine was recommended to be administered, and (3) vaccination timeliness refers to vaccination administered within a certain time since a recommended age of vaccination.

Study selection

NBM removed all duplicates and assessed all titles for relevance. Two reviewers (NBM/ALW) independently assessed all abstracts and full-text publications for eligibility using the eligibility criteria laid out. All disagreements were resolved by discussion between reviewers, and the tie-break of a third reviewer. In addition to assessment for relevance, data were also extracted independently and a quality assessment was conducted by the two reviewers (NBM/ALW). The following data were abstracted: location of study; study population; study design; sample size; study year; vaccines considered; definition of timeliness/vaccine delay; explanatory variables considered. Risk of bias was assessed using an adaptation of the Downs & Black checklist,[16] which had previously been used in a systematic review of vaccine uptake.[17] Items relating to interventions (Question 4, 8, 13–15, 19, 20, 23, 24, and 26) were dropped as they were not relevant to our inclusion criteria. As a result of eliminating the aforementioned questions, the maximum quality score was an 18. Studies were considered to be good quality if they achieved a score of 14–19, Moderate quality if they achieved a score of 9–13, and low quality if they achieved a score of <9. Further information regarding the quality assessment can be found in the Supplementary Materials.

Study quality and synthesis of findings

The goal of this review was to examine how researchers define and analyze vaccine timeliness. We present a description of the current state of studies on vaccination timeliness, and discuss the temporal and geographical distribution, the study and analytical design, the definition of timeliness, and the inter-group comparisons of timeliness.

Ethical approval

As a systematic review of previously published manuscripts, this study was not under the purview of studies examined by the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board.

Results

Overview of included papers

The search terms yielded 488 articles for screening from PubMed, 388 papers from EBSCOhost, of which 16 were duplicates from the PubMed search, and 636 papers from Embase, of which 271 were unique non-duplicates with either PubMed or EBSCOhost. After the title, abstract, and full paper screen, 67 papers were included in the systematic review.[18-84] Full details regarding paper selection are found in Figure 1.

Figure 1.

Paper selection scheme.

Definition

Studies measuring timing were generally of two types: those using continuous vs. categorical measures, although some papers used both. Only one study[75] did not explicitly evaluate vaccine timing on either a continuous or categorical scale, but rather just measured whether children were up-to-date by a threshold date of 14 weeks. Of the 66 remaining studies, 27 (41%) papers used a continuous outcome such as median delay in days or weeks: for example, Hughes et al. graphed cumulative incidence curves of age at vaccination for BCG, pentavalent, and polio vaccine series.[39] More simply analyzed, Wagner et al. present the mean age at measles vaccination for a number of different demographic groups.[76] The majority of papers (55/66, 83%) used some categorical measure of timeliness. More than half of these papers (30/55, 55%) used variable definitions for vaccine timeliness based on the vaccine in question (i.e. different definitions for measles vs. DTP vs. HepB), whereas (21/55, 38%) used a fixed definition of timeliness for all considered vaccines. Only 7% of papers (4/55) used an interval definition between doses to determine timeliness.

The most common age benchmark used to assess timeliness was around 1 month. All but three (18/21, 86%) of papers which used a fixed definition employed this timeframe, operationalized as 1 month [9 papers], 4 weeks [3 papers], 28 days [3 papers], 4.3 weeks [1 paper] and within the same month as recommended [2 papers]. Additionally, there were a few papers which used an age benchmark for timely vaccination of around 1 month for all vaccines other than Hepatitis B vaccine, which had a timeliness definition of 1 day [2 papers]. Other age benchmarks used were 1 day (4/55, 7%), though 3 of these papers were combined with longer delay definitions for other vaccines, 1 week (3/55, 5%) or 2 weeks (6/55, 11%). Lower benchmarks (e.g. 1 day, 1 week) were typically used with birth doses (HepB, BCG, OPV0). Most papers did not explain why specific vaccination benchmarks were selected; those that did indicated that the recommendations derived from various sources, including manufacturer’s instructions[77] and/or used the WHO definition of full vaccination according to scheduled vaccination dates, not definitions of timeliness.[52] Finally, 4 papers (7%) evaluated timeliness in terms of the interval which elapsed between doses rather than the date of vaccination itself, which would result in ‘timely’ vaccination calculations if the first vaccine in the series was initiated late but all subsequent vaccines were given within the recommended interval.[21,23,73,84]

Country under study

The number of relevant studies published by year increased from 1 in 2007 to 9 in 2017, with noticeably more papers examining vaccine timing per year since 2014 (Figure 2). Overall, the largest represented WHO region was the African region (29/67, 43%). The next most represented WHO region was the Western Pacific region (19/67, 28%) with 68% of these papers (13/19) focused on China. Considerably fewer studies were conducted in the Region of the Americas (10/67, 15%), the South East Asian Region (8/67, 12%), and the Eastern Mediterranean Region (8/67, 12%). Finally, only 3 studies (4%) were conducted in the European Region (Table 2). Studies used data from 79 different countries (Table 3), with the most common countries studied being China (13/67, 19%) and Kenya (8/67, 12%). Most studies (53/67, 79%) were not nationally representative and instead involved samples from a particular district or city, 10 (15%) used nationally representative data from a single country, and 4 (6%) studies used data from multiple countries. There was no discernible pattern of specific countries as more common sources of data over the study period from 2007 to 2017.

Figure 2.

Number of papers included by year of publication, 2007–2017 (n = 67).

Table 2.

Number of papers by WHO region.

WHO Region	Number of Papers
African Region	29
Western Pacific Region	19
Region of the Americas	10
South East Asian Region	8
Eastern Mediterranean Region	8
European Region	3

Table 3.

Number of included papers by country of study.

Country	Number of Papers
China	13
Kenya	8
Burkina Faso, Ghana, India	5
Gambia, Iran, Nigeria	4
Bangladesh, Brazil, Cameroon, Malawi, Senegal, Tanzania, Uganda	3
Colombia, Cote d’Ivoire, Democratic Republic of Congo, El Salvador, Guatemala, Guinea-Bissau, Haiti, Kazakhstan, Kyrgyzstan, Mauritania, Pakistan, Peru, Sierra Leone, Togo, Uzbekistan, Yemen	2
Albania, Armenia, Belarus, Belize, Benin, Bolivia, Bosnia and Herzegovina, Burundi, Cambodia, Chad, Comoros, Djibouti, Dominican Republic, Egypt, Eritrea, Federated States of Micronesia, Gabon, Guinea, Guyana, Honduras, Iraq, Jamaica, Laos, Lesotho, Macedonia, Madagascar, Mali, Mongolia, Montenegro, Morocco, Mozambique, Namibia, Nepal, Nicaragua, Niger, Papua New Guinea, Philippines, Rwanda, Serbia, South Africa, Syria, Thailand, Trinidad and Tobago, Turkey, Venezuela, Vietnam, Wallis and Futuna, Zambia	1

Antigens

The vaccines assessed are shown in Figure 3. The most common vaccine was BCG (38/67, 57%), followed by the measles-containing vaccine (33/67, 49%), and OPV or IPV (19/67, 28%), though some studies examined multiple vaccines (Table 1). Timeliness of DTP was assessed in 13 papers (19%) and the pentavalent vaccine was explored in 6 papers (9%). However, only 21 papers (31%) examined full vaccination (i.e. receipt of all doses of all recommended EPI vaccines) as a primary or secondary study outcome measure, with most of the surveyed papers instead addressing a specific vaccination or a combination of vaccinations, but less than full vaccination, as their study measures.

Figure 3.

Number of papers assessing the timeliness of different antigens as primary or secondary outcome measures (n = 67).

Table 1.

Summary of included papers, quality assessment, and key details related to vaccine delay.

Authors & Year	Quality Score	Country (ies) of Study	Nationally representative	AgeRange	Survey Type	Analysis	Antigens	Delay Definition	Predictors
Akmatov, MK, Mikolajczyk RT. 2012.[18]	Moderate	Burkina Faso, Burundi, Cameroon, Cote d'Ivoire, Djibouti, Gambia, Ghana, Guinea-Bissau, Malawi, Mauritania, Sierra Leone, Togo, Belarus, Kazakhstan, Albania, Bosnia and Herzegovina, Macedonia, Montenegro, Serbia, Belize, Guyana, Jamaica, Trinidad and Tobago, Bangladesh, Lao People’s Democratic Republic, Thailand, Vietnam, Mongolia, Iraq, Syria, Yemen	Yes	12-59 months	Cross-sectional	Kaplan-Meier	BCG, Polio, DPT, MCV	4.3 weeks, continuous delay	SES, Child Gender, Urban/Rural Environment, Household Size
Akmatov, MK et al. 2008.[19]	Moderate	Armenia, Kazakhstan, Kyrgyzstan, Uzbekistan	Yes	12-59 months,0-35 months	Cross-sectional	Kaplan-Meier, Logistic Regression	DPT, MCV	1 month	Maternal Education, Paternal Education, Child Gender, Urban/Rural, Household Size
Babirye, JN et al. 2012.[20]	Good	Uganda	No	10-23 months	Cross-sectional	Kaplan-Meier, Cox Proportional Hazards	BCG, Polio, PENTA, MCV	variable	Birth Setting, Maternal Education, Number of Children, Paternal Age, Maternal Age, Marital Status, SES
Barman, D, Dutta, A. 2013.[21]	Moderate	India	No	12-23 months	Cross-sectional	Logistic Regression	BCG, Polio, DPT, MCV, Full Vaccination	within same month, interval	Religion, Birth Setting, Maternal Education, SES, Child Gender, Birth Order, Village Electrified, Equipment in Health Center, Drugs in Health Center…
Barman, M et al. 2015.[22]	Good	India	No	12-36 months	Cross-sectional	Kaplan-Meier, Cox Proportional Hazards	BCG, Polio, DPT, MCV	continuous delay	Maternal Education, Paternal Education, Mother's Age, Caste, Child Gender, Maternal Occupation, Paternal Occupation, SES
Bicaba, A et al. 2009.[23]	Poor	Burkina Faso	Yes	unclear	Cross-sectional	Cumulative-Age-At-Vaccination Curves	BCG, DTP, MCV, Full Vaccination	interval, continuous delay	Region
Calhoun, LM et al. 2014.[24]	Good	Kenya	No	12-23 months	Cross-sectional	Logistic Regression	BCG, Polio, PENTA, MCV, Full Vaccination	within same month	Maternal Education, Paternal Education, Child Gender, Mother's Age, Birth Order, Household Size, Distance to Clinic, Maternal and Paternal Occupation, Mother & Father Alive, Mother with child at interview, Mother looks after Child, Paternal Age, Orphan Status…
Chiabi, A et al. 2017.[25]	Good	Cameroon	No	0-11 months	Cross-sectional	Logistic Regression	BCG, Polio, PENTA, MCV, PCV, Rotavirus, Full Vaccination	2 weeks	Religion, Maternal Education, Mother's Age, Marital Status, Distance to the Clinic, Paternal Education, Maternal Occupation, Paternal Occupation
Clark, A, Sanderson C. 2009.[26]	Moderate	Bangladesh, Benin, Bolivia, Brazil, Burkina Faso, Cambodia, Cameroon, Chad, Colombia, Comoros, Congo, Côte d’Ivoire, Dominican Rep, Egypt, Eritrea, Gabon, Ghana, Guatemala, Guinea, Haiti, Honduras, India, Kenya, Kyrgyz, Lesotho, Madagascar, Malawi, Mali, Mauritania, Morocco, Mozambique, Namibia, Nicaragua, Niger, Nigeria, Peru, Rwanda, Senegal, Tanzania, Togo, Turkey, Uganda, Uzbekistan, Yemen, Zambia	Yes	0-5 years	Cross-sectional	Cumulative-Age-At-Vaccination Curves	BCG, DPT, MCV	continuous delay	Birth Setting, Maternal Education, Child Gender, Mother's Age, Child Age, Urban/Rural, Birth Order
D’Ardenne, KK et al. 2016.[27]	Good	Guatemala and Peru	No	0-5 years	Cross-sectional	Log Binomial Regression, Cumulative-Age-At-Vaccination Curves	PENTA, MMR, Rotavirus	28 days	Maternal Education, Child Gender, Mother's Age, Child Age, Household Size, Food Insecurity, Birth Cohort
Delrieu, I et al. 2015.[28]	Moderate	Burkina Faso, Ghana, Kenya, Senegal, Tanzania	Yes	0-5 years	Cross-sectional	Kaplan-Meier	DPT, MCV	continuous delay	N/A
Ettarh, RR et al. 2012.[29]	Moderate	Kenya	No	9–59 months	Cross-sectional	Kaplan-Meier, Cox Proportional Hazards	MCV	1 month	Ethnicity, Maternal Education, SES, Child Gender, Village
Fadnes, LT et al. 2011.[30]	Good	South Africa	No	0-2 years	Cluster RCT (non-vaccination outcome)	Kaplan-Meier, Cox Proportional Hazards	BCG, Polio, PENTA, Full Vaccination	variable	Birth Setting, Maternal Education, SES, Household Size, Breastfeeding, Peer Counseling, Mother's BMI, Marital Status
Fadnes, LT et al. 2011.[31]	Good	Uganda	No	0-2 years	Cluster RCT (non-vaccination outcome)	Kaplan Meier, Cox Proportional Hazards	BCG, Polio, PENTA, MCV	variable	Birth Setting, Maternal Education, SES, Child Gender, Mother's Age, Urban/Rural, Household Size, Mother's BMI, Health Counseling
Fisker, AB et al. 2014.[32]	Good	Guinea-Bissau	No	12-23 months	Cohort	Rank Sum Test	BCG, PENTA, Polio, MCV, Full Vaccination	continuous delay	Ethnicity, Maternal Education, Child Gender, Mother's Age, Child Age, Introduction of new Vaccine, Mid-Upper Arm Circumference
Flannery, B et al. 2013.[33]	Moderate	Brazil	Yes	unclear	Cross-sectional	N/A	DTP-Hib, Rotavirus	variable	N/A
Gibson, DG et al. 2015.[34]	Moderate	Kenya	No	12-23 months	Cross-sectional	Log Binomial Regression	PENTA,MCV,Full Vaccination	1 month	Maternal Education, SES, Child Gender, Mother's Age, Child Age, Marital Status, Previous Child Died, Distance to Clinic,Maternal Literacy, Number of Children <5, Mother's Phone Ownership
Gram, L. et al. 2014.[35]	Good	Ghana	No	0-1 year	Cross-sectional	Cox Proportional Hazards	BCG,Polio, PENTA, MCV,Full Vaccination	Variable, continuous delay	Religion, Birth Setting, Maternal Education, SES, Child Gender, Mother's Age, Household Size, Distance to Clinic
Hu, Y et al. 2013.[36]	Moderate	China	No	8-48 months	Cohort	Kaplan-Meier, Logistic Regression	MCV	1 month	Maternal Education, SES, Mother's Age, Household Size, Mother's Occupation
Hu, Y et al. 2014.[37]	Good	China	No	2-3 years	Cross-sectional	Kaplan-Meier, Logistic Regression	HepB, Polio, DPT, MCV	Variable,continuous delay	Birth Setting, Maternal Education, SES, Child Gender, Mother's Age, Household Size, Distance to Clinic, Mother's occupation, Resident (migrant) Status, Caregiver Attitude towards Vaccination, Awareness of Immunization, Satisfaction with Immunization Service
Hu, Y et al. 2017.[38]	Good	China	No	24-35 months	Cross-sectional	Kaplan Meier, Cox Proportional Hazards	BCG, HepB, Polio, DPT, MCV, Full Vaccination	variable	Birth Setting, Maternal Education, SES, Household Size, Maternal Age, Urban/Rural, Immigration Status
Hughes, MM et al. 2016.[39]	Good	Nepal	No	0-6 months	RCT (non-vaccination outcome)	Kaplan Meier, Cox Proportional Hazards	BCG, Polio, PENTA	continuous delay	Ethnicity, SES, Child Gender, Child Age, Birth Order, Household Size, Gestational Age, Birthweight, Breastfeeding, Literacy
Iqbal, W et al. 2015.[40]	Moderate	Pakistan	No	0-2 years	Cross-sectional	Chi-Square (unadjusted)	BCG	variable	ANC, Birth Setting, Maternal Education, Paternal Education, SES, Awareness of EPI Schedule, Place of Vaccination
Jadidi, R et al. 2015.[41]	Moderate	Iran	No	24-47 months	Cohort	Chi-Square (unadjusted)	MMR	1 week	Maternal Education, Paternal Education, Child Gender, Urban/Rural, Birth Order, Iranian/Non Iranian
Laryea, D et al. 2014.[42]	Moderate	Ghana	No	unclear	Cross-sectional	N/A	BCG, Polio, PENTA, MCV	4 weeks	ANC, Birth Setting, Child Gender, Mother's Age, Child Age, Mode of Delivery
Le Polain de Waroux, O et al. 2013.[43]	Good	Tanzania	No	12-23 months	Cross-sectional	Log Binomial Regression; Kaplan-Meier	BCG, DPT, MCV	1 month	Ethnicity, Maternal Education, SES, Child Gender, Mother's Age, Child Age, Household Size, Distance to the Clinic, Mother's Occupation, Seasonality
Li, Q et al. 2014.[44]	Poor	China	No	1-7 years	Cohort	N/A	BCG, HepB, Polio, DPT, MCV, Full Vaccination	1 month	Immigration Status, Municipality, Birth Cohort
Lin, W et al. 2014.[45]	Moderate	China	No	9-24 months	Case-Control	Logistic Regression	MCV	1 month	Child Gender, Household Size, Distance to Clinic, Residential Status, Health Status of Child, Maternal and Paternal Occupation, History of Vaccine Delay, Awareness of Necessity of Timely Vaccination, Waiting Time in EPI Clinic…
Mbengue, MA et al. 2017.[46]	Good	Senegal	Yes	12-23 months	Cross-sectional	Kaplan-Meier, Cox Proportional Hazards	BCG, Polio, PENTA, MCV, PCV, Rotavirus, Full Vaccination	4 weeks	Maternal Education, SES, Child Gender, Urban/Rural, Marital Status, Birth Order, ANC
Miyahara, R et al. 2016.[47]	Good	Gambia	No	unclear	Cross-sectional	Logistic Regression	BCG, HepB, Polio	variable (1 day, 1 week)	Ethnicity, Maternal Education, SES, Child Gender, Mother's Age, Child Age, Birth Order, Birth Year, Birth Spacing, Birth Season
Moïsi, JC et al. 2010.[48]	Moderate	Kenya	No	unclear	Cross-sectional	Kaplan-Meier, Cox Proportional Hazards	BCG, Polio, PENTA, MCV	continuous delay	Ethnicity, Maternal Education, Child Gender, Distance to Clinic, Migration Status
Mokhtari, M 2015.[49]	Poor	Iran	No	24-47 months	Cohort	Kaplan-Meier	DPT	continuous delay	Ethnicity, Maternal Education, Paternal Education, Child Gender, Urban/Rural, Birth Order, City Location, Maternal and Paternal Occupation
Montgomery, JP et al. 2015.[50]	Good	China	No	8 months - 8 years	Cross-sectional	Logistic Regression	MCV	variable	Child Gender, Urban/Rural, Birth Year
Mutua, MK et al. 2015.[51]	Good	Kenya	No	unclear	Cohort	Kaplan-Meier, Cox Proportional Hazards	BCG	continuous delay	Ethnicity, Birth Setting, Maternal Education, Child Gender, Low Birth Weight, Pregnancy Intention, Type of Health Facility, Settlement Area
Mutua, MK et al. 2016.[52]	Good	Kenya	No	12-23 months	Cohort	Kaplan-Meier	Full Vaccination	Variable,continuous delay	Ethnicity, Birth Setting, Maternal Education, SES, Child Gender, Study Location
Mvula, H et al. 2016.[53]	Good	Malawi	No	>1 year	Cohort	Poisson Regression, Cox Proportional Hazards	MCV, PCV, Rotavirus	variable, continuous delay	Maternal Education, Child Gender, Mother's Age, Child Age, Marital Status, Household Size, Mother's Occupation, Orphanhood, Place of Birth, Distance to Clinic, Vaccination due in Rainy Season
Narváez, J et al. 2017.[54]	Good	Colombia	No	0-6 years	Cross-sectional	Mixed Models	BCG, HepB, Polio, DPT, PENTA, MMR, PCV, Rotavirus, Full Vaccination	continuous delay	Ethnicity, Maternal Education, SES, Child Gender, Mother's Age, Child Age, Household Size, Insurance, Displaced by Armed Conflict
Odutola, A et al. 2015.[55]	Good	Gambia	No	12-59 months	Cross-sectional	Logistic Regression	BCG, DPT, Polio, MCV	variable, continuous delay	Ethnicity, Birth Setting, Child Gender, Child Age, Marital Status, Birth Order, Family Type: Monogamous/Polygamous/Single Parent
Olusanya BO 2010.[56]	Moderate	Nigeria	No	0-3 months	Cross-sectional	Kaplan-Meier, Logistic Regression	BCG	variable	Religion, Ethnicity, ANC, Birth Setting, Maternal Education, SES, Child Gender, Mother's Age, Child Age, Marital Status, Birth Order, Maternal Occupation, Child has Jaundice
Pezzoli, L et al. 2017.[57]	Moderate	Wallis and Futuna	No	9-11 years	Cross-sectional	Kaplan-Meier	HepB	variable	N/A
Poorolajal, J. et al. 2012.[58]	Moderate	Iran	No	12-24 months	Cross-sectional	Kaplan-Meier	BCG, HepB, Polio, DPT, MMR	variable	ANC, Maternal Education, Child Gender, Urban/Rural, Distance to Clinic, Vaccinator's Education Level
Rainey, JJ. et al. 2012.[59]	Poor	Haiti	Yes	12-23 months	Cross-sectional	Rao-Scott (unadjusted)	BCG, Polio, DPT, MR, Full Vaccination	Variable,continuous delay	Vaccine Hesitancy Questions
Rejali, M et al. 2015.[60]	Moderate	Iran	No	24-47 months	Cross-sectional	Logistic Regression	BCG, HepB, Polio, DPT, MMR	variable	Maternal Education, Paternal Education, Child Gender, Urban/Rural, Birth Order, City, Nationality, Maternal and Paternal Occupation
Sadoh, EA, Eregie OC. 2009.[61]	Moderate	Nigeria	No	unclear	Cross-sectional	N/A	Full Vaccination	4 weeks, continuous delay	Child Gender, Mother's Age, Paternal Age
Sadoh, EA et al. 2016.[62]	Moderate	Nigeria	No	unclear	Cohort	Kaplan-Meier, Cox Proportional Hazards	BCG, DPT, PENTA	continuous delay	Birth Setting, Maternal Education, Child Gender, Mother's Age, Introduction of New Vaccine, Paternal Age
Sanchez, D et al. 2015.[63]	Moderate	Venezuela	No	0-6 years	Cross-sectional	Hazard-based estimator (unadjusted)	BCG, HepB, Polio, PENTA, MMR, Rotavirus, Full Vaccination	continuous delay	N/A
Sartori, AL et al. 2017.[64]	Good	Brazil	No	0-23 months	Cohort	Log Binomial Regression	PCV	28 days, continuous delay	ANC, Maternal Education, Mother's Age, Child Birthweight
Schoeps, A. et al. 2013.[65]	Good	Burkina Faso	No	12-23 months	Cross-sectional	Logistic Regression	BCG, PENTA, MCV, Full Vaccination	28 days	Religion, Ethnicity, Maternal Education, SES, Child Gender, Mother's Age, Urban/Rural, Household Size, Season of Birth
Scott, S et al. 2014.[66]	Moderate	Gambia	Yes	12-23 months; 9-60 months	Cross-sectional	Kaplan Meier, Cox Proportional Hazards	BCG, DPT, MCV, Full Vaccination	variable,continuous delay	Ethnicity, Region, Previous Vaccine Delay
Senessie, C et al. 2007.[67]	Moderate	Sierra Leone	No	0-35 months	Cross-sectional	Chi-Square (unadjusted)	Full Vaccination	variable	Child Age, Time Relative to Civil Conflicts
Shrivastwa, N. et al. 2016.[68]	Moderate	India	Yes	0-60 months	Cross-sectional	Turnbull Estimator	BCG, DPT, MCV	continuous delay	N/A
Suárez-Castaneda, E et al. 2014.[69]	Good	El Salvador	Yes	23-59 months	Cross-sectional	Logistic Regression, Cox Proportional Hazards	BCG, Polio, PENTA, MMR, Rotavirus, Full Vaccination	variable	Ethnicity, Maternal Education, Paternal Education, Child Gender, Urban/Rural, Marital Status, Household Size, Distance to Clinic, Parental Occupation, Region, Birth Year
Suárez-Castaneda, E et al. 2015.[70]	Good	El Salvador	Yes	24-59 months	Cross-sectional	Logistic Regression, Cumulative-Age-At-Vaccination Curves	PENTA, Rotavirus	continuous delay	Parental Education, Child Gender, Urban/Rural, Marital Status, Household Size, Distance to Clinic, Parental Occupation, Primary Mode of Transportation, Presence of Organized Crime, Region
Tang, X et al. 2016.[71]	Good	China	No	18-54 months	Cross-sectional	Kaplan-Meier, Logistic Regression	MCV	variable	Ethnicity, Maternal Education, SES, Child Gender, Mother's Age, Child Age, Household Registration, Guardian Occupation, Average Duration of Time Father (and Mother) is Away from Home, Residential Status of Mother, Measles Vaccination Status of Mother
Tang, X et al. 2017.[72]	Good	China	No	18-54 months	Cross-sectional	Kaplan Meier, Cox Proportional Hazards	MCV	continuous delay	Ethnicity, Child Gender, Child Age
Tippins, A et al. 2017.[73]	Moderate	Federated States of Micronesia	No	24-35 months	Cross-sectional	Kaplan Meier, TSEIR model	TDaP, MMR	1 month, interval	Household Size
Toikilik, S et al. 2010.[74]	Moderate	Papua New Guinea	Yes	12-23 months	Cross-sectional	N/A	BCG, HepB, Polio, DPT, MCV	variable	Birth Setting, Urban/Rural, Vaccine Hesitancy Questions
Vasudevan, L et al. 2014.[75]	Moderate	Bangladesh	No	11-18 weeks	Cohort	Logistic Regression	Full Vaccination	Fixed time point	ANC, Maternal Education, SES, Child Gender, Mother's Age, Urban/Rural, Size of Infant at Birth, Infant Sickness
Wagner, AL et al. 2014.[76]	Moderate	China	No	8 months - 6 years	Cross-sectional	Wilcoxon/Kruskall-Wallis (unadjusted)	MCV	variable, continuous delay	Child Gender, Resident District, Birth Year
Wagner, AL et al. 2014.[77]	Moderate	China	No	2-7 years	Cohort	Binomial Regression, Cumulative Age-At-Vaccination Curve	Hib, PCV	variable	Child Gender, Urban/Rural, Birth Year
Wagner, AL et al. 2016.[78]	Good	China	No	9 months - 14 years	Cross-sectional	Logistic Regression	MCV, PCV	variable	Child Gender, Child Age, Urban/Rural, Birth Year, Township factors
Wallace, AS et al. 2012.[79]	Good	Philippines	No	5-7 months	Cross-sectional	Chi-Square (unadjusted)	HepB	variable	Child Gender, Introduction of New Vaccine Schedule, Provider Size, Provider Type
Wu, JN et al. 2015.[80]	Good	China	No	1-14 years	Cross-sectional	Logistic Regression	HepB	variable	Ethnicity, Birth Setting, Child Gender, Urban/Rural, HBC Serum Biomarker
Yadav, K et al. 2012.[81]	Moderate	India	No	unclear	Cross-sectional	Chi-Square (unadjusted)	BCG, Polio, DPT, MCV	variable, continuous delay	Hard to reach areas, Lower priority, Family Refusal, Lack of Knowledge
Zaidi, SM et al. 2014.[82]	Moderate	Pakistan	Yes	0-5 years	Cross-sectional	Logistic Regression	BCG, Polio, DPT, MCV	1 month	Ethnicity, ANC, Birth Setting, Maternal Education, SES, Child Gender, Child Age, Urban/Rural, Household Size, Telephone Connection, Male Household Head, Breastfeeding Duration, Maternal Tetanus Vaccine
Zhou, Y et al. 2009.[83]	Moderate	China	No	unclear	Cross-sectional	Logistic Regression	HepB	1 day	Ethnicity, Birth Setting, Maternal Education, Paternal Education, Child Gender, Urban/Rural, Possession of EPI Card, Knowledge of Birth Dose of HepB, Migrant Status
Zivich, PN et al. 2017.[84]	Moderate	Democratic Republic of Congo	No	0-6 months	Cohort	Logistic Regression	BCG, Polio, DPT, PCV	variable, interval	ANC, Maternal Education, SES, Mother's Age, Marital Status, Birth Order, Previous Child Died, Pregnancy Intention, Clinic

Predictors

Most of the included papers (62/67, 93%) adjusted for some variables in multivariate regression in an attempt to identify predictors of untimely vaccination with the other 5 papers only assessed crude measures of vaccine timeliness. In their analysis of vaccine timeliness, a majority of studies considered child’s gender (46/62, 74%) and maternal education (40/62, 65%). Maternal age (26/62, 42%), wealth/socio-economic status (25/62, 40%), urban/rural environment (20/62, 32%), household size (20/62, 32%), and birth setting (19/62, 31%–32%) were also common predictors used in multivariate regression (Figure 4), whereas very few papers examined religion (5/62, 8%) or the introduction of a new vaccine (1/62, 2%) in the context of vaccination timeliness.

Figure 4.

Number of papers reporting different variables used as predictors of vaccine timeliness in multivariable regression (n = 67).

Study design

The studies varied in design, with most using cross-sectional surveys (50/67, 75%) including the Demographic Health Survey or DHS (6/67, 9%), Multiple Indicator Cluster Survey or MICS (1/67, 2%), with the remainder using other types of non-nationally representative cross-sectional surveys (43/67, 64%). A smaller number, 13/67 (19%) of the studies were cohort studies (including data from surveillance systems and immunization information systems), and 3/67 (5%) were randomized controlled trials of which two were cluster-randomized, although none of these used vaccination as either a randomized intervention or an outcome, just as secondary analysis. There was also 1 included case-control study.

Date source(s)

A total of 62 studies (93%) reported the source of information on vaccination dates used to calculate vaccine timeliness within their study. Most studies (49/62, 79%) used data from vaccination cards (or other types of health cards, social welfare cards, or vaccination certificates based upon the country in which the study was conducted); 13% (8/62) used data from immunization information systems, 5% (3/62) used data abstracted from clinic records, and two (3%) based the date of vaccine receipt off the date of study, i.e. when study or data collection occurred on the date of vaccination.

Age

There was a wide diversity of ages in the study population across the 67 papers. In the 57 studies with a clearly stated age range, 35 different age ranges were used. The most common age range were children in the second year of life (12–23 months), with 21% of studies (12/57) including only this age range (see Figure 5 for details). Otherwise, 49% of studies (28/57) included children under 1 year of age, 51% (29/57) included children 1-2 years of age, 53% (30/57) included children 2-3 years of age, 44% (25/57) included children 3-4 years of age, 39% (22/57) included children 4-5 years of age, and 14% (8/57) included children aged 5 and older. There were 10 studies which had an unclearly defined age range.

Figure 5.

Number of papers reporting different age-ranges (n = 67).

Analysis

There were a wide variety of methodological approaches used to assess vaccination timing. Most studies included some sort of significance testing (62/67, 93%). About one-third of studies (22/62, 36%) used logistic regression, and 5/62 (8%) used log-binomial or binomial regression; 6/62 (10%) only used unadjusted chi-square tests (mostly Pearson’s test, although one used survey procedures with a Rao–Scott test). A large portion of studies (48/62, 77%) used some sort of survival analysis, like hazards estimation (16/62, 26%) or Kaplan-Meier analyses (26/62, 42%). One study used Turnbull estimators to account for some participants having information about vaccination dates, but other participants only having a recollection of having received the shot.

Discussion

Measures of age-specific aggregate coverage can obscure or conceal untimely vaccination. The measurement of vaccination timeliness is increasingly perceived as essential to an assessment of immunization system performance and estimates of vaccination coverage, and inclusion of timeliness measures should progressively become the standard in assessment of vaccine coverage in the future. In this systematic review of studies on vaccination timing in LMICs, we found a lack of agreement about what constituted a “timely” vaccination, which was compounded by different vaccination schedules in different countries. Altogether, there was significant disagreement about the definition of timely and delayed vaccinations, with some papers using continuous delay and some categorical timeliness measures. The included papers spanned broad geographic reach but were heavily skewed to the African region and China. The most commonly studied antigens were BCG, measles, and OPV/IPV. Predictors were highly variable, but maternal sociodemographic factors were commonly used. The vast majority of studies included were cross-sectional in study design, most of which used data from vaccination cards, though there was very little consensus among the age range of children to assess for timing. The diversity in operationalization of variables limits comparisons among studies.

The significant variability in defining timing found in this review presents problems for comparing timely vaccinations across countries. Categorizing vaccination timeliness requires a consensus on what constitutes a timely vs. untimely vaccine. Unfortunately, there is often not clear guidance on this. For example, the manufacturer’s instructions for PCV list a four-dose schedule which can start as early as 6 weeks. WHO, however, recommends a three-dose schedule, with several interpretations by region (6, 10, 14 weeks; 2, 4, 6 months; or the first two doses, followed by a booster between 9 and 15 months)[85]. Clearly, any country-specific evaluation of timeliness should follow guidelines from that country’s Ministry of Health, but the definition of timeliness (e.g. 1 week vs 1 month after recommended vaccination date) has different implications when doses within a series are spaced by 4 weeks vs. 2 months.

This variability and the difficulty in comparing findings across studies is compounded by the uneven geographic distribution represented in this review. Almost half of the included papers were focused on the African WHO Region, and another third were from the Western Pacific, specifically China. These papers thus do not paint a representative picture of vaccination timing among LMICs in the European, Eastern Mediterranean, South East Asian, and American Region. In particular, timeliness studies in South and Southeast Asia are needed given the population concentration there and a large burden of vaccine-preventable diseases.

Overall, the most commonly investigated vaccines (measles, BCG, DTP, and polio vaccine) seemed to be relative to international concern about a given disease. For example, measles and polio are both subject to global eradication plans,[86,87] perhaps a reason for their high representation in timeliness studies. Vaccination of measles in a timely manner is particularly important given that the disease is highly contagious and earlier vaccination can reduce the number of susceptible persons in the population,[88] although vaccination at 8–11 months of age is also associated with reduced effectiveness[89]. DTP dose 3 is often used in international assessments of vaccination programs,[90] not only because the vaccine protects against three serious diseases, but also because the vaccine has been widely available for decades, and the third dose represents an immunization system’s ability to revaccinate a child on multiple occasions. A large number of studies investigated BCG, even though the vaccine has seen declining use; more countries have stopped routine BCG vaccination programs since 1981, though most of these are high-income countries in western Europe.[91] Newer vaccines, like rotavirus, PCV, and Hib vaccine, understandably were less likely to be included; these vaccines were added to the WHO EPI in 2013, 2012, and 2006, respectively[2]. However, with the increasing complexity of the EPI schedule and new vaccines being regularly added in different countries, there is also a need for future timeliness studies to address a broader array of antigens, particularly those that are strictly age-limited like Rotavirus and Hib.

Beyond the antigens assessed, there was also considerable variability in the predictors explored as correlates of untimely or delayed vaccination. Gender, education, socio-economic status, urban/rural residence, maternal age, maternal occupation, and birth setting can be situation-specific and highly informative. Interestingly, religion, caste, tribe, and distance to the clinic are potentially important predictors which were very rarely represented in this review but should be considered for inclusion in timing studies depending on the countries involved. Future studies should examine the contribution of vaccine hesitancy to vaccination timeliness and other vaccination metrics.

Vaccination date sources were also surprisingly variable. While most studies used vaccination cards, restricting a study population to those who have vaccination cards can reduce the generalizability of findings as those who possess vaccination cards might not be representative of the general population as a whole. Additionally, vaccination cards can be prone to data recording errors. Immunization information systems (IIS) are preferred in terms of providing an objective and accurate source of vaccination data without depending on retention of a vaccination card, and also allows for following a child longitudinally. Finally, IISs can assist with timeliness determinations while making vaccination data available to the family and clinicians. However, most LMICs do not have functional IISs and large, nationally representative surveys like DHS will continue to be a critical source of information about vaccination coverage and timeliness and researchers using these data sources should include timeliness measures in their analysis, though most included studies did not use nationally representative datasets like DHS.

Finally, the age benchmarks used to include children in timeliness studies were exceedingly variable, which is another, albeit very important, reason why a continuous measure is preferred in addition to improving comparability across studies. Certain historically used age groups (e.g. 23 months) will continue to be important seminal markers of ‘on-time’ vaccination coverage, although the use of continuous measure allows for uninterrupted use of those while also permitting much greater flexibility in looking at vaccination coverage and timeliness over the childhood life span. While analytic approaches also varied, the inclusion of survival analysis techniques in a majority of papers points to the utility, again, of continuous measures of vaccination delay as an interpretable and comparable metric.

Weaknesses in current studies and potential solutions

Any study on vaccination timing will require information about birth and vaccination dates. These are common data fields in electronic registries, such as immunization information systems, and should be included on all vaccination cards or booklets held by families. However, the retention of these cards can be low. In one study from the Southern Nations, Nationalities, and Peoples’ region of Ethiopia, 51% of respondents did not have a vaccination card; however, of those without a vaccination card, 62% previously had a card – but were unable to find it for the interview. Accordingly, only 20% never had a vaccination card given to them.[92] Relying on parental recall of vaccination dates is also likely fraught with problems. A systematic review of the validity of vaccination cards vs parental recall found that coverage (not including any timing or date information) from recall was an underestimate of 7 percentage points vs medical records, and recall had a median sensitivity of 90% and specificity of 50%.[93] Therefore, in some situations, analyses of immunization timing may be biased towards individuals who are more likely to retain a vaccination card. For example, according to the 2011 Ethiopia Demographic and Health Survey (DHS), 57.0% of children with a vaccination card were fully vaccinated, compared to 11.2% who did not have a card.[94]

Future studies on vaccination timing in LMICs will likely continue to rely heavily on recall of vaccines administered which means that some participants will have vaccination date information, and others will not. The use of statistics that take into account both left and right censoring – such as nonparametric Turnbull estimators or parametric-accelerated failure time models – would permit greater use of vaccination data from those both with and without vaccination cards. Turnbull estimation has been used in recent papers from India.[68,95] The Turnbull estimator allows for both right censoring (e.g. the child is not vaccinated at the time of the survey) and left censoring (the child has been vaccinated, but with an unknown date that is at the very latest, the day of the interview). These statistics can be brought up in SAS using the lifereg procedure (SAS Institute, Inc., Cary, NC, USA) or in R using the survival and SurvRegCensCov packages (R Foundation for Statistical Computing, Vienna, Austria). These procedures can be thought of as an extension of typical methods of survival analysis (e.g. Cox regression), which only consider right censoring.

We also recommend that future studies employ precise methods to estimate months. Months can vary between 28 and 31 days, according to the Western calendar, and some countries use different systems of measurement – for example, the Ethiopian calendar consists of 30-day months, in addition to an intercalated month of 5 or 6 days. As a shortcut, many studies employ 30 days as an average. In one study, where “month” and “year” of vaccination, but not “day,” was collected, vaccinations were considered timely if it was given in the month it was due based on the child’s birth date.[24] However, this would underestimate timely vaccinations (relative to a 1-month benchmark). A better method is to identify the actual date that a vaccine administration would be untimely. For instance, a child born on February 14 2007, who should receive a certain vaccine within one month for it to be timely would need to be vaccinated by March 14. These dates can be identified through certain coding steps (for instance, using an intnx statement in a data step in SAS).

Vaccination timeliness can be considered as both continuous (e.g. mean number of days delayed, or average age at vaccination) and categorical measures (e.g. % with a timely vaccination). Due to differences between countries in vaccination schedules and a lack of concordance in the literature on what constitutes a “timely” dose, a continuous measure may be more appropriate. Cumulative incidence curves (or reverse Kaplan-Meier plots) can graphically depict this information in an easy-to-understand manner. If a categorization is desired, multiple groupings (e.g. both timeliness based on 1 week and timeliness based on 1 month) is preferable, in the absence of any country-specific guidance on what constitutes a timely dose.

Finally, certain statistical basics should always be followed. Studies with clustering of the sample should specify survey procedures in the regression analysis (for instance, using Taylor series estimation of standard errors). In cross-sectional or cohort studies, multivariable regression with an outcome of the “prevalence” or “likelihood” of a timely vaccine can utilize a binomial or log-binomial regression, which would output a “risk ratio” or “prevalence ratio” which is more easily understood than an odds ratio.[96]

Strengths and limitations

This study has several limitations. We limited ourselves to studies published in PubMed, EBSCOhost, and Embase (in addition to studies found through an abstract search), however, manuscripts on timeliness could have also been submitted to regional journals not indexed in international databases. We also limited our search to LMICs, as these are countries where timely administration of vaccines can be an especially important mechanism to reduce morbidity from infectious diseases, and it was beyond the purview of this review to examine differences in timeliness between high-income countries and LMICs.

Conclusions

Ensuring children have access to timely vaccinations is a critically important and cost-effective way to decrease the incidence of disease and reduce morbidity within a population. In this systematic review of 67 papers from LMICs, analysis of vaccination timing has become more common over time, and papers represented all 6 WHO regions. However, there is a lack of concordance in the literature on what constitutes a “timely” vaccination – with studies variously using benchmarks of 1 day, 1 week, 2 weeks, 4 weeks, 1 month, etc. As such, providing information on the continuous spread of date of vaccination administration (e.g. graphing the cumulative incidence of vaccination by increasing age), or characterizing median delay in vaccine receipt from the suggested schedule might provide more valuable information than a categorization (e.g. stating the proportion vaccinated by a certain benchmark), although both can be presented within one study.