| Literature DB >> 27349841 |
Felicity T Cutts1, Pierre Claquin2, M Carolina Danovaro-Holliday3, Dale A Rhoda4.
Abstract
Vaccination coverage is a widely used indicator of programme performance, measured by registries, routine administrative reports or household surveys. Because the population denominator and the reported number of vaccinations used in administrative estimates are often inaccurate, survey data are often considered to be more reliable. Many countries obtain survey data on vaccination coverage every 3-5years from large-scale multi-purpose survey programs. Additional surveys may be needed to evaluate coverage in Supplemental Immunization Activities such as measles or polio campaigns, or after major changes have occurred in the vaccination programme or its context. When a coverage survey is undertaken, rigorous statistical principles and field protocols should be followed to avoid selection bias and information bias. This requires substantial time, expertise and resources hence the role of vaccination coverage surveys in programme monitoring needs to be carefully defined. At times, programmatic monitoring may be more appropriate and provides data to guide program improvement. Practical field methods such as health facility-based assessments can evaluate multiple aspects of service provision, costs, coverage (among clinic attendees) and data quality. Similarly, purposeful sampling or censuses of specific populations can help local health workers evaluate their own performance and understand community attitudes, without trying to claim that the results are representative of the entire population. Administrative reports enable programme managers to do real-time monitoring, investigate potential problems and take timely remedial action, thus improvement of administrative estimates is of high priority. Most importantly, investment in collecting data needs to be complemented by investment in acting on results to improve performance.Entities:
Keywords: Coverage; Health facility surveys; Program monitoring; Surveys; Vaccination
Mesh:
Year: 2016 PMID: 27349841 PMCID: PMC4967442 DOI: 10.1016/j.vaccine.2016.06.053
Source DB: PubMed Journal: Vaccine ISSN: 0264-410X Impact factor: 3.641
Advantages and disadvantages of methods to measure vaccination coverage.
| Method | Advantages | Disadvantages |
|---|---|---|
| Register-based (electronic) | Can give complete and accurate real-time data on cumulative vaccination status of individual persons and populations | Need good computer access |
| Routine reports of vaccinations delivered | Can be simple in conception | Population denominators often inaccurate, especially at local levels |
| Community-based surveys | If well-conducted, evaluate coverage in routine services and/or in SIAs | Accessibility to populations to survey depends on geographic, climatic and security issues, and high-risk subgroups (e.g., migrants, street children) often missed, compromising representativeness of survey results |
Adapted from Table 70-3 in [10].
Illustrative timeframe for a probability sample household coverage survey.
| Stage | Activity | Dates |
|---|---|---|
| Planning and survey preparation | Form steering committee and technical subcommittees, identify implementing agency, agree on methods to recruit field coordinators, supervisors and interviewers, agree on use or not of digital technology for data collection, identify technical assistance if required, set up liaison with statistics or census office, order and obtain supplies & identify transport | Months 1–4 (may take longer if Request for Proposals issued for selection of implementing agency, if complex survey design with multiple indicators, depending on ethics committee procedures and timetable, and depending on time needed to obtain and release funding) |
| Survey design and modification to fit resource availability | ||
| Obtain funding for the survey | ||
| Obtain ethical review as required | ||
| Sample selection (including obtaining enumeration area-EA maps) | ||
| Visit health facilities in the areas selected for the survey to explain survey and obtain co-operation | ||
| Questionnaire design, pretest and translation | ||
| Preparation of digital entry procedures, if used | ||
| Pretest household sampling procedures | ||
| Preparation of manuals/standard operating procedures | ||
| Preparation of training site(s) and materials | ||
| Preparation of database | ||
| Training | Train field workers and supervisors: household listing, collection of GIS coordinates, conducting interviews, getting data from health facilities, checking completed questionnaires, digital data entry where relevant, ensuring SOPs are followed | Month 5 (longer for large surveys; allow 2 weeks for every 30 field staff being recruited) |
| Train data entry staff if paper forms are used | ||
| Data collection | Mapping and household listing | Months 6 (if small survey), or 6–8 (for survey with multiple strata) |
| Data management and analysis | Data double-entry and editing (if paper forms used) | Months 6–7 (small survey) or 6–9 (large survey) |
| Final data checking and cleaning | ||
| Data analysis, produce tables and graphs | ||
| Report generation and dissemination | Preparation/review of preliminary report | Months 10–12 (may be sooner if small and focused survey is done) |
| Prepare final report, with summary of key findings | ||
| National feedback and develop action plan | ||
| Prepare reports/fact sheets for health workers | ||
| Workshops with health workers at sub-national levels | ||
Sample size and number of households that must be visited in a post-SIA survey with and without inclusion of routine immunization (RI) coverage assessment.
| N (Completed Interviews) | HH (Households to Visit) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Expected Coverage | Post-SIA Alone | RI DTPCV3 12-23m | Post-SIA Alone | RI DTPCV3 12-23m | |||||
| SIA | RI | ±3% | ±3% | ±5% | ±10% | ±3% | ±3% | ±5% | ±10% |
| 95% | 90% | 708 | 1554 | 645 | 210 | 1416 | 7770 | 3225 | 1050 |
| 95% | 85% | 708 | 1989 | 795 | 240 | 1416 | 9945 | 3975 | 1200 |
| 95% | 75% | 708 | 2676 | 1020 | 285 | 1416 | 13,380 | 5100 | 1425 |
| 90% | 60% | 1036 | 3291 | 1203 | 309 | 2072 | 16,455 | 6015 | 1545 |
SIA = Supplemental Immunization Activity; RI = Routine Immunization; DTPCV3 = 3 Doses of Diphtheria Tetanus Pertussis Containing Vaccine.
Notes: This table assumes a post-SIA design effect of 2.0 and an RI design effect of 3.0, as intracluster correlation is expected to be higher for the latter. It also assumes that eligible respondents for post-SIA survey are 9m-15y of age and that it will be necessary to visit two households, on average, to find each eligible and cooperative respondent. Eligible RI respondents are 12–23 m of age and it will be necessary to visit 5 households, on average, to find each eligible and cooperative respondent.
Choice of methods according to the purpose of monitoring.
| Monitoring/evaluation purpose | Cluster survey with probability sampling and excellent quality control | Health facility-based surveys (note: results refer only to persons using the facilities sampled) | Purposeful sampling or censuses of selected populations | ||
|---|---|---|---|---|---|
| National | Every province | Selected provinces/districts | |||
| To measure coverage in an SIA at national level | Yes | Yes | No | No | No |
| To evaluate SIA coverage at national | No | Yes | No | No | No |
| To measure national routine vaccination coverage | Yes | Yes | No | No | No |
| To measure routine coverage at sub-national levels | No | Yes, though not feasible in all districts | Yes, for selected areas only | No | Yes, in the areas included, with no sampling error |
| To classify coverage at sub-national levels | No | Yes | Yes, for selected areas only | Yes, among children who’ve received DTPCV1 | Not relevant |
| To determine if an intervention has successfully increased coverage | Possibly, if sample size adequate (based on baseline and follow-up sample precision) | Yes, if sample size adequate to detect the difference. Especially useful if some areas had the intervention while others did not, and if baseline and post-intervention are available for areas with and without the intervention | Can assess if an intervention has increased coverage of subsequent vaccines among children who have received DTPCV1 | Theoretically possible, but probability samples of larger areas usually preferred | |
| To assess dropout, timeliness, missed opportunities | Can measure dropout and missed opportunities but does not assess their health-system causes | Can assess both prevalence and causes | Can assess prevalence in the areas canvassed | ||
| To identify reasons for lack of receipt of DTPCV1 | Yes if suitable questions added to the survey, ideally informed by prior qualitative research (and need appropriate sample size calculations for hypothesis testing), but does not directly evaluate health system barriers | Can assess health system barriers to attendance (e.g. vaccine stock-outs, health worker absence) | Yes, but only in the areas canvassed | ||