Estimation of nationwide vaccination coverage and comparison of interview and telephone survey methodology for estimating vaccination status.

This study compared interview and telephone surveys to select the better method for regularly estimating nationwide vaccination coverage rates in Korea. Interview surveys using multi-stage cluster sampling and telephone surveys using stratified random sampling were conducted. Nationwide coverage rates were estimated in subjects with vaccination cards in the interview survey. The interview survey relative to the telephone survey showed a higher response rate, lower missing rate, higher validity and a less difference in vaccination coverage rates between card owners and non-owners. Primary vaccination coverage rate was greater than 90% except for the fourth dose of DTaP (diphtheria/tetanus/pertussis), the third dose of polio, and the third dose of Japanese B encephalitis (JBE). The DTaP4: Polio3: MMR1 fully vaccination rate was 62.0% and BCG1:HepB3:DTaP4:Polio3:MMR1 was 59.5%. For age-appropriate vaccination, the coverage rate was 50%-80%. We concluded that the interview survey was better than the telephone survey. These results can be applied to countries with incomplete registry and decreasing rates of landline telephone coverage due to increased cell phone usage and countries. Among mandatory vaccines, efforts to increase vaccination rate for the fourth dose of DTaP, the third dose of polio, JBE and regular vaccinations at recommended periods should be conducted in Korea.

INTRODUCTION

Vaccinations are one of the most cost effective means to reduce morbidity and mortality associated with infectious diseases. The introduction of the vaccine has led to nearly a 90-100 percent reduction in target disease morbidity and mortality (1, 2). In Korea, bacille Calmette-Guérin vaccine (BCG), hepatitis B, DTaP (diphtheria/tetanus/pertussis), MMR (measles/mumps/rubella), polio, Japanese B encephalitis (JBE), and varicella are recommended for children by the National Immunization Program (NIP).

The vaccination coverage rate can be estimated by population-based surveys or administrative systems such as the nationwide registry for vaccination (3). Although vaccination coverage rate is a prior method to evaluate childhood immunization programs, Korea does not regularly conduct nationwide vaccination coverage surveys and the 'Immunization Registry' system needs improvement due to insufficient completeness.

Population surveys include mail, telephone, face to face interview or web-based method, and different survey modes may yield substantially different results (4, 5). Therefore, selecting appropriate methods tailored for different countries is important. Therefore, in this study, we conducted two nationwide surveys for vaccination coverage and compared the two survey methods, that were interview and telephone, in order to select the better method for estimating the nationwide vaccination coverage rate of children aged 0-6 yr in Korea. In addition, we estimated the nationwide vaccination coverage rate using the interview survey that we determined as the better method.

MATERIALS AND METHODS

Sample size calculation

The sample size required was determined using 65% coverage obtained in 1989 (6), 3% precision, type 1 error of 5% and the equation N = Za/22 p (1-p)/d2 (7, 8) where Za/2 was type 1 error (1.96), P was estimated vaccination coverage rate (0.65) and D was precision (0.03). Thus, the calculated number of children was 971, and the number of subjects was allocated based on the sample size 1,000.

Study design and survey method

A cross sectional survey was conducted in Korea between October 28th and December 7th, 2008. Children aged 0 to 6 yr who resided in Korea was the target population, and mandatory vaccines recommended by Korea NIP such as BCG, hepatitis B, DTaP, MMR, polio, JBE and varicella were the target vaccines.

To select the better survey method, two surveys were conducted. For the interview survey, we used multi-stage cluster sampling for the target population. First, we stratified the regions by Korean provinces/metropolitan cities. In Seoul, the target population was big, so we divided Seoul into four regions and 19 regional strata were made. The metropolitan cities were stratified by Gu and the provinces were stratified by Si and Gun. We designated the number of cluster in each Si, Gun and Gu proportionally 0-36 months and 37-72 months age groups. The final selected clusters in each stratum were Dong for Si and Gu area, and Eup, Myun for Gun area.

We randomly selected 103 clusters stratified over 19 Korean provinces/metropolitan cities proportionally the number of target population in the strata. Second, ten children were selected per cluster. We randomly chose one household and identified the first household with a child between the ages of 0 and 6. At the first home, we asked where the next home located more than five houses away from the first home with a child of the target age and visited the house for interview.

For the telephone survey, we used stratified random sampling. First, we stratified over 19 regional strata as interview survey and targeted children in the following age groups: 0-36 months; 37-72 months. Then, we selected telephone numbers proportionately to the number of children using the Computer Assisted Telephone Interview (CATI). The sampling frame was the directory list of Korea Telecom's landline telephone numbers. Considering non-response, 60 times of the number of allocated telephone numbers were selected. Five calls were conducted for ring-no-answer numbers and 10 calls for residential numbers then moved to next number. After acquiring consent, the interviewers asked whether children aged 6 yr old or less and terminated the telephone survey if the numbers did not have the children. Before conducting survey, educations for interviewers were conducted twice. In the education, the role-play as interviewers and interviewees were done.

If houses with two or more eligible children were present, the child whose month and day of birth was closest to the study date was selected for the interview and telephone survey. A total of 1,026 children between the ages of 0 and 6 were selected for face to face interview and 1,051 were surveyed by telephone. This covered about 0.03% of the target population aged 0-6 yr. In the interview survey, 95.5% of the children owned vaccination card and 51.0% in telephone survey. The general characteristics of the study population are summarized in Table 1.

Table 1

General characteristics of study subjects who responded

We collected information from vaccination cards or memory of the child's caregiver. Vaccination dates were collected from only vaccination cards. Additional questions were added in the interview questionnaire including the name of the telecommunication company and whether the telephone number was listed in the directory to estimate the degree of non-coverage of the telephone survey.

Comparison between interview and telephone survey

We determined the more ideal survey method for estimating vaccination coverage rate of the two survey methods (interview and telephone) based on the following five items: response rate; missing rate according to vaccine doses; validity with the individual vaccination history from the registry data; differences of vaccination coverage rate estimated in both card owners and non-owners; coverage proportion of the final target population that can be estimated by the interview or telephone surveys relative to the theoretical target population.

The response and missing rate of the two methods were compared using the survey results. To assess validity of the vaccination history, we selected children who were vaccinated at public health centers in the total study population and matched them to the immunization registry data according to telephone number, place of residence and birth, date of birth, and twin status. Korea's immunization registry data is valid since information was collected based on children's medical charts in the health centers and entered in the web database by trained nurses. We compared information of shots in our survey results with the immunization registry data. To assess the differences in vaccination coverage rate in card owners and non-owners, we calculated vaccination coverage rate in both groups and 95% confidence intervals (CIs). The target population in the telephone survey may be restricted due to landline telephone retention proportion in the total population and the share proportion of the telecommunication companies, while the interview survey can be accessed without the limitations of the telephone survey. The coverage proportions of the final target population in the telephone survey relative to the theoretical target population were estimated by additional questions during face to face interviews. Question included whether participants had landline telephones, name of telecommunication companies if they had one or more landline telephones, and whether telephone numbers were listed.

Definition and statistical analysis

Based on vaccination card records for the more ideal survey method, vaccination coverage rates were calculated in children who were one month older than the recommended age for each vaccine. The full vaccination rate was defined in two ways. 4:3:1 full vaccination was defined as children who completed the fourth dose of DTaP, the third dose of polio and the first dose of MMR, and 4:3:1:3:1 full vaccination was defined as children who completed the first dose of BCG, the third dose of hepatitis B, the fourth dose of DTaP, the third dose of polio, and the first dose of MMR. Children who were more than 19 months old and had a vaccination card were included in the calculation.

Age-appropriate immunization rates were calculated in those who received a vaccination and owned a vaccination card with the recorded date of vaccination. We subtracted the birth date from the vaccination date and regard those whose results were included within the recommended period as age-appropriate vaccinated.

Weighted proportions and 95% CIs according to the inverse of sampling and response rate were applied to estimate vaccination coverage rate and fully vaccination rate. Crude proportions and 95% CIs were calculated for age-appropriate vaccination rate. Crude vaccination coverage rates were calculated and we estimated the weighted rates according to response rate and sampling fraction of the population as the other methods did (9, 10). SAS version 9.1 was used for statistical analysis.

Ethics statement

The study protocol was approved by the institutional review boards of Seoul National University Hospital (Approval number: C-0806-058-247). We received informed consents from all the participants in the interview survey but for telephone survey, we received an exemption from written consent form because direct contact was difficult.

RESULTS

Comparison of the result between interview and telephone survey

Table 2 compares the results of the interview and telephone surveys. The response rate in the interview survey was 33.6%, while that in the telephone survey was 1.5%. Among non-respondents in the telephone survey, 54.3% was not-answering numbers at the time of survey, 7.8% refused before confirmation of target aged children, 31.2% were households without age-eligible children, and 5.2% refused although eligible children resided. In the face to face interview surveys, 22.7% refused before asking whether target aged children resided or not, 12.0% were households without age-eligible children, and 31.7% refused although eligible children resided.

Table 2

Comparison of interview and telephone survey methods for vaccination coverage in Korea

*see Appendix 1; †According to additional questions in the interview survey, 77% of participants had a landline telephone and of them, 77% used Korea Telecom. Among them, 20% had listed telephone numbers. The coverage proportions was estimated by multiply them. JBE, Japanese B encephalitis.

The missing rate was higher in the telephone survey. The missing rate of the interview survey was less than 1% in vaccination card owners (median missing rate 0%, range 0%-0.8%) and less than 3% in non-vaccination card owners (median missing rate 0.3%, range 0%-3.0%). The median missing rate of the telephone survey was 0.6% (range 0%-2%) in vaccination card owners and 19.2% (range 1.2%-38.7%) in non-vaccination card owners (Table 2, Appendix 1).

To compare validity with the immunization registry data, 29 children (273 shots) in the face to face interview surveys and 125 children (1,329 shots) in the telephone survey were matched. All the shots were concordant in the interview survey, while the fourth dose of polio showed low agreement (3/8 = 37.5%) in the telephone survey although other results were concordant completely.

When comparing the coverage rate among vaccination card owners and non-owners, the coverage rates in non-owners were consistently lower. However, the differences were not significant except for the third dose of JBE in the face to face interview survey (Fig. 1), while the difference for the majority of most vaccination coverages were statistically significant in the telephone survey except for the first dose of hepatitis B and the first dose of polio (Fig. 2). The differences were two to three times greater in the telephone survey that is partially caused by the high missing rate among non-card owners in the telephone survey.

Fig. 1

Comparison of the vaccination coverage rate between vaccination card owners and non-owners in the interview survey. BCG, Bacille Calmette-Guérin vaccine; HepB, Hepatitis B vaccine; DTaP, Diphtheria-Tetanus-Pertussis vaccine; MMR, Measles-Mumps-Rubella vaccine; Var, Varicella vaccine; JBE, Japanese B encephalitis vaccine.

Fig. 2

Comparison of the vaccination coverage rate between vaccination card owners and non-owners in the telephone survey. BCG, Bacille Calmette-Guérin vaccine; HepB, Hepatitis B vaccine; DTaP, Diphtheria-Tetanus-Pertussis vaccine; MMR, Measles-Mumps-Rubella vaccine; Var, Varicella vaccine; JBE, Japanese B encephalitis vaccine.

According to additional questions in the interview survey, 77% of participants had a landline telephone and of them, 77% used Korea Telecom. Among them, 20% had listed telephone numbers. Therefore, we estimated the coverage proportions of the final target population in the telephone survey relative to the theoretical target population to only 12% (0.77 × 0.77 × 0.2 = 0.12), while the coverage proportion of the final target population in the interview survey was 100% of the theoretical target population because of fewer limitations. Because of the above-mentioned five items, we determined that face to face interview survey was a more appropriate methodology for estimating vaccination coverage rate.

Coverage rate of mandatory vaccines

The coverage rate of each vaccine per dose is presented in Table 3. Vaccination coverages for BCG, hepatitis B, first to third dose of DTaP, first and second dose of polio, MMR, and varicella were above 90% and decreased as the number of doses increased, and the coverage rate of the fourth dose of DTaP showed 65.7% and the third dose of polio was 89.2%. The coverage of JBE was low with 80.9% for the first/second dose and 53.0% for the third dose.

Table 3

Vaccination coverage in children aged 0-6 yr and complete vaccination coverage rate for those with a vaccination card in the interview survey

Age-appropriate vaccination was calculated by dividing the number of children getting vaccinated during recommended period (A) by the number of study population according to kind of vaccine (B). *With children who received a vaccination and owned a vaccination card with the recorded date of vaccination. BCG, Bacille Calmette-Guérin vaccine; HepB, Hepatitis B vaccine; DTaP, Diphtheria-Tetanus-Pertussis vaccine; MMR, Measles-Mumps-Rubella vaccine; Var, Varicella vaccine; JBE, Japanese B encephalitis vaccine.

Among children aged 19-72 months with a vaccination card, 62.3% were fully vaccinated [DTaP4:Polio3:MMR1] and 60.2% was fully vaccinated [BCG1: HBV3:DTaP4:Polio3:MMR1]. Children aged 36-72 months showed a higher rate of complete vaccination and a significant [DTaP4:Polio3:MMR1] full vaccination rate only (Table 3).

The age-appropriate immunization rates were presented in Table 3. Age-appropriate vaccination rates were between 50%-80% except the first dose of hepatitis B that was greater than 90%. Age-appropriate immunization rate of hepatitis B decreased significantly as the number of shots increased. In the case of DTaP and polio, age-appropriate immunization rate was lowest for the first dose and highest for the second dose and decreased again as the number of shots increased.

DISCUSSION

We conducted two nationwide surveys for vaccination coverage to select a better method. The first was face to face interview survey using multi-stage cluster sampling method, while the second was telephone survey with stratified random sampling based on the directory listing of Korea Telecom's landline telephone numbers. When calculating coverage rate, we included subjects with vaccination cards to confirm the validity.

When comparing interview and telephone survey methods, the telephone survey methodology falls short in five aspects that are as follows: higher non-response rate; higher missing rate; lower validity of the vaccination information relative to the registry data; large differences in vaccination coverage rate between card owners and non-owners lower coverage proportion of the final target population.

The sampling frame for our study included only households with listed phone numbers whose telephone service provider was Korea Telecom. Data from the Statistics Korea showed that households without telephones were approximately 8.4% and Korea Telecom was the telephone provider for nearly 90% of households with landline telephones. Also, people in their twenties and thirties are less likely to have landline telephone and the proportion is rapidly increasing (11). In the survey, the coverage proportion of the target population in the telephone survey was estimated to cover 12% of the theoretical target population. Random digit dialing (RDD) that uses a randomly generated telephone number may be an alternative to permit sampling of all persons with listed or unlisted telephone numbers. However, this can be time consuming because of time spent calling non-working numbers. The telephone survey does not include cell phone numbers because the sampling frames of cell phones and landlines overlap, cell phone numbers do not provide information on residential area, and the financial burden is great (12). In the United States, to compensate for non-coverage of households without landline telephone numbers, questions about telephone interruption were included in the survey and vaccination coverage of households experiencing interruption were used in estimating coverage of households without landline telephone (13-15). In order to do this, data from immunization survey, the National Health Interview Survey, the National Center for Health Statistics natality data were used (13-15). However, in the countries with the limited data from other sources and high rate of not having a landline telephone, inclusion of cell phone numbers is important.

In interview survey, to reduce bias caused by non-response, 10% of the number of samples could be oversampled (16) or advance letter could be sent to houses (13). If the lists of all households could be obtained, systematic or random sampling could be applied (17). However, it was difficult to know all the houses with eligible aged children, these methods appears to be hard to apply to survey for vaccination coverage among children.

The response rate of the telephone survey for the vaccination coverage rate was lower than previous telephone surveys in Korea targeting adults (6.9%-8.3%) (18, 19) or people aged 14 yr or more (33.6%) (20). Because we needed to identify target households before receiving consent, time was spent for identifying the eligible population and the final response rate in the telephone survey was about 1.5%. When compared two methods, the denominator of the interview survey was the number of contacting households but that of the telephone was the number of making phone calls. Among the total number of the calls, more than 50% were not-answering numbers including invalid and fax numbers and it caused wider gap between two methods.

Not only the high rate of unanswered phone calls but also identifying eligible households required increased cost and time and decreased efficiency. In general, telephone survey has been thought that it has advantages in saving cost and time for conducting survey and convenience in supervising but has disadvantages in lower response rate and incomprehensiveness of target population compared to face to face interview survey (12, 21, 22). In the aspect of cost, interview survey could save time and costs by asking first households about other eligible house but because it took time to identify houses with target aged children, telephone survey seemed to take longer time and similar cost compared to interview survey.

Compared to the immunization registry record, information from the vaccination card from the interview survey showed 100% agreement but there was discordance between data from vaccination cards from telephone survey and immunization registry records in the telephone survey. This may be because in the telephone survey, respondents referred to the vaccination cards that could lead to inaccurate information, while trained interviewers confirmed information from the vaccination card with the interview survey. In previous studies, the quality of information about visible factors, interview survey was more accurate (12, 22). Also, missing rate was higher in the telephone survey, especially those without vaccination cards. In addition, missing rate for the telephone survey among children with vaccination cards was similar that for the interview survey among children without card and missing rate of the interview survey among children with vaccination cards was less than 1%. Data gathered by interview survey is more reliable and valid than the telephone survey for estimating vaccination coverage survey. Because of the above-mentioned five items, we determined that face to face interview survey was a more appropriate methodology for estimating vaccination coverage rate.

Proportion of those who owned vaccination card showed big differences between interview and telephone survey. The proportion in telephone survey was 51% and it was lower than the survey conducted in 1989 (Table 4). Considering increasing vaccination card owing rate, proportion in telephone survey might not be representative. In interview survey, proportion of owning vaccination card was 95.5%. It was similar results from one study conducted in Korea (23). As a result, it was suggested that participants of interview survey reflected total population better.

Table 4

Comparison of the study methods and result to that of previous study

*Number of the participants was not presented; †Percent of the vaccination card owner was not presented; ‡Confidence interval could not be calculate because the number of subjects was not presented; §For comparison with 1989 study, subjects were restricted according to the standards applied in the 1989 study (6-29 months old) and vaccination card owning was not considered; ∥For comparison with 1994 study, subjects were restricted according to the standards applied in the 1994 study (0-54 months old) and vaccination card owning was not considered; ¶For comparison with 2000 study, subjects were restricted according to the standards applied in the 2000 study (0-29 months old) and vaccination card owning was not considered. BCG, Bacille Calmette-Guérin vaccine; HepB, Hepatitis B vaccine; DTaP, Diphtheria-Tetanus-Pertussis vaccine; MMR, Measles-Mumps-Rubella vaccine; Var, Varicella vaccine. JBE, Japanese B encephalitis vaccine.

Our result from the interview survey showed that coverage rates of most of the mandatory vaccines to be completed before 18 months of age were over 90% in Korea. However, as the number of shots increased, coverage rate decreased and the fourth dose of DTaP showed less than 70% coverage and the third dose of polio was 89.2%. The full vaccination rate was around 60%, and the age-appropriate immunization rate was between 50%-80%. As the number of shots increased, age-appropriate immunization rate tended to decrease. Compared with previous nationwide vaccination studies conducted in 1989 (6), 1994 (24), and 2000 (25), we calculated coverage rate with those aged same standard to each study irrespective of vaccination card (Table 4) and concluded overall vaccination coverage rate was increased.

This study has several limitations. The sampling methods of interview survey and telephone survey was not same because we could get the list of telephone numbers using Korea Telecom's landline telephone and opening to the public but could not get all of the household list in the strata (19 provinces/metropolitan cities). Due to this difference, we applied cluster based sampling in interview survey and random sampling in telephone survey. It could cause different kind of selection bias in interview and telephone survey and less adequate comparison between two methods.

Also when validity of interview and telephone survey was compared, because only 29 and 125 children (273 and 1,329 shots) in interview survey and telephone survey respectively were matched, the validity that we confirmed might be caused by chance. All the shots were concordant in the interview survey, while the fourth dose of polio showed low agreement (3/8 = 37.5%) in the telephone survey although other results were concordant completely.

Coverage of target population was calculated with the information of participants of the interview survey. It may cause selection bias because we could not consider the characteristics of non-participants. As a result, true coverage rate might be different according to the information of non-participants. Compared to the data from the Statistics Korea, the proportion of landline telephone owner and the people using Korea Telecom was lower in this survey but 90% of the respondents were less than 40 yr old, telephone possession rate could be lower than total population.

In the process of sampling in the interview survey, we sampled the first child randomly and then convenience sampling method was applied. Though random sampling might be considered ideal, we applied the similar selection method with '30 by 7 sampling method' developed by the WHO to obtain the efficiency. Our starting point was selected near the village office and selected the next child residing at least five houses from the previous household to prevent "pocketing" (26).

Because the target age was wide (0-6 yr old) and only those with vaccination card were included, the number of the children for each vaccine was different and precision of coverage rate for each vaccine varied. We estimated coverage rate with only those having vaccination cards to get valid result, and it might overestimate coverage rate because they were more aware of immunization of the children (11, 27-29). In our result, coverage rate was also higher in those with vaccination card but not significant in interview survey. Also, when we calculated the rate of owing vaccination card according to age, as the age increased, the rate tended to be lower. Considering older age of the child was associated with suboptimal compliance to vaccination (29) and older children were less included for vaccination coverage calculation in this study it might overestimate coverage rate, too.

In spite of these limitations, this study compared two kinds of methodology that are interview and telephone. This study suggests that the interview survey is a better methodology for vaccination coverage survey. It could also be suggested that in countries with incomplete registry, a high cell phone coverage rate and decreasing landline telephone coverage rate as well as countries which used the vaccination card as a tool keeping track of and lists all immunization shots children have received in the past, interview survey is the better methodology especially for vaccination coverage.

Among mandatory vaccines, efforts to increase the coverage rate of the fourth dose of DTaP and the third dose of polio, JBE recommended after 1 yr old and vaccination at the recommended time should be implemented in Korea.