Immunization milestones: a more comprehensive picture of age-appropriate vaccination.

A challenge facing immunization registries is developing measures of childhood immunization coverage that contain more information for setting policy than present vaccine series up-to-date (UTD) rates. This study combined milestone analysis with provider encounter data to determine when children either do not receive indicated immunizations during medical encounters or fail to visit providers. Milestone analysis measures immunization status at key times between birth and age 2, when recommended immunizations first become late. The immunization status of a large population of children in the Oregon ALERT immunization registry and in the Oregon Health Plan was tracked across milestone ages. Findings indicate that the majority of children went back and forth with regard to having complete age-appropriate immunizations over time. We also found that immunization UTD rates when used alone are biased towards relating non-UTD status to a lack of visits to providers, instead of to provider visits on which recommended immunizations are not given.

1. Introduction

A challenge to increasing early childhood immunizations on a state or local level is the limited ability of standard summary measures of up-to-date (UTD) rates to identify barriers to improvement. Typically, early childhood UTD rates are based on the number of doses of recommended vaccines, by individual antigen or in total, that a cohort of children receive by either a fixed age or a fixed date of assessment [1]. For the National Immunization Survey (NIS), this represents the proportion of 19- to 35-month-old children having all recommended doses for up to 7 vaccine types. Similarly, health plans utilize a Healthcare Effectiveness Data and Information Set (HEDIS) immunization measure, which counts the number of doses received by 24 months of age [2, 3]. However, a low UTD rate among a population, derived using these measures, does not aid in determining why the rate is low [4]. The growing complexity of the early childhood immunization schedule, with up to 19 vaccine doses recommended across at least 6 visits by age 2, means that there are many points of time and many reasons by which children can fall behind on immunizations [5]. While immunization summary UTD rates for 2-year-olds can identify general problems, there is also a need for more detailed assessment tools to describe local immunization coverage and more specific vaccine usage [6].

One alternative method of evaluation is to consider age-appropriate vaccinations. A substantial body of prior work exists comparing summary UTD measures against more specific assessments either for complete antigen series or combinations when actually due without considering late catch-up [7-9], or in comparison of individual antigens and shots to when they are expected or late [10]. To the extent that early childhood immunizations are often used as a proxy for the quality of early childhood routine care, the timeliness of immunizations is a relevant measure—delayed or lagging immunizations may reflect other issues with early childhood care.

One perspective on age-appropriate immunizations is to track children's progress through immunization milestones between birth and age 2 [11]. Immunization milestones are the ages at which recommended immunizations first become late according to the schedule developed by the Advisory Committee on Immunization Practices (ACIP). These milestones occur at 3, 5, 7, 16, 19, and 24 months of age. When immunizations are tracked using this approach, children's progression through milestones unfolds as a story of falling behind and catching up with recommended doses. This is a beneficial method as it facilitates identifying provider failure to give all or some of the immunizations that are due at healthcare encounters (missed opportunities) or parental failure to bring children to providers for vaccination-eligible encounters (missed visits). The prevalence of missed opportunities and missed visits at each milestone age can guide immunization interventions. However, basing milestone analysis exclusively on immunization record data may misclassify missed opportunities as missed visits, and shift the apparent burden of children who are not appropriately immunized for their age from providers to parents. This can occur because healthcare encounters during which no vaccinations are administered will not be captured into immunization record datasets.

This study provides an example of using a milestone approach to assess a specific population and their progression through early childhood immunizations, where both payor-administrative and state-level immunization information system (IIS) data are available. Combining children's healthcare encounter information from payer records with their immunization records gives a more accurate assessment at each milestone age of the effect of missed opportunities and missed visits on age-appropriate immunizations and overall immunization rates.

2. Methods

2.1. Data Sources

The study population consisted of a birth cohort of Oregon children enrolled in the Oregon Health Plan (OHP) and whose immunization records were in the Oregon ALERT Immunization Information System (ALERT IIS). ALERT IIS immunization records were merged with provider encounter records from the OHP for this population. The ALERT IIS is a statewide immunization registry which receives immunization records from 97% of Oregon private healthcare providers and 100% of the immunization records from public providers. The OHP is Oregon's public healthcare plan that provides healthcare coverage for children in families living below 185% of the federal poverty level and covers both those with traditional Medicaid eligibility as well as an expanded State Children's Health Insurance Program (SCHIP) population. The majority of OHP-enrolled children are placed in commercially available managed care plans. Immunization records for OHP children are available both from OHP collected records for billing and encounters as well as from direct provider record submissions to ALERT.

2.2. Data Setup

For this study, ALERT IIS records and OHP encounter records were selected from their respective data systems for children born in 2005. Immunization records and encounter data were restricted to those received through the child's 24th month of age. Children's records were merged across the two data systems based on the child's name, date of birth, and county of residence. The matching process was based on the observation that within the 2005 Oregon birth cohort, a combination of name and date of birth was over 99.9% unique, with almost all exceptions resolving with the inclusion of residence. This high-probability matching process was selected over the usual process ALERT uses to incorporate OHP and other administrative data, wherein a hard-match is required also on additional information such as address or phone number.

Immunization records were selected for the six vaccines (including combination vaccines) included in the recommended “4:3:1:3:3:1” series. The 4:3:1:3:3:1 series consists of 4 diphtheria, tetanus toxoid, and acellular pertussis (DTaP); 3 poliovirus (IPV); 1 measles, mumps, and rubella (MMR), 3 Haemophilus influenzae type b (Hib); 3 hepatitis B (HepB), and 1 varicella. (VAR) vaccines. Records for these vaccines were reviewed for appropriate age and interval between doses, according to the 2007 ACIP Immunization Schedule. Doses given too early or with insufficient spacing between doses to be considered valid were removed from the analysis.

To ensure that records of encounters would be available to compare with immunizations at all of the milestone ages, children with limited enrollment or nonenrollment at key ages were excluded from the final analysis dataset. The study population was restricted to children enrolled in OHP within 30 days of birth, with a cumulative total of at least 365 days of enrollment by age 2, and continuous enrollment across the key period of 15 to 18 months, when the 4th dose of DTaP is due. Because OHP enrollment generally occurs in 12-month blocks, the majority of children meeting the above requirements were also continuously enrolled through their second birthday. Children born outside Oregon were excluded since possibly both early encounters and immunizations would not be reported to ALERT or OHP. Children with only a birth dose of hepatitis B and no other vaccines in ALERT also were excluded.

The OHP requires health plans and providers to submit detailed encounter records on enrolled children for all services received. For OHP-enrolled children, a subset of vaccination-eligible encounters was created from all encounters, based on a review of ICD-9 and CPT coding in OHP encounter records. A vaccination-eligible encounter was defined as an encounter occurring in a nonemergent or noninpatient setting with a medical provider, and with either a CPT procedure code indicating that routine care or evaluation was performed, or an ICD-9 diagnostic code indicating that the purpose of the encounter was consistent with routine care and immunization evaluation. These criteria were used to identify not only visits that providers would define as well-child visits, but also other visits during which immunizations could have been given, and include nonemergent “sick” visits. In a few cases, ALERT had a record of an immunization visit for which there was no matching OHP encounter record. This was usually found to reflect free vaccinations without administration fees at sites outside of those normally reporting to OHP, such as school clinics, and some public health departments. These visits were also counted as vaccination-eligible encounters. Also if CPT codes for vaccine administration were found in OHP data without other evidence of a shot-eligible encounter, these were taken as indicating that a vaccine encounter occurred.

2.3. Analysis

The definitions of milestone periods were taken from Luman and Chu [11], and reflect the dates at which recommended immunizations are first late according to the 2007 ACIP schedule. The milestone periods occur at the start of 3 months, 5 months, 7 months, 16 months, 19 months, and 24 months of age. Immunization (UTD) status at each milestone was evaluated for the timely receipt of all doses due in the 4:3:1:3:3:1 series by age 2. Additionally children were categorized according to whether they had vaccination-eligible encounters in the period prior to each milestone. In cases in which a non-UTD child at a milestone had multiple encounters in the prior period, and received vaccinations at some encounters and not at others, they were counted as having a vaccination visit. Schematically, this classification is presented in Table 1.

Table 1

Immunization Encounter Classification.

	Had immunizations by milestone?
Had provider encounters?	Yes, UTD	Yes, Non-UTD	No
Yes	Immunization Visit	Missed Opportunity	Missed Opportunity
No	—	—	Missed Visit

Where UTD is up to date status for ACIP recommended immunizations in the DTaP, HepB, Hib, Polio, and Varicella series by each milestone age.

At each milestone age, immunization status was compared with the immunization status at the previous milestone age to determine whether children had remained UTD, remained non-UTD, fallen behind due to a missed visit, fallen behind due to a missed opportunity, or caught up with the immunization schedule. Children remained UTD if they were UTD at the prior milestone age and UTD at the current milestone age. Children remained non-UTD if they were non-UTD at the prior milestone age and non-UTD at the current milestone age. This classification is presented in Table 2.

Table 2

Immunization Status Categorization.

	At prior milestone
At current milestone	UTD	Non-UTD
Up to date (UTD)	Remain UTD	Catch Up
Non-UTD	Fall Behind	Stay Non-UTD

UTD is up to date status for ACIP recommended immunizations in the DTaP, HepB, Hib, Polio, and Varicella series.

Children fell behind due to a missed visit if they were UTD at the prior milestone age, non-UTD at the current milestone age, and had no record of a valid healthcare visit or immunization record during the period in between. Children fell behind due to a missed opportunity if they were UTD at the prior milestone age, non-UTD at the current milestone age, and had a vaccination-eligible healthcare visit during the period in between. Children caught up if they were non-UTD at the prior milestone age and UTD at the current milestone age. An exception to the missed opportunity calculation is for the 16-month milestone, which includes the first MMR vaccine. The MMR is not valid before 12 months of age, so encounters between 7 months and 12 months were not counted as potential missed opportunities.

As a check on the completeness of immunization visits represented by this merged dataset, a Lincoln-Peterson capture-recapture method was used to estimate the percentage of immunization visits for the study population not captured by the ALERT IIS either by provider records or by OHP administrative records. The total number of immunization visits, both captured and uncaptured, was estimated by

where N is the total estimated number of visits, A is the number of visits captured by provider reports in ALERT, B is the number of visits captured by OHP billing and administrative reports, and AB is the number of visits captured in both by date.

The principal assessment tool of this study is a time-based progression of young children across milestones and age-appropriate immunizations, presented in a novel form for easier depiction of change between milestones. The data by milestone are presented for whether children were complete on age-appropriate immunizations along with categories for catching up and falling behind by milestone, and by missed opportunities versus missed visits for non-UTD children. Finally, a comparison of age-appropriate milestone results is made to a summary UTD measure for the 4:3:1:3:3:1 immunization series assessed at 24 to 35 months of age.

3. Results

Of 20,411 children born in 2005 who were enrolled in the Oregon Health Plan for some period of time, 13,199 met the requirements to be counted among the study population. The reasons and scope of exclusions are presented in Table 3. The numbers of those excluded are listed in the order they were excluded and do not reflect the total prevalence of each criterion in the population; for example, of the 1,004 children excluded for being born out of Oregon, the majority would also have been excluded for length of enrollment. The primary reason for exclusions from the study population is nonenrollment after 1 year of age, so that no encounter data would be reported to OHP. Also 2% of the study population met all enrollment criteria except that they did not have any reported immunizations. These may represent true nonimmunized children, and were also excluded.

Table 3

Population Size and Exclusions.

OHP children in ALERT:	20,411
Exclusions (in order applied)
Not born in Oregon	1,004
<180 days of enrollment by age 2	1,123
<365 days of enrollment by age 2	583
Not enrolled within 30 days of birth	345
Not enrolled after 1 year of age	2,884
Not enrolled through 15–18-month period	1,009
No immunizations reported post birth	264
Total exclusions	7,212
Total study population	13,199

OHP is the Oregon Health Plan ALERT is the ALERT Immunization Information System.

The count of total immunization visits across both datasets was 74,919. The capture-recapture estimate of total immunization visits for the study population was 76,087. Thus the dataset appears relatively complete for all immunization visits of the study population, with the combination of ALERT IIS provider reports and OHP administrative data capturing 98.4% of estimated immunization visits among the study population.

As shown in Figure 1, only 32% of children had all ACIP recommended immunizations on time at all milestones, while 14% were not complete at only one milestone, 15% were not complete at 2 milestones, and 9% were not complete at any milestone. Also 41% of children had vaccination-eligible encounters in all of the periods before each milestone, and 35% of children were missing encounters in only one of the periods before milestones. Another 14% were missing encounters in 2 periods, and 10% were missing encounters in 3 or more periods.

Figure 1

Children by Count of Milestones with All Age-Appropriate Immunizations.

Rates of completeness of age-appropriate immunizations per the ACIP schedule varied among the milestone ages, from a high of 82.4% at 3 months of age to a low of 52.5% at 19 months of age. For the final milestone at 24-months, no further vaccinations were due, and the final 24 month completion rate for the study population was 68.6%, with 16.1% catching up from the prior 19-month milestone. The pattern of completion, falling behind, and catching up by milestone period is presented in Figure 2.

Figure 2

Immunization Completeness At Milestone Ages (for Two Year Olds in 2007, N = 13,199).

While 17.6% of the study population had fallen behind by the 3-month milestone, representing a late start on immunizations, the most salient episode of falling behind occurred at the 5-month milestone, where 21.0% of children fell behind. In this analysis, children fell behind at all milestone ages at which new immunizations were added. The 19-month milestone adds 4 antigens beyond the 16-month milestone, including the fourth DTaP and varicella; and the risk of falling behind between these milestones is calculated from Table 2 as (16.7/61.3) = 27.2%. This is interpreted as, for those who are on schedule at 16 months, 27.2% will fall behind by 19 months. The largest total percentage of children without age-appropriate immunizations, 47.5%, also occurred at the 19-month milestone. Overall, the percentages of children who were not complete by milestone from Table 2 with missed opportunities was 68.5% at 3 months, 72.2% at 5 months, 71.7% at 7 months, 72.5% at 16 months, 58.4% at 19 months, and 60.0% at 24 months.

Another approach to interpreting the reasons for children falling behind is to further examine missed visits, missed opportunities, and vaccination visits at each milestone for those who were not complete. Figure 3 describes how children who are not complete for age-appropriate immunizations at any milestone have either missed visits or missed opportunities, where missed opportunities are divided between provider encounters with no shots received versus encounters where some shots are received.

Figure 3

Provider Encounters by Milestones for Children Lacking Complete Age-Appropriate Immunizations.

The percentage of noncomplete children per milestone with provider encounters on which some shots were received (vaccination visits) ranged from a high of 50.8% prior to the 3-month milestone to a low of 15.2% before the 24-month milestone. The percentage of children with provider encounters and no shots, and without vaccination visits, in each period is a measure of the amount by which immunization-record-only data would misclassify missed opportunities as missed visits. This potential misclassified percentage of noncomplete children having vaccination-eligible encounters with no reported vaccinations ranged from a high of 44.8% at the 24-month milestone, to a low of 21.4% at the 5-month milestone. At the 3-month milestone, noncomplete children were evenly divided between those with no encounters on record during the period (31.4%), those with encounters but no vaccinations (34.1%), and those with encounters during which some vaccinations were given (34.5%).

As a final analysis, the results by milestone for encounters and completeness were stratified by children's status on a summary UTD measure for having all shots in a 4:3:1:3:3:1 series by age 24 to 35 months, and using a fixed date of assessment. Overall 77.8% of the study population were UTD by 24 to 35 months for the 4:3:1:3:3:1 series. The comparison across milestones for children who were UTD is presented in Figure 4.

Figure 4

Milestone Completeness and Provider Encounters for Children UTD for 4:3:1:3:3:1 Series by 24 to 35 Months.

Of the children who were UTD by the date of assessment, only 68% were complete for age-appropriate immunizations by the 19-month milestone, 72% were complete by the 16-month milestone, 67% were complete at the 7-month milestone, and 71% were complete at the 5-month milestone. The majority of those who were not complete at any milestone, with the exception of the 24-month milestone, also had encounters with providers on which some shots were given.

Figure 5 presents the same analysis across milestones for the 22.2% of the study population who were not UTD for the 4:3:1:3:3:1 series at 24 to 35 months.

Figure 5

Milestone Completeness and Provider Encounters for Children Non-UTD for 4:3:1:3:3:1 Series by 24 to 35 Months.

Children not UTD at 24–35 months were also not complete for age-appropriate immunizations at 19 and 24 months by definition of which shots were required at these milestones. Overall in Figure 5 the majority of non-UTD children at 24 to 35 months who also were not complete at milestones had substantial volumes of encounters with providers. An analysis using only shot-record data, however, would reach, falsely, the conclusion that the majority of non-UTD children were missing provider encounters at each milestone.

4. Discussion

The reality of children's immunizations in the present study population is a story of falling behind and catching up with recommended immunizations. This finding is consistent with prior application of a milestone age approach [11]. What this study adds to the understanding of milestones and immunizations is a more accurate representation of how missed opportunities and missed visits contribute to children not being up-to-date for recommended vaccines. The present finding that for many children, periods of missed visits in immunization record data are actually periods of missed opportunities is a first in the analysis of larger, population-based data systems such as immunization registries. This finding should lead at least to caution in assigning reasons regarding why children are not up-to-date according to either state-level immunization registries or other immunization record data, including the National Immunization Survey. The importance of this is that the prevalence of either true missed visits or true missed opportunities should lead to different interventions to improve immunization rates. Focusing on methods to improve rates of missed visits, such as reminder-recalls to parents of children who appear to have missing vaccinations and visits, may be of limited utility if the greater issue is that the parents have brought their children in to providers across milestones without receiving needed vaccinations. As a recommendation to correct this problem, immunization record data for at least a sample of covered children should be compared with encounter records from billing and administrative sources before considering appropriate immunization interventions. Also, data collected from samples of provider records for immunization assessment should include basic information on all encounters, whether vaccinations were given or not.

In this study, the majority of children who were not catching up at each milestone were having encounters with providers. These encounters were potentially ones in which missing vaccinations could be administered; however, converting these missed opportunities to vaccination visits may be difficult. Provider reluctance to administer vaccinations during sick or other nonroutine visits is a known barrier to improving immunization rates, and may be difficult to change [12-14]. The type of encounter may also be a barrier to receiving vaccinations in many clinics and healthcare providers; for example, in urgent care encounters, when limited time and a press of higher priority needs make it difficult to include review of records and delivery of vaccinations [15-17]. While parental reluctance in such circumstances is likely a factor, at least one study has found that the barrier in such visits is more likely to be provider-based than parental [18]. Also, reimbursement levels may not be sufficient to encourage providers to expand immunizations outside of well-child visits [19]. Finally parents who bring their children in for sick visits or other encounters without immunizations may easily believe that their child has received all needed care, including immunizations. Most parents of children who are not UTD believe their child has received all needed immunizations and may not understand the difference between well-child and other types of encounters [20]. Solutions to this problem may lie in the redesign of early childhood care encounters within clinics and healthcare providers that are concerned about their immunization rates, to deemphasize urgent care or access to short, single-purpose visits in favor of longer appointments during which aspects of routine care such as immunizations are also reviewed.

A strength of the present study is the combination of administrative data on all encounters with immunization records reported separately to the ALERT IIS. This approach could potentially serve as a standard for the evaluation of immunizations given to health plan participants and public populations in areas that have strong immunization information systems such as ALERT. A similar approach by Dombkowski et al. [21] has previously demonstrated the utility of combining registry and Medicaid data in Michigan for assessing missed opportunities to vaccinate asthmatics against influenza. The potential for missed opportunities to be misclassified as missed visits when conducting milestone analysis solely from immunization record data without all encounters should lead to caution in interpreting the balance of responsibility between parents and providers for children not being up-to-date. Also the present study does not address the extent to which parental reluctance to accept all age-appropriate immunizations may limit the ability of provider-based interventions to improve milestone immunization completeness.

From the perspective of a state immunization program with concerns for improving immunization rates, the development of a roadmap showing where and how children are falling behind is invaluable for setting policy. While national measurements such as the NIS can identify variations in rates between states, state-level programs have been left on their own to identify what in-state factors are affecting their rates [6]. Also because immunization levels are often taken as a measure for overall quality of care in early childhood [1], counting doses by age two is not as strong a proxy as is the checking of timely receipt of age-appropriate immunizations across the entire period from birth to age two. A risk of solely depending on immunization results at age two is that UTD and non-UTD status may be taken as discrete categories, irrespective of age-appropriate history. Searching for explanatory factors for these two categories may be misleading for developing an understanding of where barriers exist and where interventions are needed. This is illustrated in the present study by the observation that only a minority of children were consistently on schedule at all milestone ages, and that the majority fell behind at one or more point in receiving immunizations. Falling behind by milestone period is a more useful concept for intervention than final UTD status. A useful model then is that most children are at great risk of falling behind at many points, and that their final status reflects the work that providers do to catch them up to standard.

The concept of milestone ages and the charting of children's progress through the milestones, as advanced by Luman and Chu [11], provides such a roadmap for use by local programs. Local variations in patterns of falling behind and catching up, however, argue for analyzing milestones with available local or state-level data to determine where problems are most salient. Yet, while specific findings may differ, the present study confirms the utility of the milestone approach for a local population.

The present study is representative only of a single state population, and of children enrolled through the Oregon Health Plan. Because children in the OHP are generally enrolled in the same health plans, with the same networks of providers and benefits, as privately insured children, their encounters are potentially similar to the wider state population. Overall in 2007, the OIP estimated that among all OHP-enrolled 2-year-olds, the UTD rate for a 4:3:1:3:3:1 series was 75.2%, as compared to 72.9% among non-OHP-enrolled children. However, the present study population also reflects a group with stable, long-term enrollment in OHP. Children with short-term enrollment or who disenrolled after age 1 are not represented and may have substantially different patterns of falling behind and catching up to recommended immunizations. Also it is expected that individual health plans under the OHP are a significant factor in the receipt of age-appropriate immunizations; however, this information was not included in the present analysis dataset.

Another limitation on the present results is that the definition of encounters was deliberately set broadly, to reflect any encounters in which vaccinations could have been delivered as opposed to well-child visits, during which immunization screening should be routine. As such, the possibility of raising immunization rates by converting all missed opportunities here should be taken as an upper figure to what is possible. Institutional, scheduling, reimbursement and parental acceptance are all potential factors on what proportion of encounters without vaccinations could incorporate immunization screening.

Also, encounters were not stratified by type of provider or principal reason for each encounter. While some research suggests that provider type is not a key factor for immunization performance when the volume of well-child visits is taken account [22], no provider information was included in the present study dataset to confirm or identify other relevant provider features. Whether encounters without immunizations are due to children using a spectrum of different provider types, to parental reluctance, or are due to use of settings such as urgent care in place of scheduled well-child visits cannot be determined from the data of this study.

5. Conclusion

The milestone approach to evaluating early childhood immunizations provides a useful perspective for understanding the time-based progression of children through immunization periods. However, the results of this study warrant some caution in the use of immunization record data only in assessing failure to have age-appropriate immunizations because of the chance of misclassification of missed opportunities by providers as missed visits by parents. Nevertheless, for local assessment by public agencies or health plans, and for the design of interventions to improve immunization rates, looking at the patterns by which children fall behind or catch up on immunizations at milestone periods is a valuable next step beyond the count of vaccine doses received by age two.