Tennessee's 3-Star Report: Using Available Data Systems to Reduce Missed Opportunities to Vaccinate Preteens.

All preteens should receive tetanus-diphtheria-pertussis vaccine (Tdap), quadrivalent meningococcal vaccine (Men-ACWY), and the human papillomavirus (HPV) cancer vaccine series. In Tennessee, HPV vaccination rates have stagnated at low levels for a decade. Three fundamental strategies to reduce missed opportunities for immunization include administering all recommended vaccines at the same visit, making strong recommendations for vaccines, and auditing and feedback. In Tennessee, during each summer, a surge of preteens visit local health departments (LHDs) to receive a required Tdap vaccine before entering seventh grade, presenting an opportunity to administer Men-ACWY and HPV. The Tennessee Immunization Program (TIP) coined the term "3-Star visit" for such encounters and developed a monthly report to track them using data from the Patient Tracking Billing Management Information System (PTBMIS) used by LHDs across Tennessee. Implementation of this quality improvement report has correlated with a substantial increase in 3-Star visits from 2013 to 2016, particularly during the summer months.

Introduction

Vaccines recommended by the U.S. Centers for Disease Control and Prevention for preteens and teens can prevent serious and life-threatening conditions1; however, immunization providers have failed over the last decade to reach the high rates of immunization in this age group that have been achieved for vaccines given in infancy.2,3 Three vaccines are recommended for all children aged 11–12 years: a booster dose of tetanus–diphtheria–pertussis (Tdap) vaccine,4 a dose of quadrivalent meningococcal vaccine (Men-ACWY),5 and a three-dose series of vaccine against select strains of the human papillomavirus (HPV) capable of causing a spectrum of diseases, including several types of cancer in men and women.6–9

Despite the burden of preventable disease, preteen immunization coverage for HPV has, in particular, remained unacceptably low, even as Tdap and Men-ACWY coverage rates have increased.2 In the CDC’s 2014 National Immunization Survey-Teen, Tennessee performed poorly in the measures of HPV vaccination for both males and females aged 13–17 years. Among males, Tennessee fell in the bottom quartile of states for both initiation (30.5%, ±8.5) and series completion (14.0%, ±6.6); among females, the point estimate of HPV initiation (47.8%, ±9.8) ranked 49th among the 50 states and series completion (20.1%, ±6.7) was 50th of 50 states, while Tdap and Men-ACWY coverage was 86.0% (±4.5) and 74.0% (±5.8), respectively, which is close to national averages.2

Factors contributing to low HPV vaccine uptake include misinformation raising scientifically unsubstantiated public concerns about its safety, an erroneous belief that the vaccine could lead to early sexual activity or is not needed until sexual activity is expected, and the lack of a strong immunization provider recommendation for administering the vaccine at the age of 11 or 12 years, as recommended by experts.10–13 In the face of an increasing number of young people missing out on this important development in cancer prevention, numerous strategies have been recommended to improve timely HPV vaccination, with several outlined by the President’s Cancer Panel Annual Report 2012–2013.14 Included among these is an emphasis on giving HPV vaccine when other vaccines are given, a version of which is the principle of “bundling,” or administering all three routine adolescent vaccines at the same visit.14–16 By giving all age-appropriate vaccines on the same day and recommending them in the same way, as in the infant immunization schedule, clinic staff can avoid implying that HPV vaccine is somehow optional, questionable, or exceptional. Another principle is the importance of a strong provider recommendation; most parents will follow their healthcare provider’s advice regarding immunizations if the recommendation is clear and unequivocal.14,17 In addition, immunization provider audit and feedback is a well-established, evidence-based, and cost-effective strategy for increasing immunization coverage in general, consisting of periodically assessing providers’ retrospective performance and sharing this information with them.18,19

The Tennessee Immunization Program (TIP) of the Tennessee Department of Health (TDH) developed a quality improvement (QI) program for local health departments (LHDs) across the state aimed at increasing administration of Tdap, Men-ACWY, and HPV vaccines during the same visit at the recommended ages of 11 and 12 years. This simple, low-cost QI program uses audit and feedback focused on bundling of adolescent vaccines, leveraging the existing information system used by the LHDs. This article describes the development, implementation, and outcomes of the QI program as a public health practice case study. Other states or primary care practices could adapt this simple approach to their existing information systems to increase the administration of bundled adolescent vaccines.

Methods

Overview of QI program

Across Tennessee, LHDs offer all recommended childhood immunizations, including adolescent vaccines, through the Vaccines for Children (VFC) federal entitlement program, eliminating cost as a barrier to access. Although few Tennessee children receive all immunizations at an LHD,20 the LHDs are a convenient option for parents during the back-to-school rush, when all students entering seventh grade in Tennessee, who are typically around 12 years old, must have received a Tdap booster vaccine before classes begin in August.21 The rush creates an opportunity for LHD public health nurses to administer all three recommended adolescent vaccines at once. Training around the importance of evidence-based principles of bundling and strong recommendations are provided annually to regional THD immunization representatives, who serve as both auditors and coaches to the VFC clinics in their regions. In addition, all VFC Program participants review strategies for increasing immunization during an annual immunization review conference conducted by state and CDC immunization program staff. To help them remember the three recommended vaccines, the term “3-Star” visit, a reference to the three stars on the state flag, was coined by the TIP in early 2013 to describe a complete first preteen immunization visit.

From the concept of the 3-Star visit came a desire to track these visits to promote performance improvement through regular feedback. In April 2013, the TIP developed a succinct monthly report to track 1-, 2-, and 3-Star visits by leveraging the standard statewide data available on immunizations given at any LHD in the Patient Tracking Billing Management Information System (PTBMIS). Since then, TIP has distributed the monthly report with feedback to state and regional clinical leadership on the performance of their LHDs, tracking trends over time and comparing regions to one another to support their internal efforts to improve and create healthy competition.

Reports are distributed electronically each month to recipients that include state Community Health Services and Family Health and Wellness Division leadership, to key state leaders interested in HPV vaccine promotion, and to regional public health nursing and medical leadership. Recipients use and share the information with LHD clinic staff as desired to evaluate and improve public health practice: no specific response is required by the program.

Population

The monthly 3-Star Report includes children eligible for the VFC Program aged 11–13 years who presented during the month to any LHD and received at least one routine preteen vaccine (Tdap, HPV, or Men-ACWY) during that encounter. Because the report’s intent is to reflect as purely as possible public health nurse performance, children for whom out-of-pocket costs at the LHD could be a barrier, including commercially insured children, were excluded from the assessment. Children under the age of 19 years are entitled to VFC vaccines if they are uninsured, Medicaid eligible, American Indian/Alaskan Native, or if their insurance plan does not cover vaccination as a benefit. Roughly half of all preteens are eligible for VFC vaccines. Underinsured VFC-eligible children were excluded from 3-Star reports because they would be VFC-eligible only for a vaccine not covered by insurance.

Information system

All LHDs in Tennessee, until recently, have used PTBMIS for patient and service tracking. PTBMIS is COBOL based and uses a DB2 database manager that runs on i/OS for an iSeries AS400 computer from International Business Machines (IBM®). PTBMIS is a highly networked and distributed environment involving 13 regional computers into which regional public health data are entered and a 14th computer at the state level that integrates these 13 separate systems for statewide reporting. This system was first installed in 1987 and has been used in its present form statewide since 1998. In 2014, one of the 13 regions, a single metropolitan county health department, replaced PTBMIS with a new electronic health record (EHR) system, resulting in its absence from original 3-Star reports from May through September 2014. As a result of county clinic staff demand, this LHD began exporting the necessary data from their EHR to a Microsoft Excel file and securely emailing the file to TIP. Since then, the county data have been appended to the iSeries data from the remaining 12 regional PTBMIS systems for analysis. Summary data presented here include this county’s data for the entire reporting period.

The report

An automated process for data analysis, report generation, and e-mail distribution was developed using SAS® version 9.3 (SAS Institute Inc.). Each month, as shown in Figure 1, data are extracted and analyzed on every immunization visit by an eligible child during the previous month. Variables include clinic location, appointment date, patient date of birth, VFC eligibility status, and administered immunizations. The generic vaccine CVX codes included in the analysis are 115 for Tdap, 114 and 136 for Men-ACWY, and 62, 118, and 165 for HPV. The number and proportion of visits that were 1-Star (one of the three vaccines given), 2-Star (two of the three vaccines given), and 3-Star (all three vaccines given) are obtained at the county, regional, and state levels. Results are presented in tabular and graphic forms. Reports are output in portable document format (PDF) and distributed via e-mail to state and regional users. A total of 14 reports are generated: one for each public health region and one statewide report. To ensure that recipients understand the report, a brief explanatory document accompanies it each time it is distributed.

Figure 1

Flowchart of steps for creating the Tennessee Department of Health monthly 3-Star Report.

This article presents a sample 3-Star Report for one public health region report for illustration purposes in Figure 2. In addition, descriptive trends in the monthly proportion of 3-Star Visits and Non-HPV 1-Star Visits are reported for January 2013 to August 2016. The proportion of visits was used rather than the absolute number of visits to adjust for differences in population size and patient volume across regions and across time. The numerators and denominators that comprise the proportions are provided in the Supplementary tables. Since this was a QI program in the context of public health practice and not a hypothesis-driven research study with a controlled research design, statistical tests of changes in trends were not performed. In a practice context, the observed changes over time were of programmatic significance.

Figure 2

Example of the 3-Star Report, August 2016, Tennessee Department of Health Northeast Region (NER). Page 1 (left image): a table of results for each county within NER sits above a chart showing the regional results of the 268 eligible patients for the current month and previous 12 months (x-axis). In the chart, bar height corresponds to the number of eligible patients (left y-axis). Colors within each bar represent the proportion of each type of visit: non-HPV 1-Star (red); HPV-only (green/red hatched); 2-Star (yellow); 3-Star (green). The black line with stars tracks the percentage of 3-Star visits over time (right y-axis). Page 2 (right image) displays the statewide results table for the 5,175 eligible patients seen in August 2016; the upper chart shows the number of eligible visits that month in each of the 13 public health regions, using the same colors as the first chart. The bottom chart shows the individual percentages of 3-Star visits in each region for the month, with a horizontal line representing the statewide result of 42% 3-Star visits in August 2016.

Results

Figure 2 illustrates the 3-Star Report layout using data from the counties of the Department of Health Northeast region (NER) in August 2016 as an example. The report begins with a county-level data table of all eligible immunization visits by children 11–13 years of age where at least one of the three vaccines was administered and the proportion of those visits that were 1-, 2-, or 3-Star visits; a total of 269 eligible children in this region were evaluated. One-Star visits are broken down by HPV-only visits and other 1-Star visits. The chart at the bottom of the first page graphically represents the region’s results for the current month and the previous 12 months. Bars represent the number of non-HPV 1-Star (red), HPV-only (green/red hatched), 2-Star (yellow), and 3-Star (green) visits each month. The left y-axis represents the scale of numbers of patients. Stars connected by a line represent the percentage of 3-Star visits each month. The right y-axis is the scale of the 3-Star percentage line.

The second page of the sample report provides statewide information for the month. A statewide data table at the top provides raw numbers and proportions of each type of eligible visit: 5,192 eligible children were included in the statewide report in August 2016. The bar chart showing the results for each of the 13 public health regions is shown separately from the percent of 3-Star visits in each region to simplify interpretation. A straight line indicates that 42% (2,179 of 5,175) of all assessed visits statewide were 3-Star visits.

Figure 3 displays statewide results for the monthly proportion of 3-Star visits (solid lines) for 2013–2015 and for the first six months of 2016. The initial report of April 2013 results was distributed in May 2013; due to PTBMIS changes to vaccine coding in October 2012, it was not possible to obtain comparable baseline data for the summer of 2012. The year-over-year increases in 3-Star visits each summer during the back-to-school rush are clear, with a 63.9% increase in the proportion of 3-Star visits during the peak month of July between 2013 (978 of 3,831 eligible children, or 25.5%) and 2016 (1,352 of 3,239 eligible children, or 41.7%). Certain regions have consistently achieved greater than 50% 3-Star visits during summer months.

Figure 3

Percentage of 3-Star visits each month among VFC-eligible children aged 11–13 years who received a Tdap, Men-ACWY, and/or HPV vaccine at a Tennessee LHD, January 2013 through August 2016. Months of the year are on the x-axis, and percentages are on the y-axis.

Figure 3 also demonstrates the seasonality of 3-Star visits, with greater increases in the summer months. Between September 2015 and August 2016, 16,156 visits of eligible children immunized in LHDs were assessed as part of the 3-Star Report, with 52% of these visits (8,431) occurring in July and August. As 3-Star visits increased during each successive summer, the proportion of HPV-only visits correspondingly increased during the fall and winter months because children initiating the 3-dose HPV series in the summer were advised to return for dose two and three of the HPV series at recommended intervals (1–2 months later for dose 2, 6 months later for dose 3). HPV-only visit data are available in Supplementary tables.

Figure 4 shows a different perspective on the improvement in immunization practice, which is the percentage of non-HPV 1-Star visits during the same period and on the same scale as Figure 3. Here, the consistent decline in the proportion of non-HPV 1-Star visits across years and throughout all seasons is clear. Exact numbers are provided in Supplementary tables.

Figure 4

Percentage of non-HPV 1-Star visits each month among VFC-eligible children aged 11–13 years who received a Tdap, Men-ACWY, and/or HPV vaccine at a Tennessee LHD, January 2013 through August 2016. Months of the year are on the x-axis, and percentages are on the y-axis.

No formal evaluation has been conducted of exactly how report recipients have chosen to use this monthly measurement and feedback. However, during TIP leadership site visits to LHDs across the state, staff have been asked how they are using the reports. One LHD that consistently had the lowest 3-Star rates established a temporary incentive program in February 2014. They provided the nurse with the most 3-Star visits each month a certificate of excellence and a small gift card. In that region, 3-Star visits, which ranged from 0% to 7% between July 2013 and January 2014, spiked to almost 50% in March 2014, before settling to around 20% during all summer months thereafter (results not shown). Other nursing supervisors have reported how they have tried different approaches to convincing parents and teens to receive HPV vaccine in order to increase their 3-Star rates. They also have shared anecdotally how they are pleased when they walk out of a room and know that they accomplished a 3-Star visit that will show up on the next report.

Discussion

The TIP’s 3-Star Report represents a QI tool that leverages the state’s existing public health informatics infrastructure, PTBMIS, to encourage LHD nurses to administer HPV and Men-ACWY in the same visit with the required Tdap vaccine. The implementation of this QI effort has corresponded with a substantial increase in 3-Star visits in LHDs across the state of Tennessee from 2013 to 2016, in particular during the summer surges of back-to-school vaccination visits for students entering seventh grade, peaking at 42% of visits in July and August of 2016. While the findings cannot be interpreted to assert causality, the observed increases in the proportion of 3-Star Visits have programmatic significance as successful performance on this QI measure.

Many states, including Tennessee, have faced challenges to achieve high vaccination rates against HPV compared to Tdap and Men-ACWY, leaving unvaccinated teens to face preventable cancer diagnoses later in life.14 Ideally, HPV vaccines are given at the age of 11–12 years, when the preteen immune system produces a more robust immune response than older immune systems and when very few have been exposed to HPV.22 As a vaccine against a virus that is often acquired through intimate skin-to-skin contact and with potential cancer diagnoses occurring later in adulthood, it may appear easier to postpone or decline for preteens than vaccines against pertussis or meningitis that may seem more immediate threats. Unfortunately, missed opportunities can result in young men and women delaying vaccination until after they acquire a serious strain of HPV or they may never be vaccinated and remain fully susceptible to HPV-related cancers.

Common strategies to improve vaccine coverage within practices that care for preteens and teens include practice-level immunization coverage assessments, reminders for age-eligible patients, and recall notices to patients overdue for vaccinations.18,19 These strategies work best for clinics that serve as a preteen’s medical home. In Tennessee, most LHDs do not function as a medical home, but they do offer immunizations to preteens and teens if they visit. LHDs experience a seasonal surge of preteen immunization visits, with more than half of such visits for the year taking place during the summer back-to-school season when state regulations require all students entering seventh grade to provide proof of a Tdap vaccination. Students without a medical home or whose primary healthcare provider cannot see them promptly may visit any LHD to be vaccinated.

TIP’s experience with the 3-Star Report and the corresponding substantial increase in 3-Star visit rates shows how straightforward audit and feedback on a well-selected performance measure can be used to motivate continuous QI efforts in immunization. Tennessee faced substantial barriers to the use of these more comprehensive traditional measures of immunization performance. Because the state immunization information system (IIS) is a voluntary reporting system, it remains too incomplete for any type of population assessment. Because LHDs are often only an occasional source for immunization, primarily in the back-to-school rush for this age group, efforts on the part of LHD staff must be opportunistic and focused on the single encounter. To our advantage, because TIP has access to standardized data from all LHDs statewide, it is possible to generate consistent reports over time of performance at the state and local levels across all public health sites, generating a sense of friendly competition.

TIP has deliberately encouraged flexible use of the feedback reports. This report allows them to explore different strategies over time. In one case, individual nurses were identified and rewarded for their performance, a rare opportunity in clinics that do not typically examine or highlight frontline clinic nurse excellence. In other cases, individual regions emerged as high performers as a result of other concerted efforts led by LHD leadership teams. TIP encourages regions wishing to improve to learn from more successful peers. The report will continue to be refined to respond to LHD staff feedback.

To our knowledge, the 3-Star report is unique. However, the concepts behind it could be adapted to any individual clinic or network of clinics interested in increasing preteen immunization rates through bundling and interested in rewarding staff for strong recommendations for HPV vaccine. An elaborate system is not necessary to track individual immunization visits.

The 3-Star Report is not without limitations. Because this report tracks only vaccines administered during one visit, it is not possible to know whether a specific child needed fewer than three vaccines. For this reason, no target proportion of 3-Star visits was established. However, if a child comes to the LHD for a school-required Tdap, it is likely that they have not yet had any of the preteen vaccines. In addition, while the report documents HPV-only visits, it does not specifically attempt to track HPV completion rates in individual patients, given the limitations of the information available in the PTBMIS.

Furthermore, since this was a QI project and not a controlled research study, the data cannot be used to test for cause and effect or to prove that observed increases were only due to the 3-Star Report. The analysis cannot account for external factors that may have also contributed to increased parent receptiveness when the public health nurses made a 3-Star recommendation. Throughout the three years since the report was introduced, multiple efforts were ongoing to help parents understand the importance of timely HPV vaccination of boys and girls. Initiatives at the national level have been led by the Centers for Disease Control and Prevention, American Cancer Society, American Academy of Pediatrics (AAP), and other organizations. Several smaller scale efforts in focused geographic areas in the state have been led by the TIP, the Cervical Cancer Free Tennessee coalition, Vanderbilt-Ingram Cancer Center, Meharry Medical College, Cumberland Pediatric Foundation, Knox County Health Department, University of Tennessee, and others. In 2016, one of the manufacturers also introduced a new national advertising campaign directed at parents. Such educational efforts may have made it more likely that parents would accept the recom mendation for on-time immunization with HPV vaccine. However, the relatively stagnant HPV vaccination coverage rates for the state suggest that there has not been a widespread cultural shift in parental acceptance during this time period that would explain the increases that were observed in LHD clinics.

HPV vaccine is capable of drastically reducing HPV-related cancers in the next generation, but has been exceptionally underutilized. While many parents may have questions about it, we also know that most will accept it when it is recommended and used the same way as other routine vaccines.17 A simple QI report such as this places value on the bundled visit and keeps staff focused on the expectation that the three vaccines are given to all preteens at their first immunization visit.

Conclusion

Effective strategies to increase the low rates of preteen immunization against HPV include bundling of all routinely recommended vaccines at one visit, a strong provider recommendation for immunization, and periodic audit and feedback to providers.13,14 The 3-Star Report demonstrates that feedback strategies do not require sophisticated and comprehensive systems to be effective. The measurement of performance at an individual visit focuses the public health nurse’s attention on the manageable challenge of an individual encounter. The key to the impact of this report is its relative simplicity and consistency, as well as its documentation of trends over time and comparisons against other regions. Healthy competition can promote creative ideas that may become best practices that spread to others and increase vaccination rates broadly.

This simple and effective performance QI tool can be replicated by state immunization programs, LHDs, or primary care practices with basic electronic record systems. The principles can even be carried out on a piece of notebook paper posted in a work station. Overall, it achieves its objective without putting undue pressure on busy clinic staff. Regular performance feedback incentivizes desirable immunization provider behaviors that result in preteens receiving all recommended vaccines at a single visit by focusing on and measuring 3-Star success at each patient encounter.

Supplementary tables with the numerators, denominators, and percentages represented in Figures 3 and 4, as well as the numbers and proportions of HPV-only visits each month between January 2013 and August 2016 are provided.