Implementation of a Military Emergency Department Influenza Vaccination Program: Lessons from Failure.

INTRODUCTION: Influenza infection in the United States results in hundreds of thousands of hospitalizations and 12,000 to 60,000 yearly deaths. Influenza season sees a sharp increase in emergency department (ED) patients nationwide, as primary care offices become overwhelmed. Because the ED is unique in its reach of underserved communities, ED vaccination programs can help maximize the number of people protected by vaccination. Influenza vaccination is one of the only vaccines that occurs yearly; therefore, successes with ED distribution of the influenza vaccine can potentially be translated into efforts of vaccinating the U.S. population more efficiently against other viral illnesses, such COVID-19. There has also not been a previous description in the literature of a vaccination program being used in a military setting. The original purpose of this study was to measure the effect of an ED vaccination program on our vaccine penetration and ED length of stay as well as to analyze the impact of provider education on vaccine uptake on vaccine refusal.
METHODS: This was an observational, quality improvement project in the Wright-Patterson Medical Center Emergency Department evaluating an influenza vaccination program set to last from October 1, 2020 to April 1, 2021. Patients were to be surveyed to assess prior vaccination status, identify those at high risk for influenza complications, and to measure the effects of point-of-care education on vaccine acceptance. Separate measurements included average ED length of stay and the study's impact on how quickly the base population could be vaccinated.
RESULTS: The effort was determined no longer feasible on November 20, 2020 because of the significant barriers. Although no data were gathered, we were able to glean important information that is vital in future efforts to implement ED-based vaccination programs. Reasons for program failure were multi-factorial, but were mainly attributed to rapid implementation, issues with Pyxis ordering and vaccine shipments, and vaccine storage capacity. The program also lacked a multidisciplinary implementation team of nurses and technicians, which could have better anticipated critical barriers.
CONCLUSION: Influenza has caused multiple worldwide pandemics, contributed to countless deaths, and continues to be a challenge. ED-based influenza vaccination programs have been trialed to augment the primary care system in their effort to prevent deaths from influenza. The literature has shown that these programs are cost-effective and efficacious, but significant barriers have stunted their widespread utilization. Examining the rapid implementation and failure of this program highlights the importance of implementation models, process and barrier mapping, and proper operationalization. It is also the first such program that has been trialed in a military treatment facility. In consideration of the recent pandemic, successful ED-based vaccination programs can also offer a model for additional dissemination of other vaccines, such as the COVID-19 vaccine. © The Association of Military Surgeons of the United States 2022. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

INTRODUCTION

Over the past decade in the United States, influenza has been the cause of 4.3-21 million doctor’s visits, 140,000-810,000 hospitalizations, and 12,000-61,000 deaths.[1] The annual economic burden has been tremendous, with recent estimates at approaching $11.2 billion, nearly $8 billion of which were tied up in indirect costs, such as absenteeism.[2] Despite widespread availability of the influenza vaccine and its ability to reduce risk of illness by 40%-60%, the National Immunization Survey shows that we continue to fall short of 80% vaccination rate required for herd immunity.[3,4] The resultant spike in patients overwhelms primary care offices, leading to more emergency department (ED) utilization.[5,6] Even since 1987, this has led to the ED being studied as a space to augment vaccination efforts, particularly to reach underserved communities with limited access to primary care.[7,8] Despite continued skepticism, the literature has repeatedly demonstrated case prevention, complication reduction, and cost-effectiveness.[9-11] Successes with ED distribution of the influenza vaccine can potentially translate to efforts to better vaccinate the U.S. population against other diseases, such as against COVID-19. There has also not been a previous description in the literature of a vaccination program being used in a military setting.

The original purpose of this study was to measure the effect of an ED vaccination program on the hospital’s vaccine penetration and ED length of stay. In addition, we hoped to analyze the impact of provider education on vaccine refusal. After a 2-month effort, the program was deemed by the ED leadership running the program to be infeasible because of unforeseen barriers and was decommissioned. The following is an analysis of why the program failed; an analysis that is critical to the success of future endeavors for ED-based influenza programs through the understanding of potential pitfalls that may cause program failure. Upon reflection, success would have been more certain if the team implementing the program had utilized strategies of process mapping and program implementation in addition to having significantly more time to prepare before program start.

METHODS

Study Design

This was an observational, quality improvement project at the Wright-Patterson Medical Center (WPMC) evaluating an influenza vaccination program implemented by the ED set to last from October 1, 2020 to April 1, 2021. On average over a 6-month period, the Wright-Patterson ED sees about 6500 patients, the majority of which are either active duty military, their families, or retirees.

Inclusion and Exclusion Criteria

Any patient older than 6 months of age was eligible for the inclusion in the study. Exclusion criteria were derived from the CDC, but were broadened because of the nature of the ED patient population. Exclusion criteria included: patients who did not speak English or were unable to give consent (e.g., altered mental status), had fever >100 F, are in need for emergent transfer to another facility, are being admitted, and had history of Guillain-Barre or severe vaccine reactions.

Study Procedures

The patient or guardian was provided a survey package on arrival. Study information and protocols were stored in a binder in an open area and were readily available for reference. It included information like the influenza vaccine information sheet, cards linking patients to network pharmacies, and vaccine documentation cards. Completed surveys were to be stored in a locked container, and data were to be de-identified and tabulated in a password protected Microsoft Excel file. Surveys without ID numbers or dates were to be discarded.

The survey package was made of three primary components:

A survey information sheet outlining risks and benefits of study participation.

The primary survey containing questions aimed at interest for receiving the vaccine, relevant past medical history, reasons for potential refusal, and effects of the pre-vaccination information sheet on refusal.

A pre-vaccination information sheet that was a modified version of a form created by the Immunizations Action Coalition (Fig. 1).

FIGURE 1.

Vaccine information sheet. 

Vaccine Documentation, Administration, Source, and Storage

Vaccine administration was to be documented in the Aeromedical Services Information Management System (ASIMS), and vaccine doses were to be stored in the Pyxis medstation refrigerator. Physicians, nurses, and emergency medical technicians/paramedic technicians who had completed an online training course could administer the vaccine.

In most Air Force military treatment facilities (MTFs), there is an immunization clinic that is responsible for a large portion of the routine vaccinations for the active duty, beneficiary, and veteran population. Once they received the vaccine, they were to contact the ED who would retrieve the vaccine and store it in the Pyxis.

Data Analysis Plan

The number of patients hospital-wide who received the flu vaccine over the project time was to be pulled from ASIMS and compared to the same period over the preceding 5 years. Analysis was to include changes in ED length of stay and the effects of the information sheet on vaccine refusal. The data were then to be analyzed by a statistician to calculate any statistical significance primarily using a t-test to compare across groups.

RESULTS

The effort was determined no longer feasible by the ED staff running the program on November 20, 2020 because of significant barriers. At that time, the ED had not yet received any influenza vaccine (51 days post-start date). Although no data were gathered, we were able to glean important information that is vital in future efforts to implement ED-based vaccination programs. Some of these barriers were foreseen and addressed in the implementation structure, and others occurred during process implementation. These barriers were investigated in-depth to provide guidance for the formation of future programs.

Anticipated Barriers

Higher acuity population

Because of the higher acuity nature of the ED population, the exclusion criteria were broadened beyond CDC recommendations. In addition to higher medical acuity, other aspects were considered by the ED team such as concerns for delayed patient transfer or potential to cause fever that could confound clinical inpatient medical care. Addressing this barrier, communication is required between the immunizations clinic regarding their exclusion criteria for administering the influenza vaccine and the ED leadership about their concerns of potential interruptions to workflow. Once this barrier was addressed, the ED leadership more readily supported the program implementation.

Staff concerns with appropriateness

This was a significant barrier that not only affected this project, but also would affect all efforts to implement similar programs at other facilities. The primary issue came from other physicians, with the core of the argument being that “we are not primary care physicians, and this should not be our responsibility.” To counteract this, in-person and emailed staff education was sent aimed at discussions of the current literature and necessity. The full ramification of this issue is unclear as there was no true data collection. From what has been shown in the literature, this barrier manifests primarily in physicians refusing to order vaccines and/or refusing to participate in the implementation effort because of this perception.

Staff concerns for increased task saturation

This was a concern that has occurred in other studies and was one of the biggest factors that negatively impacts staff willingness to participate in these programs. In addition to providing staff education through in-person discussions and email, we also added measurements of ED length of stay into the study design. This recognition of possible effects helped to reassure the staff that their concerns were being heard. As stated with the previous barrier, the full effects of this barrier could not be measured because the implementation effort was terminated.

Changes in shift triage procedures

During the day, there was a dedicated triage worker at the front desk who could provide the patient with a face sheet. Outside of this time, the patients did not receive the face sheets and triage is completed by the primary nurse. Because of this variation, a dual system was developed for enrolling the patients in the study and administering the vaccine during the night shift.

Clarification of staff members able to administer vaccine

In many MTFs, many of the ED technicians are also emergency medical technicians or paramedics, allowing them to serve as part of the pool of providers credentialed to administer the vaccine. All ED staff, including all nurses and technicians, were required to complete the online training to be certified to administer the vaccine.

Concerns for errors in documentation

It is not routine for the ED staff to document routine vaccinations in ASIMS, and there were concerns that implementation of the program would lead to documentation errors. To address these concerns and mitigate documentation errors, staff were educated about the ASISM system in addition to being provided standard operating procedures readily available in the ED.

Unanticipated Barriers

Rapid implementation

Rapid implementation is a seldom considered barrier in military medicine that profoundly affected all aspects of the program. The time between program inception and implementation was only 4 weeks. On average, implementation of similar programs has taken about a year. Many of the barriers and systems issues discovered in this effort may have been discovered prior to program start if more time was given to properly plan and research. We did not foresee the degree to which lack of time for planning affected the project’s feasibility.

Issues with Pyxis ordering

This barrier was seen as one of the primary contributors to the intervention’s failure. In our MTF, vaccines are provided by two separate sources. The first was from the immunizations clinic, who provided all routine vaccinations and had reserves of rabies and tetanus vaccine, and the second was from the pharmacy. The refrigerator was locked by the Pyxis system and contains other medications required to perform emergency care. The issue became that the vaccine for the study came from the immunizations clinic, and not from pharmacy, meaning that there was no associated order in the Pyxis system. A generic “ED” order would have to be used to open the refrigerator instead of an order linked to the patient’s chart. This would also mean inventory would have to be kept by the ED staff on paper, which increased the risk of user error.

Irregularities with vaccine shipments

This barrier was one of the primary contributors to the intervention’s failure. There was uncertainty as to what vaccine we were receiving, even in late September before the project began. Many civilian institutions and pharmacies received their influenza vaccines in early September, but our MTF did not receive vaccinations until late October. Once the vaccine was received, it was in the form of multi-dose vials, which cannot be used in the ED setting. Single-dose vials were not received until late November, and it was only a few hundred doses. Most of these doses were used by the immunization clinic for active duty members before the project implementers were informed of their availability. In addition to their late arrival, the immunizations clinic was unsure of the quantity and the timing of the next vaccine shipment. This effected the program’s bottom line, and without steady vaccine supply the program was deemed not worth the cost of sustainment. This irregularity was investigated and found to be secondary to two main factors.

First, the orders for influenza vaccine are placed by each Air Force MTF about a year in advance. This program was only given 4 weeks from inception to start time, and so additional vaccinations were not ordered. If given more time, the number and correct type of vaccine could have been planned for. Second, all orders for influenza vaccine from every Air Force MTF are passed to the defense supply agency, who distributes the vaccine. In the summer, the agency begins to receive parts of the entire order from the manufacturer, and as vaccine becomes available it is evenly distributed among all MTFs. For reasons that are unknown, the agency does not receive the largest parts of the orders until almost December, which is about the time the MTFs also receive their vaccine (accounting for the delay we experienced).

Storage size

The WPMC ED is relatively small and there were issues with the size of the refrigerator linked to our Pyxis system. As there was limited space, the ED could only hold 20-30 vaccines at a time (as other medications that are required for emergency care are in the same refrigerator). This may not be the case at other large institutions, but most of MTFs in the United States more similarly represent rural EDs as opposed to large centers. As the immunizations clinic uses their influenza vaccine supply so quickly, once the ED administered their small supply, it would be unlikely for them to immediately obtain more. They would have had to wait until the next shipment to continue the program, with possible time frames for new shipments being weeks to months.

External pressures

One of the ED’s program implementers was a colonel and the main driver behind the program’s inception, creating external pressure for the program’s success. When concerns for the programs’ viability arose, there was a push to continue the program by creating “workarounds.” There were pushes to obtain a larger fridge when storage became an issue or finding methods to create connections between immunizations and the Pyxis to solve our inventory issues during the intervention. These were ultimately not implemented because of concerns that this would go beyond what other MTFs would be willing to do, limiting the program’s scalability. There were also concerns about lack of time to create the workarounds, as they were being implemented mid-program.

DISCUSSION

The understanding of why this program failed is vital for the success of future similar programs within the military medical system. This program was based on successes and failures in the civilian sector, as a thorough search of the literature revealed no previous programs in MTFs. The critical barriers the ED team experienced were likely caused by inadequate pre-implementation planning, likely caused by an underestimation of the significant differences between MTFs and compared to civilian facilities. In addition to significantly more planning time, success would have been more certain if validated methods were used that aimed at determining if the innovation was a good fit for WPMC ED.

One such implementation model is called The Hexagon,[12] a tool validated by the National Implementation Research Network. It was originally designed for use by a team evaluating a program as a consultant, but it has all the essentials of a successful program. Another such model that can be used to plan an implementation is the Iowa Model of Evidence-Based Practice.[13] These models should be used to focus research questions, create more sophisticated designs, and better tailor the implementation to their specific work environment.

Barriers: One Step Ahead of the Game

While working within an implementation framework, addressing possible barriers is an essential next step. As demonstrated by this project, failure to consider critical barriers can result in program failure. In addition to a thorough literature review, leaders should create a step-by-step visual representation of the implementation process called process mapping.[14] Healthcare environments are all unique, and using the process mapping allows program leaders to tailor programs to their individual work environments. Ideally, this mapping is done with a multidisciplinary team composed of providers, nurses, and technicians in conjunction with ED leadership. If you are using an implementation model, the importance of this team will become clear. Including different levels of healthcare providers on this team provides perspectives and ideas that may have been otherwise overlooked. For example, although there was very basic process mapping completed, it was done without a multidisciplinary team of nurses and technicians because of the constraints on time and staffing. This led to overlooked barriers that nurses may have been more aware of, such as issues with vaccine storage and with obtaining vaccine from the Pyxis system.

Core Components: The Essential Ingredients

Formal process mapping leads to ways to incorporate essential project components, called core components, which are essential for optimal functioning. As with barriers, many core components are elucidated by a thorough literature search, taking care to tailor them to each individual healthcare environment. Process mapping is then used to visually place core components at different places along the map, purposefully using the strengths of the system you are working with to your advantage.[15,16]

After identifying the program’s core components, they need to be expressed in a way that can be taught and assessed, a method called operationalization. This is done by creating a practice profile, which creates a standard operating procedure for each component and lays out what is expected of those performing said component. In this project, a core component was active engagement and education of staff. If a practice profile was created on this education piece, then those who were providing education to staff would clearly understand what is expected and what is unaceptable.[16] An in-depth discussion of core components and practice profiles of ED-based influenza vaccination programs is beyond the scope of this paper.

CONCLUSION

Influenza has caused multiple worldwide pandemics, contributed to countless deaths, and continues to be a challenge for our health care system. ED-based influenza vaccination programs have been trialed to augment the primary care system in their effort to prevent deaths from influenza. The literature has shown that these programs are cost-effective and efficacious, but significant barriers have stunted their widespread utilization. The military medical system also offers unique barriers that have not been well elucidated. Examining the rapid implementation and failure of this program highlights the importance of implementation models, process and barrier mapping, and proper operationalization. To our knowledge, it is also the first such program that has been trialed in the ED of an MTF. In consideration of the recent pandemic, successful ED-based vaccination programs can also offer a model for the additional dissemination of other vaccines, such as the COVID-19 vaccine.