An Evaluation Tool for National-Level Pandemic Influenza Planning.

The historical threat of pandemic influenza and the circulation of novel influenza viruses have led countries to strengthen their efforts in pandemic influenza preparedness planning. A cornerstone of these efforts is the creation of a comprehensive national plan that addresses all of the capacities required to prevent, detect, and respond to novel influenza outbreaks. In 2017 and 2018, the World Health Organization issued updated guidance for national pandemic planning efforts, based on lessons learned from the 2009 influenza A (H1N1) outbreak, the International Health Regulations (World Health Organization, 2005a), and other developments in health security. We have created a tool to assess national-level plans based on these updated guidelines. This tool will allow for countries to identify both strengths and weaknesses in their national plans, identify capacities and sectors that require improvement, and to help frame the updating or drafting of plans in line with the most updated guidance.

Background

Pandemic influenza represents a significant threat to global health security, with the potential to kill several hundred thousand to several millions of people once a virus begins to circulate (Gates, 2017; Horby, 2018). The 1918 “Spanish influenza” pandemic resulted in an estimated 50–100 million deaths and held major economic consequences (Johnson & Mueller, 2002). Although pandemic influenza outbreaks have only occurred three times in the past century—in 1957, 1968, and 2009—the twin phenomena of urbanization and globalization have resulted in a situation in which the world is more vulnerable than ever before (Alirol, Getaz, Stoll, Chappuis, & Loutan, 2011; Webster & Govorkova, 2006; World Health Organization [WHO], 2005c).

The circulation of novel influenza viruses in animal populations that have a risk of human pandemic potential—such as H7N9, H3N2, and H5N1—have put influenza preparedness at the forefront of health security efforts. An effective and early response to a novel influenza virus outbreak could contain an outbreak prior to it escalating into a pandemic. Accordingly, several international efforts have sought to mitigate the risks posed by a future influenza pandemic through preparedness efforts. These include the International Health Regulations (WHO, 2005a), the Global Health Security Agenda, and the WHO preparedness standards (WHO, 2005a, 2005b, 2005c).

National‐level plans are also critical to responding to influenza outbreaks as they communicate regionally, nationally, and internationally to key stakeholders for each of the core capacities. National‐level pandemic planning begins with drafting plans according to the most recent guidance, testing and evaluating systems in the interpandemic period, and updating plans according to evaluation findings (European Centres for Disease Control, 2018). Numerous countries have publicly available national‐level plans, although many of these plans were published prior to the most recent 2009 influenza A (H1N1) pandemic and have not been updated since (WHO, 2018c).

This is of importance, as WHO influenza pandemic preparedness guidance has shifted over time. The 2005 WHO Global Influenza Preparedness Plan provides a framework for many of the aforementioned plans and orients preparedness efforts around the six WHO pandemic phases. This guidance details five national objective categories for each phase: (i) planning and coordination, (ii) situation monitoring and assessment, (iii) prevention and containment, (iv) health system response, and (v) communications (WHO, 2005c). The document highlights both WHO and national objectives and actions for each phase of the pandemic period.

In 2017, WHO updated and replaced the guidance published in 2005 with the WHO Pandemic Preparedness Framework, Pandemic Influenza Risk Management (WHO, 2017). This was followed by the publication of a supplemental guidance document titled “A Checklist for Pandemic Influenza Risk and impact Management: Building Capacity for Pandemic Response” under the guidelines of the pandemic framework (WHO, 2018a). These revisions sought to integrate lessons learned from the 2009 influenza A (H1N1) pandemic, and incorporate developments from both the International Health Regulation (WHO, 2005a) and the Joint External Evaluation (JEE) Tool (WHO, 2016). The revisions also include risk and severity assessments and advocate for an emergency risk management for health (ERMH) approach to pandemic planning.

While the pandemic influenza checklist is thorough in discussing the essential and desirable elements of nation‐level plans, it evaluates capacities in a binary fashion. However, evaluating capacities beyond this binary checklist is important for providing a more complete account of the robustness of a national‐level plan. The WHO Pandemic Influenza Framework notes that the checklist is an essential part of planning, in coordination with partnership contributions from country ministers and partners, including the United States Centers for Disease Control (U.S. CDC), European Centres for Disease Control (ECDC), and the regional WHO offices (WHO, 2018b). In their respective guidance, U.S. CDC, the ECDC, and the Pan American Health Organization (PAHO) all note the centrality of planning to ensure an effective response to an outbreak (European Centres for Disease Control, 2017; Pan American Health Organization, 2018; United States Centers for Disease Control and Prevention, 2016).

The purpose of this paper is to present a tool to evaluate national‐level pandemic influenza preparedness plans on a continuum and in accordance with the most recent WHO guidance. This tool is not meant to alter the WHO guidance, but to provide a foundation—built upon the WHO framework—from which national authorities can assess existing preparedness plans and analyze for areas of strength, gaps, and systemic weaknesses that must be addressed (Box 1).

Identify the baseline level of preparedness explicitly recorded in national pandemic plans.

Identify and analyze strengths and weaknesses within national‐level plans in each of the required core capacities.

Align national‐level plans with the most up‐to‐date WHO guidance for pandemic influenza.

Methods

We reviewed the essential elements of the 32 subcapacities of the WHO 2018 checklist (Table 1) to identify specific and measurable objectives or actions. Based on this, we designed an assessment tool (Supplementary File 1) with a hierarchical methodology for subcapacities. The tool follows the categorical numbering system of the WHO Influenza Checklist, as outlined in Table 1. There is no category 1.0 because this section serves as an introductory chapter in the WHO checklist document.

Table 1

Summary of 2018 WHO Checklist for Pandemic Influenza Risk and Impact Management—Key Capacities

Core Capacity	Sub‐Capacities
2.0 Preparing for an Emergency	2.1 Planning, coordination, and resources
	2.1.1 Response planning
	2.1.2 Coordination
	2.1.3 Resources
	2.2 Legal and policy issues
	2.3 Ethical issues
	2.4 Risk communication and community engagement
	2.5 Points of entry
	2.6 Travel restrictions
3.0 Surveillance, Investigation and Assessment	3.1 Laboratories
	3.2 Seasonal influenza (interpandemic) surveillance
	3.3 Non‐seasonal (novel) influenza surveillance
	3.4 Outbreak investigation
	3.5 Pandemic surveillance
	3.5.1 Verification and detection
	3.5.2 Monitoring the pandemic
	3.6 Risk and severity assessment
4.0 Health Services and Clinical Management	4.1 Health services
	4.1.1 Health service continuity
	4.1.2 Facilities
	4.1.3 Personnel
	4.1.4 Essential medicines, supplies and medical devices
	4.1.5 Excess mortality
	4.2 Clinical management
	4.2.1 Treatment and patient management
	4.2.2 Infection prevention and control in healthcare settings
5.0 Preventing Illness in the Community	5.1 Medical countermeasures
	5.1.1 Seasonal influenza vaccination
	5.1.2 Pandemic influenza vaccination
	5.1.3 Antiviral drugs for prophylaxis
	5.2 Non‐pharmaceutical interventions
	5.2.1 Personal non‐pharmaceutical interventions
	5.2.2 Community non‐pharmaceutical interventions
6.0 Maintaining Essential Services and Recovery	6.1 Essential service continuity
	6.2 Recovery
7.0 Research and Development	7.1 Research and development
8.0 Evaluation, Testing, and Revising Plans	8.1 Evaluation
	8.2 Testing and revising plans

Subcapacities scores range from 1 to 5, and each subcapacity receives a single score based on the level of detail outlined in a national‐level influenza plan. Similar to the JEE tool, the assessment system is also color coded. Indicators that receive a score of a 1 (i.e., no capacity) are colored red; indicators receiving scores of 2 or 3 (i.e., limited capacity or moderate capacity) are colored yellow; and indicators that receive scores of 4 or 5 (i.e., developed capacity or fully developed capacity) are colored green.

Two research teams independently scored the national‐level plans of Singapore, Norway, Taiwan, the United Kingdom, and the United States to pilot the tool. These plans were purposively selected for geographic diversity and because all plans were published after 2014. Researchers reviewed plans and assigned a capacity score for each subcapacity. Scorers debated major (>2 points) and minor (≤2 points) discrepancies until a consensus score was reached. Based on these discussions, minor changes were made to the language of the tool to refine the tool and assessment process. The piloted results for the United Kingdom are available in the Supplementary File 2.

How Should the Tool Be Used?

The target users of this tool are government officials, including ministers of health, agriculture, officials from respective national public health agencies, and any other relevant stakeholders that participate in national influenza preparedness, surveillance, clinical management, and communication and evaluation efforts. The tool can also be used by researchers and other relevant experts to analyze publicly available plans.

Prior to evaluating a plan, a methodology should be agreed upon by those who will be conducting the assessments. This includes reaching a common understanding of the level of detail required in a plan and the terminology used throughout the tool. Multiple individuals, ideally from different public health‐related sectors, should then individually score the national‐level plan, providing a brief justification for the scores they assign. Following the assessment procedure, these individuals should convene with their results and discuss and any assessment discrepancies until a consensus is reached.

Once the tool is used and an assessment complete, governmental officials can then (i) identify areas in which capacity strengthening efforts are required and seek any necessary assistance from cross‐country collaborators or multi‐sectoral organizations and (ii) draft a new, updated plan based upon the evaluation results.

Discussion

Developing, evaluating, and refining national‐level pandemic influenza plans is important for mitigating the threats posed by influenza viruses with pandemic potential. To our knowledge, our tool is currently the only resource by which countries can quantitatively assess their national plans for core capacities and key indicators that is aligned with the most recent WHO guidance.

Although the tool does not ensure that plans are operational, it does represent a useful resource for starting discussions focused on developing or revising pandemic influenza plans. The quantitative assessment methods used allow for a hierarchal analysis of a country's readiness for an influenza pandemic. The methods also facilitate reaching an understanding of how capacity gaps could be filled so that a country may achieve fully developed capacities. This could hold important implications for prioritizing projects intended to develop pandemic capacities.

Should countries use this tool to score multiple plans over time, governments can also hold themselves accountable, both by recognizing areas in which progress has not been achieved and by demonstrating capacity development. This assessment over a period of time could also facilitate the sharing of best practices. Should a country desire to develop a certain capacity, it may seek guidance from other nations on the most efficient way to do so.

There are several challenges related to evaluating a national‐level pandemic plan. First, inputs and knowledge are required across a wide range of fields. The WHO checklist, and by extension our tool, requires considering aspects such as legal and policy issues, points of entry, disease surveillance, clinical management. At a minimum, these require extensive knowledge from a diverse suite of fields including law, government, public health, medicine, and the sciences. A methodology that relies on the judgment of individuals assumes sufficient knowledge of these important considerations. Should an individual not possess this knowledge, a national‐level plan may be inaccurately assessed. This speaks to the importance of including multiple individuals in the assessment process. Conversely, if too many individuals are involved in the assessment, it may prove difficult to reach the consensus required for the assessment procedure. Ensuring appropriate individuals are assessing plans is of great importance.

Second, this tool does not address any real or perceived limitations of the WHO guidance itself. For instance, the WHO guidance does not address areas such as One Health—the intersection of human, environmental, and animal health—that is central to pandemic preparedness. Planning efforts that want to incorporate these approaches must incorporate resources beyond the WHO checklist and our tool.

Third, given the scope of the capacities now required to be included in national‐level plans, many may be detailed in other national‐level or subnational guidance documents. For example, business continuity guidance may be detailed in other documents. However, the tool presented is only designed to evaluate national‐level influenza plans. This challenge may be overcome by agreeing to include these additional documents in the assessment if they are explicitly mentioned. At a minimum, if this guidance does exist elsewhere, an effort should be made to incorporate it into revised national‐level influenza plans.

Throughout our piloting of the tool, there were interpretive differences in assessment methodology. This most often occurred when a plan referenced a capacity or discussed a capacity in detail. For example, when evaluating the required essential medicines list, a discrepancy could arise if a plan referenced the existence of a list versus detailing the list in the plan itself. Differences in interpretation also arose from linguistic barriers. For example, “mapping” used in the context of “mapping of existing public and private healthcare facilities” was not intuitive to non‐English speakers. An inherent limitation of the tool is that the assessment will often rely on the scorer's interpretation of phrases and wording. To help overcome this, scorers should agree on an approach prior to evaluating plans and provide a brief justification of their scores to facilitate the discussion required to reach a consensus score.

Preparedness for pandemic influenza represents a top concern for global health security in today's globalized and interconnected world. However, health policy does not reflect this. Should an influenza pandemic occur tomorrow, the world's response would be dated and lack key considerations. The updated WHO guidance provides a policy window to address these gaps by developing or revising national‐level guidance to improve preparedness for pandemic influenza. Our tool provides one method for quantitatively assessing existing plans to identify which areas need to be revisited in order to comply with the most current WHO guidance.

Notes

Conflicts of interest: None declared.

Corresponding author: Rebecca Katz, Rebecca.Katz@georgetown.edu

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.