Reviewing Health Security Capacities in Nigeria Using the Updated WHO Joint External Evaluation and WHO Benchmarks Tool: Experience from a Country-Led Self-Assessment Exercise.

Across the world, the level of pandemic preparedness varies and no country is fully prepared to respond to all public health events. The International Health Regulations 2005 require state parties to develop core capacities to prevent, detect, and respond to public health events of international concern. In addition to annual self-assessment, these capacities are peer reviewed once every 5 years through the voluntary Joint External Evaluation (JEE). In this article, we share Nigeria's experience of conducting a country-led midterm self-assessment using a slightly modified application of the second edition of the World Health Organization (WHO) JEE and the new WHO benchmarks tool. Despite more stringent scoring criteria in the revised JEE tool, average scoring showed modest capacity improvements in 2019 compared with 2017. Of the 19 technical areas assessed, 11 improved, 5 did not change, and 3 had lower scores. No technical area attained the highest-level scoring of 5. Understanding the level of, and gaps in, pandemic preparedness enables state parties to develop plans to improve health security; the outcome of the assessment included the development of a 12-month operational plan. Countries need to intentionally invest in preparedness by using existing frameworks (eg, JEE) to better understand the status of their preparedness. This will ensure ownership of developed plans with shared responsibilities by all key stakeholders across all levels of government.

Introduction

Infectious disease outbreaks do not respect borders.[1] The speed at which disease outbreaks spread has drastically increased with globalization and ease of travel, as demonstrated by the international spread of diseases including Ebola,[1-3] Middle East respiratory syndrome,[4] severe acute respiratory syndrome,[5] and COVID-19.[6-9] The impact of these outbreaks and epidemics on health systems and populations is substantial, slowing and sometimes reversing socioeconomic gains.[10,11] Several disease outbreaks have been recorded in Nigeria in the last 10 years with imported cases of Ebola in 2014,[12] annual outbreaks of yellow fever,[13,14] Lassa fever,[15-17] cerebrospinal meningitis,[18] cholera,[19,20] measles,[21] and most recently, COVID-19.[22] Infectious disease outbreaks have exacerbated the humanitarian crises continuing in northeast Nigeria, reflecting the interaction between humans, animals, and the environment.[23-25] Consequently, there is a continuous emphasis on strengthening pandemic preparedness while prioritizing response efforts to protect the population from disease threats and ultimately improve health security.[25-27]

In response to lessons learned from the severe acute respiratory syndrome outbreak in 2003, the World Health Organization (WHO) accelerated the revision of the International Health Regulations (IHR), which were adopted in 2005[28] and came into force in 2007. The IHR are essential for addressing global health security[29] and its implementation is underpinned by a comprehensive IHR monitoring and evaluation framework.[30-32] The framework, with its 4 components—State Party Self-Assessment Annual Reporting tool, voluntary Joint External Evaluation (JEE), after-action reviews, and simulation exercises—aims to provide a comprehensive, accurate, country-level overview of the implementation requirements under the IHR. The need for a monitoring and evaluation framework for IHR compliance was highlighted by the inadequacies of the annual self-reporting previously used as the sole indicator of emergency preparedness and IHR compliance.[33] The JEE is an important structure on which to base health security strengthening efforts, even if it does not predict country performance during a pandemic. The most critical aspect to countries, more than the JEE scores, is how countries respond.[34]

Using the JEE to Strengthen Health Security in Nigeria

Nigeria is a state party to the IHR and mandated to report annually on its compliance status at the World Health Assembly. In June 2017, Nigeria conducted a JEE,[35,36] which provided an opportunity for IHR stakeholders across government ministries, departments, and agencies (MDAs) to jointly identify gaps in the country's health security capacity.

Following the JEE,[31] countries[37] use the findings to develop a costed, multiyear National Action Plan for Health Security (NAPHS)[38] aimed at addressing identified health security gaps. In Nigeria, the NAPHS was developed with a 5-year timeline, costed, and launched,[36,39,40] which enabled country stakeholders to implement a plan to strengthen health security capacities using a One Health approach. In Nigeria, developing the NAPHS facilitated access to a credit facility from the World Bank Regional Disease Surveillance Systems Enhancement project.[41] As the national IHR Focal Point, the Nigeria Centre for Disease Control worked with other MDAs to begin the implementation of activities within the NAPHS,[36] with support from the Regional Disease Surveillance Systems Enhancement project[41] and other partners.

Given the long interval of 4 to 5 years before repeating the JEE,[30] Nigeria developed a framework for a midterm JEE to review its IHR core capacities in health security. Many systems improvements had been made since the initial JEE was conducted in 2017 and stakeholders needed to align with a multisectoral plan to address key priorities. The Nigeria Centre for Disease Control sought guidance from WHO and from partner organizations working on IHR implementation that served as national external experts, including US Centers for Disease Control and Prevention, Public Health England, and Resolve to Save Lives.

The country-led, midterm JEE was conducted from November 18 to 22, 2019.[42] The objectives were to: (1) conduct a self-assessment of IHR implementation using the updated second edition JEE tool, (2) review progress and identify challenges in the implementation of the NAPHS, (3) compare scores from the self-assessment with the WHO benchmarks tool to identify core activities for health security implementation and develop annual operational plans specific to the JEE technical areas, and (4) map prioritized health security activities to government and partner resources.

The JEE tool used in 2017 was updated by WHO in 2018.[43] The revised JEE tool and the WHO benchmarks tool[44] now include a minimum set of prescriptions that state parties should aspire to for each technical area to demonstrate sufficient core capacity. Scoring in this updated tool is dependent on multisectoral coordination and accomplished through a One Health approach.[29,43]

By sharing Nigeria's experience, we intend to reemphasize the need for countries to invest in similar efforts that ensure collective contribution to strengthening health security capacities. We envisage that implementing the lessons from this process will stimulate a renewed drive among countries for a self-sustaining model that embodies the tenets of the IHR monitoring and evaluation framework, through self-appraisal. These self-appraisals will potentially lead to the development and execution of feasible national and subnational action plans that support IHR compliance and efforts to strengthen health security and health systems using a multisectoral approach, with all relevant stakeholders.

Country-Led Process for the Midterm JEE

The midterm JEE in Nigeria was a hybrid process that incorporated an internal assessment and review by a team of evaluators from development partners and public health institutions working in health security implementation.

The internal assessment commenced with a preworkshop where technical leads: (1) were updated on the revised second edition JEE and the WHO benchmarks tool, (2) conducted a review of self-assessed JEE scores with feedback provided by a team of subject matter experts on the draft report documenting progress made since the 2017 JEE including the rationale for scores, and (3) prioritized 2 to 3 benchmark actions for implementation in 2020 using the WHO benchmarks tool, based on the implementation challenges observed since the NAPHS was launched in 2018 and on systems performance as measured by real-world events, including after-action reviews of MDAs.[44] Before the midterm assessment, technical area leads reviewed results of previously conducted after-action reviews and simulation exercises to prioritize selected benchmarks, demonstrate progress, and spotlight challenges with implementing recommendations from the first JEE. The benchmark tool, which was used as a planning rather than an assessment tool, helped identify actions to move to the next level. Participation during the midterm JEE included state epidemiologists, given their critical roles in coordinating response to public health events at the subnational level.[45] The evaluation included discussions (at plenary) around the progress and challenges since the development of the NAPHS, and reaching consensus on capacity scores and priority actions for implementation. We used mean scores to determine changes in Nigeria's JEE scores by technical area and indicator between 2017 and 2019 (Table 1).

Table 1.

Changes in Nigeria's JEE Scores by Technical Area and Indicator, 2017 to 2019

Subsequently, a 1-year operational plan for 2020 was developed, and responsibilities were assigned to respective MDAs with a proposed implementation timeline and tracker. Feedback was provided to the country team at a high-level stakeholders' meeting after a team of external evaluators from development partner agencies and host-country experts had obtained full agreement on all aspects of the report findings and recommendations. Core to this process was the wide engagement of key stakeholders across relevant MDAs with a role to play in IHR implementation.

Status of Health Security Capacity Following the Midterm JEE

Of the 19 technical areas assessed, 11 recorded increases in the mean score of their composite indicators compared with the 2017 JEE (Table 1), 5 showed no change in scoring, and 3 showed lower scores (Table 2). No indicator attained the highest-level scoring of 5. Key national priorities for action (Table 3) were identified following the exercise. JEE scores indicate the level of health security capacity on a scale from 1 to 5: 1 = no capacity, 2 = limited capacity, 3, developed capacity, 4 = demonstrated capacity, 5 = sustainable capacity.[43]

Table 2.

Overall Change in Nigeria's JEE Scores by Technical Area, 2017 to 2019

Change[a]	Technical Area	Number of Technical Areas
Increased[b]	National legislation, policy, and financingBiosafety and biosecurityNational laboratory systemReportingEmergency preparednessEmergency response operationsLinking public health and security agenciesMedical countermeasuresRisk communicationPoint of entryZoonotic disease	11
No change	IHR coordination, communication, and advocacyAntimicrobial resistanceSurveillanceChemical eventsRadiation emergencies	5
Decreased	Food safetyImmunizationHuman resources	3

Average score per technical area.

Technical areas increased but have not reached the highest scoring level of 5 (sustainable capacity).

Abbreviation: IHR, International Health Regulations.

Table 3.

Key National Priorities in Nigeria Following the Midterm JEE

	National Priorities	Description
1	Budgetary allocation	Increasing domestic budget allocation, release, and tracking for health securityHarnessing resources in the Regional Disease Surveillance Systems Enhancement project across government ministries, departments, and agencies
2	Improved coordination	Implementing activities in technical areas, which requires leadership from the highest levelImproving coordination across government ministries, departments, and agencies and working closely with the national IHR Focal Point (Nigeria Centre for Disease Control)—legally defined after the JEE in 2017Training and ensuring availability of dedicated staff to prioritize activities identified in the NAPHS for government ministries, departments, and agencies
3	Implementation and monitoring	Provision of oversight by the honourable ministers in relevant government ministries, departments, and agencies (health, agriculture, and environment) through request for IHR/NAPHS implementation updates on a frequent basis

Abbreviations: IHR, International Health Regulations; JEE, Joint External Evaluation; NAPHS, National Action Plan for Health Security.

Prevent

Scores for national legislation, policy, and financing; zoonotic diseases; and biosafety and biosecurity increased under the Prevent core area between 2017 and 2019. The score for national legislation and policy increased from 1 to 2. Two new indicators were added to the financing component of this technical area and served as new baseline scoring. Challenges identified included an incomplete review of the Public Health Act Bill (2013)[46] and the National Health Act (2014)[47] by the legislative arm of government. It was evident that there was an overwhelming need for training of dedicated technical officers across IHR implementing MDAs on the relevance of IHR and awareness of existing guiding policies.

In the zoonotic diseases technical area, scores increased for 2 indicators as follows: indicator P4.1 (coordinated surveillance systems in place) increased from 2 to 3, and indicator P4.2 (mechanisms for responding to diseases) increased from 1 to 3. A key challenge was the discordance between prioritized NAPHS activities and findings from the 2017 JEE. Therefore, there was no mutual accountability for the implementation of these activities by the responsible MDAs. The score for 1 indicator, P6.1 (whole-of-government biosafety and biosecurity) under the biosafety and biosecurity technical area, increased from 1 to 2. Institutionalizing training programs for biosafety laboratory staff was identified as key to addressing the limited numbers of skilled personnel across the laboratories.

Scores for indicators in antimicrobial resistance, food safety, and immunization decreased. A score of 4 was attained for new indicator P3.1 (effective multisectoral coordination on antimicrobial resistance). For indicators P3.3 (infection prevention and control) and P3.4 (optimize use of antimicrobial medicines in human and animal health and agriculture), scoring decreased compared with 2017, from 2 to 1. A major gap was the need to explicitly capture activities that focus on the environment within the One Health operational plan, and publishing documents addressing appropriate use, availability, and quality of antimicrobials to optimize the use of antimicrobial medicines in human and animal health. Key challenges experienced in this technical area included a delay in presidential assent to an amended Animal Disease Control Act, and nonexistent specific regulations for antimicrobial stewardship for both human and animal health.

Scoring for food safety technical area decreased from 2 to 1 but reflected a more stringent and rigorous core capacity requirement. Despite the existence of a framework to respond to and manage food safety emergencies, there was no evidence of the functionality of such frameworks. Furthermore, multisectoral efforts on response to reported food-related outbreaks have been ad hoc, while recommendations in the approved National Policy on Food Safety and Implementation Strategy[48] have been partially implemented. In the immunization technical area, the score for measles immunization coverage decreased from 3 to 2, based on more recent coverage survey data rather than the use of administrative data, which was used in the original JEE. Immunization-related challenges are being addressed through the implementation of a comprehensive approach including: (1) nationwide and targeted measles vaccination campaigns (with the goal of 84.5% national coverage); (2) using community health influencers, promoters, and services; (3) leveraging the newly established Basic Healthcare Provision Fund to improve access to routine immunization via primary healthcare services; (4) coordinating the equitable improvement of immunization coverage through a National Emergency Routine Immunization Coordination Centre; and (5) the approval of a cold chain equipment optimization plan to be implemented by the government of Nigeria with funding support from Gavi, the Vaccine Alliance.

Detect

Most indicators in the Detect core area recorded an increase in scores. Nigeria's highest scores were 4 for D1.1 (laboratory testing for detection of priority diseases) and D3.1 (systems for efficient reporting externally); however, Nigeria's previous high score was 4 for D4.4 (field epidemiology training program or other applied epidemiology training program is in place), which decreased to 3 in the midterm JEE. One new indicator, D4.3 (in-service trainings are available), was added to the human resources technical area.

Laboratory technical area scores increased for 3 of the 4 indicators: D1.1 (laboratory testing for detection of priority diseases) increased from 3 to 4, D1.2 (specimen referral and transport system) increased from 1 to 2, and D1.3 (effective national diagnostic network) increased from 2 to 3. However, there was no evidence that quality improvement guidance documents were followed or available. Sample transportation between levels was poorly coordinated and ad hoc, laboratory information management systems are not available, and overall, there was a poor governance structure in the laboratory system. In the surveillance technical area, scores increased from 2 to 3 for indicator D2.2 (use of electronic tools) and decreased from 3 to 2 for D2.1 (surveillance systems). The decrease in D2.1 was ascribed to the requirement for indicator-based surveillance and event-based surveillance at the subnational level. Challenges in this technical area included minimal involvement of private health facilities in reporting surveillance data and a nonoperational interoperability system at national and subnational levels. Indicator D4.3 (in-service trainings are available) attained a score of 3 while there was a decrease in scoring from 3 to 2 for indicator D4.4 (field epidemiology training program or other applied epidemiology training program is in place). Key challenges included the inability to continue the Nigeria Field Epidemiology and Laboratory Training Program basic level training, having sufficient public health workforce at subnational level, and the creation of an incentivized career path for already trained personnel. These challenges are in addition to difficulties experienced coordinating the cadre of human and animal healthcare counterparts to establish unified human resources competencies.

Respond

Indicators in this core area recorded an increase in scores between the 2 assessments. Scores increased either from 1 to 2, or 2 to 3. Indicators R2.3 (emergency exercise management program) and R5.5 (addressing perceptions, risky behaviors, and misinformation) increased from 3 to 4. In the emergency preparedness technical area, the score increased from 2 to 3 for indicator R1.1 (strategy emergency risk assessments conducted and emergency resources identified and mapped) and remained at 1 for R1.2 (national multisectoral, multihazard emergency preparedness measures, including emergency response plans, are developed, implemented, and tested). The scores of all 3 indicators in the emergency response operations technical area increased between the assessments, including from 3 to 4 for indicator R2.3 (emergency exercise management program). Scores increased from 2 to 3 for the remaining 2 indicators: R2.1 (emergency response coordination) and R2.2 (emergency operation center capacities, procedures, and plans). Improvement in scoring was a result of the implementation of key recommendations from the 2017 JEE, especially around conducting joint simulation exercises between public health and security agencies. In addition, establishing public health emergency operation centers across several states in Nigeria provided a platform for improved emergency response coordination to other disease outbreaks. Challenges in this technical area include limited staff and operational funding at the subnational level for the emergency operations centers. The coordination mechanisms also need to be strengthened as there was no evidence these emergency operation centers were activated within the required 120 minutes of notification of a public health event.

Scoring for indicator R3.1 (public health and security authorities are linked during a suspect or confirmed biological event) increased from 1 to 2. A major challenge in this technical area was the inability to conduct capacity-strengthening simulation exercises for staff across MDAs despite identifying and establishing points of contact across relevant civil MDAs and security formations for the proposed engagements. In the medical countermeasures and personnel deployment technical area, only 1 of 3 indicators increased in scoring: R4.1 (system in place for activating and coordinating medical countermeasures during a public health emergency) increased from 1 to 2.

Of the 5 indicators in the risk communications technical area, 3 indicators showed increased scores. Indicator R5.5 (addressing perceptions, risky behaviors, and misinformation) increased from 3 to 4, reflecting the use of the connect center to collect information on rumors and misinformation. Data from these platforms supported the development of targeted messaging and risk communication approaches to managing rumors during outbreaks. Scores increased from 1 to 2 for the remaining 2 indicators: R5.1 (risk communication systems for unusual/unexpected events and emergencies) and R5.3 (public communication for emergencies). Challenges in this technical area included limited funding for risk communications activities, lack of a monitoring and evaluation framework, and lack of a coordination structure for risk communications at the subnational levels.

Other IHR-Related Hazards and Points of Entry

Scores remained the same for all indicators except POE.1 (routine capacities established at points of entry), which increased from 1 to 3. Challenges identified in this technical area included inadequate national legislative backing for port health services at points of entry, limited cross-border collaboration at ground crossings, and suboptimal all-hazards coordination at points of entry—chemical and radiation components in existing plans and protocols were not all-inclusive or robust enough.

Discussion

Countries should continuously conduct self-appraisal exercises to determine their level of pandemic preparedness and develop realistic plans to strengthen health security core capacities, especially during periods between events. These self-appraisals include system assessments (eg, JEE and State Party Self-Assessment Annual Reporting) and real-world events including after-action reviews. Nigeria has conducted after-action reviews for previous outbreaks,[49,50] which helped identify major gaps in preparedness and response to outbreaks. Both assessments in 2017 and 2019 suggest that more work is required[27,51] to strengthen health security capacities in Nigeria.

The country-led assessment provided an opportunity to reenergize high-level cross-government political commitment to strengthening Nigeria's health security capacities. Furthermore, the assessment enabled the integration of the lessons from previous outbreaks into the larger operational planning process, prioritizing recent and relevant risks, and institutionalizing the recommendations from such exercises. Through these reviews, activities such as the establishment of state public health emergency operation centers were introduced and have proven to be of value in Nigeria's COVID-19 response coordination. Specifically, the public health emergency operation centers have continued to serve as a useful framework for driving the subnational level response to the COVID-19 pandemic and as a common rallying point for a multisectoral response by stakeholders at the strategic, tactical, and operational levels. The country-led assessment reflected a commitment to collective action, self-improvement, learning, and transparency. The country was able to leverage the momentum garnered across key IHR stakeholders to review the status of the activities prioritized in the NAPHS and to self-appraise its performance. This was done while providing data representative of the implementation status of the World Bank-funded Regional Disease Surveillance Systems Enhancement project, as part of midpoint evaluations for the country and the World Bank.

Implementing the IHR is not the sole responsibility of the Nigeria Ministry of Health[33] or its implementing agencies. Although a nascent national public health institute, the Nigeria Centre for Disease Control organized and galvanized support for strengthening health security by bringing key stakeholders together from different sectors like the experience during the 2017 JEE. Consequently, MDAs—especially in health, agriculture, and environment—have continued to work in a continuum irrespective of the mandate of the supervising parent ministry. Sustaining this work will be important especially through the newly formed IHR interministerial technical working group tasked with ensuring ongoing coordination of IHR activities across the MDAs.

The midterm assessment was anchored on transparency, peer-to-peer reviews, and trust—tenets that are core to the JEE. The JEE second edition tool,[43] now more specific (eg, emphasizing subnational core capacities and spotlighting the animal health sector), enabled the collection of new in-depth information on the status of key technical areas while retaining the integrity of the original tool. Updated and more relevant baseline scores were obtained as a result. Due to the integration with operational planning, the exercise provided clear direction in identifying high-priority activities to be implemented in 2020. The use of the WHO benchmarks tool by the team of evaluators, subject matter experts, and in-country technical leads was of immense value in helping to reach consensus on priorities for implementation for each technical area based on the country's assessed JEE scores. The benchmarks are a useful supplement to the existing IHR monitoring and evaluation framework; they are not an assessment tool but enabled easy reference to prioritize actions to attain the next capacity level of the JEE. In a few technical areas, where the implementation of activities had just commenced or were still ongoing since 2017, scoring did not change substantially because all the prescriptions per the benchmark tool have not been attained. These ongoing activities are noted as comments in the final report.[52] With the revised, more stringent tool, performing a midterm assessment may make it more challenging for countries to demonstrate progress after the initial JEE; the new scores might not reflect the amount of progress made in a single sector or technical area, but may reflect some progress that is more conservatively measured and robust. This challenge was limited to only a few indicators, however, and such nuances should be well communicated and documented. Using this approach, the scores were more reflective of actual progress and more objective.

The JEE, and its approach, is one of several efforts to spotlight the need for better preparedness and ensure compliance with IHR by countries signatory to it. Although the JEE was not developed to compare pandemic preparedness across countries, with the COVID-19 pandemic, there has been more focus on the JEE's utility to better understand response indicators such as time-to-first-case detection.[53] The JEE was designed for epidemic-prone diseases and not necessarily pandemics. A major distinction between both public health events is that epidemics are expected to be primarily managed by the health sector, whereas pandemics require a whole-of-society approach including cutting through bureaucratic red tape across different levels of society. It is also evident that the JEE does not measure critical indicators on pandemic preparedness, such as health system resilience.[54] However, recognizing this limitation has helped guide and support advocacy efforts for strengthening health security.[55] The JEE has highlighted that health security capacity may not necessarily translate to having the capability to respond to public health events. Such capabilities can only be observed during real-world events—a gap that after-action reviews[56,57] have been designed to fill in addition to other IHR monitoring and evaluation framework components. Regardless, the JEE remains critical to help structure national action planning for health security, providing a common language and framework for countries to work toward. In Nigeria, the midterm JEE was carried out because the country had made substantial progress in the 2 years since the first JEE. The midterm review was a useful process to assess this progress, define operational priorities, and integrate with country planning cycles, which was not explicitly part of the first JEE process. As such, countries can decide how and when similar assessments can be conducted, once or twice a year, as part of a continuous improvement process. The 2019 midterm JEE assessment in Nigeria highlighted where progress had been made since 2017 and enabled reprioritization of systems strengthening efforts. An important point to also consider going forward is that generalization of the findings from the initial JEE—by extrapolating preparedness scores to all administrative entities—could be misleading, especially in countries with autonomous subnational governing structures like in Nigeria.

Nigeria is governed through a federal system. The constitution provides for a substantial degree of autonomy across the 3 tiers of government—federal, state, and local. There are 36 states each with their own budget, priorities, and constitutional authority for health sector interventions. Participation by state epidemiologists and their feedback provided for a balanced view on potential IHR implementation challenges at the subnational level, from the states and local government areas with an acknowledgment of these limitations in the 1999 Constitution of the Federal Republic of Nigeria (as amended).[58] Within Nigeria's constitution, there are limitations around the development and review of legislation. Specifically, only the federal government can legislate and establish laws on items on the exclusive list, while federal and state governments can establish laws on items on the concurrent list. Where there is an existing federal law, however, states must defer to the former. Items on the residual list are solely the responsibility of state governments. Public health (eg, disease detection, surveillance, reporting, coordination) is on the residual list.[59] The power to establish legislation requires improved involvement of the subnational level on assessments such as the JEE and subsequent planning for investments in health security. With Nigeria's federated nature of governance, geographic and bureaucratic distance from the frontlines to the center of governance,[60] the role of the state epidemiologists, and contribution during the midterm JEE, as well as other personnel at the subnational level[45] are critical to strengthening health security. This was evident with scoring in the surveillance technical area, for instance, which required commensurate development of similar capacities at the subnational level. This type of limitation has spurred conversations on the importance of assessments of health security capacities at the subnational level. The need for this type of granular assessment is even more evident with the varied response to the COVID-19 pandemic in Nigeria and role of key personnel—especially at the subnational level. As such, subsequent JEEs or health security assessments need to capture the nuances on preparedness within these entities versus having a national representative scoring. This may be misleading, giving a false sense of preparedness. We recommend that countries endeavor to have representation from stakeholders at the subnational level during such discourse, to provide insights from their unique experience working at that level. Furthermore, countries can domesticate the JEE tool to better capture more qualitative and formative information that existing health information systems cannot unearth. This can help to identify specific areas that require additional efforts for strengthening, within the confines of the constitution or other existing laws.

Assessing health security capacities at the subnational level can be expensive and time-consuming. However, several granular assessments have been proposed using data that already exist in health information and disease surveillance systems. Examples include the timeliness milestones and metrics.[61] Standardized outbreak milestones enable countries to define and calculate relevant timeliness metrics, designed around milestones, to address their own needs. Milestones can include the dates when a country predicts, detects, verifies, and responds to an outbreak; notifies authorities; and launches a multisectoral investigation, conducts laboratory tests, implements control measures, and informs the public. A timeliness metric is then measured as the time interval between 2 relevant outbreak milestones—and importantly, can be calculated at the subnational level. WHO has used a combination of these and other indicators, such as the State Party Self-Assessment Annual Reporting tool indicators for preparedness capacity, and immunization indicators to capture real-world event data in its “triple billion” methods and make a distinction between measures of capacity and capability.[62] Other measures include a new global target of 7-1-7, whereby every suspected outbreak is identified within 7 days of emergence, reported to public health authorities with initiation of investigation and response efforts within 1 day, and effectively responded to—as defined by objective benchmarks—also within 7 days.[61] Each of these can be applied at both country and subnational levels to entrench a transparent framework for assessing outbreak response capabilities and identify gaps in health security capacities within the framework of health systems strengthening.

One of the objectives of the midterm JEE was to inform the review of the country's NAPHS. With the review, it was evident that ownership and accountability were weak despite the availability of performance management tools—a monitoring and evaluation framework and an implementation tracker. Other issues identified were: (1) the NAPHS monitoring and evaluation framework was not sufficiently robust to track output and outcomes and was laden with numerous activities that needed to be pruned[36]; (2) electronic tools (eg, NAPHS tracker) developed to track the status of implementation were difficult to implement especially where the legislation, coordination, communication, review, and provision of feedback across various IHR implementing MDAs was weak and suboptimal; (3) NAPHS and related plans need to be created for shorter periods (eg, annually or every 2 years); (4) subsequent planning needs to depend on the outcome of the evaluation of activities previously implemented[36]; and (5) strategic realignment of priority activities can help maximize already existing funding sources (eg, the Regional Disease Surveillance Systems Enhancement project) to support strengthening health security capacities. In addition, the NAPHS has too many, often ambitious, prioritized annual activities. These activities are not entirely included in the annual work plans of the MDAs and therefore often have no budget. Hence implementation was severely hampered. Of the 634 planned activities for fiscal year 2019, only 4% were completed by responsible MDAs. Implementation status of many other planned activities was either unknown, past due, or not updated. IHR participating or implementing MDAs also had limited understanding of their roles as it relates to IHR. This further compounded the implementation and follow-through of prioritized activities—even when funding was available.

Poorly executed NAPHS activities by countries may have also contributed to the weak response to COVID-19 across countries, as gaps identified after the 2017 JEE were not addressed. Coordination among relevant MDAs with a role in health security can help improve the implementation of activities that address health security gaps. In Nigeria, following the midterm JEE, the Nigeria Centre for Disease Control led and advocated for the establishment of an interministerial technical working group for IHR. This was, with equitable representation, from relevant MDAs to drive and sustain the implementation of health security activities, and champion advocacy efforts as required. A key requirement for membership in the technical working group was for members to be capable of making decisions or influencing decision makers. Although the Nigeria Centre for Disease Control is the national IHR Focal Point, within the federated structure in Nigeria, the various MDAs are independent and thus need to be mutually accountable with a governance model that enables this alignment. As such, members from the MDAs developed terms of reference for the national technical working group to describe their roles and responsibilities, including reporting processes. The NAPHS tracker was developed to support the reporting process across the MDAs and foster the desire of the technical working group to institutionalize its mutual accountability. Challenges with deployment of the NAPHS tracker have been discussed here and elsewhere.[36]

Conclusion

Although evidence suggests that the IHR monitoring and evaluation framework, and particularly the JEE indicators, do not provide a complete assessment of pandemic preparedness,[53] we found that the framework provides readily available tools that can be deployed to help identify gaps in preparedness. Assessments such as this are also equally important and needed by state parties to guide strategic development of operational plans (not just the NAPHS) for prioritizing the most immediate health security needs. Based on these assessments, state parties can develop holistic and realistic plans to further strengthen health security and build requisite core capacities as stipulated in IHR. Despite the challenges with the JEE and its representation of preparedness before the COVID-19 pandemic, it has helped to improve understanding of the status of pandemic preparedness across countries.[27] State parties need to consistently and intentionally invest in pandemic preparedness, and strengthening health security should be seen as a collaboration beyond the ministries of health alone.[63]

It is now more critical that preparedness activities are led by countries, based on the nuances and peculiarities of governance and governing structures. This will help foster sustainability and ownership of developed plans, which need to be backed by consistent funding and anchored on multisectoral engagement across various subnational entities and relevant MDAs—led by a national public health institute—for a safer world.

Acknowledgments

The authors would like to acknowledge the active participation of national experts in preparing for the midterm JEE mission. We also recognize the invaluable partnerships with governments including the governments of the United Kingdom and the United States for providing technical experts for this country-led review process; with other intergovernmental organizations, particularly FAO, OIE, West Africa Health Organization, Resolve to Save Lives, World Bank, Georgetown University, Public Health England, US Centers for Disease Control and Prevention, Pro-Health International, African Field Epidemiology Network, Volte Health, and University of Maryland, Baltimore for their contribution of expertise. We appreciate the financial support provided by Resolve to Save Lives and Public Health England for this activity and the WHO Nigeria Country Office for technical, planning, and financial support. The authors would like to acknowledge the continuing support and commitment of all of these to the implementation and principles of the IHR.