The global pool of simulation exercise materials in health emergency preparedness and response: a scoping review with a health system perspective.

Simulation Exercises (SimEx) are an established tool in defence and allied security sectors, applied extensively in health security initiatives under national or international legislative requirements, particularly the International Health Regulations (2005). There is, however, a paucity of information on SimEx application to test the functionality of health systems alongside emergency preparedness, response and recovery. Given the important implications health services resilience has for the protection and improvement of human life, this scoping review was undertaken to determine how the publicly available body of existing global SimEx materials considers health systems, together with health security functions in the event of disruptive emergencies. The global review identified 668 articles from literature and 73 products from institutional sources. Relevant screening identified 51 materials suitable to examine from a health system lens using the six health system building blocks as per the WHO Health System Framework. Eight materials were identified for further examination of their ability to test health system functionality from a resilience perspective. SimEx are an effective approach used extensively within health security and emergency response sectors but is not yet adequately used to test health system resilience. Currently available SimEx materials lack an integrated health system perspective and have a limited focus on the quality of services delivered within the context of response to a public health emergency. The materials do not focus on the ability of systems to effectively maintain core services during response. Without adjustment of the scope and focus, currently available SimEx materials do not have the capacity to test health systems to support the development of resilient health systems. Dedicated SimEx materials are urgently needed to fill this gap and harness their potential as an operational tool to contribute to improvements in health systems. They can act as effective global goods to allow testing of different functional aspects of health systems and service delivery alongside emergency preparedness and response. The work was conducted within the scope of the Tackling Deadly Diseases in Africa Programme, funded by the UK Department for International Development, which seeks to strengthen collaboration between the health system and health security clusters to promote health security and build resilient health systems.

Simulation Exercises (SimEx) are established, practised in defence and security sectors which are later recognised in national and international regulations (e.g International Health Regulations, 2005).

The importance of SimEx as an improvement tool has yet to be identified to test capacities for health system resilience alongside emergency preparedness.

Currently available global pool of SimEx materials lack an integrated health system perspective with a limited focus on the maintenance of routine quality health services delivered during response to a public health emergency.

Without adjustment of the scope and focus, currently available global pool of publicly available SimEx materials do not have the capacity to test health system resilience.

Participation of health system authorities or stakeholders in the development, conduct of SimEx and accountability of findings for improvement in health system strengthening is needed.

Dedicated SimEx materials are urgently needed to fill gaps identified in global resources to harness the potential of SimEx as an operational tool to contribute to improvements in health system strengthening.

Introduction

With 28 600 cases and 11 325 deaths, the West African outbreak of Ebola virus disease (EVD) from 2014 to 2016 focused global attention on the importance of resilient health systems by exposing their vulnerability to disease outbreaks.1 Health workers were significantly more likely to be infected than the general population, with this increased risk largely attributable to the poor quality of infection prevention and control (IPC) practices and emergency preparedness.1–3 Guinea, Liberia and Sierra Leone lost between 1.5% and 8.1% of their country’s doctors, nurses and midwives to EVD, translating into significant reductions in the healthcare provision.2 Ineffective surveillance systems enabled EVD to spread locally as well as across borders.4 5

During an outbreak, as with any public health emergency (PHE), the resilience of a country’s health system is tested in real time. The capacity to respond to an outbreak and maintain essential services creates a surge in demand for critical resources. Health system resilience has been defined as ‘the capacity of health actors, institutions, and populations to prepare for and effectively respond to crises; maintain core functions when a crisis hits; and, informed by lessons learned during the crisis, reorganize if conditions require it.’4 Resilient health systems protect human life and are linked with positive health outcomes during a PHE as well as in its aftermath.4 A lack of health system resilience is associated with excess morbidity and mortality due to the PHE as well as from other causes, which can be at least partially attributed to a reduction in access to quality health services.6–8 It has been estimated that a 50% reduction in access to healthcare services during the West African EVD resulted in increased child and maternal mortality as well as increased mortality from other infectious diseases.6 8 These indirect deaths—10 623—were not insignificant.6 Thus, it is critical that a mechanism exists to test and build resilient health systems in order to reduce excess morbidity and mortality from future PHEs.

National and international preparedness for PHEs has long been discussed on a global scale, especially in health security forums that seek to address challenges and gaps in meeting core International Health Regulations (IHR) (2005) capacities. One component of the revised IHR (2005) monitoring and evaluation framework includes Simulation Exercises (SimEx), which are defined by the WHO as ‘forms of practice, training, monitoring or evaluation of capabilities involving the description or simulation of an emergency, to which a described or simulated response is made.’9 SimEx have historically been an established tool in defence and allied security sectors, with recent adaptation to health security efforts in the context of disruptive emergencies, natural and man-made. They are now being used as a tool to assess compliance with national—for example, the UK Civil Contingencies Act 2004—and international legislative requirements, notably IHR 2005.10 There is, however, a paucity of information on the application of these exercises to test the functionality of health systems alongside emergency preparedness, response and recovery.

SimEx are also an ideal opportunity to test preparedness of the various functions of the health system, particularly health service delivery, in response to PHEs. This would contribute to the development of strong linkages between health systems and health security sectors supporting an integrated approach towards building resilient health systems. Given the important implications health services resilience has for the protection and improvement of human life, this scoping review was undertaken to determine how the body of existing global SimEx materials considers the testing of health system resilience, together with health security preparedness, and response functions in the event of acute and/or protracted PHEs.

Methods

Recognising the dispersion of SimEx materials, a scoping review of academic literature as well as an institutional search was undertaken.11 A wide search strategy was employed to identify all relevant materials. Searches of different hazard types, exercise types and subject areas were combined (online supplementary appendix 1). The initial search returned 668 articles for screening with an additional 17 identified through bibliography review. Titles and abstracts were reviewed against inclusion and exclusion criteria (table 1), reducing the number for full-text review to 32.

Table 1

Inclusion and exclusion criteria

Inclusion criteria	Exclusion criteria
English language Published from 2007 onwards Test aspects of preparedness, response or recovery to PHE (including business continuity)	No public health relevance Insufficient exercise detail Part of wider curriculum or course Materials supporting development of exercises

PHE, public health emergency.

For the institutional search, institutions involved in the development, implementation or evaluation of SimEx were identified by experts in the field, as well as through relevant articles. Their websites were searched for applicable materials. Ultimately, 31 institutions, ranging from academic to national response agencies, were reviewed (online supplementary appendix 2), identifying 73 materials which were reduced to 19 following the screening of aims and objectives (figure 1).

Figure 1

Flow chart of literature and institutional search and screening. n/a, not applicable; PH, public health; SimEx, Simulation Exercise.

The final materials identified (n=51; online supplementary appendix 3) (32 emerging from the literature and 19 emerging from institutional review) were analysed by two independent reviewers for their scope to test aspects of health systems using the WHO Health System Framework, consisting of six independent but inter-related building blocks (table 2). Resilience was considered to be addressed if materials demonstrated evidence of an integrated emergency response, maintenance of essential functions or reorganisation of services within the context of a PHE. A smaller number of materials (n=8) were identified for examination of their scope to test resilience from a functional perspective (figure 1 and box 1). Materials were excluded from this portion of the analysis if they did not report either an exercise or objectives in sufficient detail to support this analysis. This portion of the analysis (n=8) involved five journal articles and three guidance materials (online supplementary appendix 4). A library of publicly available materials to support the development of SimEx was compiled in parallel to the review process.

Table 2

WHO health system building blocks—components and resilience attributes

Health system building blocks	Example components	Resilience attributes
Service delivery	High-quality case managementStandard operating procedures (SOP)Facilities—fit for purposeEquipment, transport	System aspects of health service deliveryImpact of public health emergencies on routine servicesIntegrated functional capacitiesQuality of care
Workforce	Number of staffTraining/knowledgeSurge capacity	Quality of surge workforceCapacity remaining for routine servicesHealth and safety of workforce
Medicines and technology	Access to medicines and supplies	Emergency procurement systems and plansFunctional capacity/timeliness of deliverySupply chain considerations
Leadership and governance	National/local plans, structuresRoles and responsibilitiesGuidelinesRisk identification	Governance and coordination structuresWell-defined command and control structuresAlignment with national plans/guidanceLeadership at facility level
Financing	Identification of contingency resourcesAccess to contingency resources	Mechanisms to access contingency fundsAbility to meet multiple public health emergencies (alongside routine services)
Information systems	Surveillance systemsInformation sharing	Integration of surveillanceActivation of plansRisk communicationIncident command system

Does the material look at aspects of health service delivery in the event of a simulated event?

Does the material test a function of specific building blocks of a health system in consideration of other related building blocks?

Does the material test preparedness and response of healthcare facilities alongside national incident management system?

Does the material look at impact of emergencies on provision of routine essential health services?

Does the material look at how PHEs impact health service delivery at subnational and district-level facilities?

Does the material examine the standard of care during PHE?

Patient and public involvement

There was no involvement of patients or the public in this study, in either the design of the methods or in the conduct of the study.

Results

SimEx consideration of health system building blocks

The majority (63%) of the materials (n=51) identified exclusively tested preparedness and response to a PHE, without considering them as typical functions of a resilient health system (figure 2). A limited number were identified that tested aspects of health system resilience or recovery (figure 2). The categorisation of the scope of material was based on the research team’s interpretation, that is, if the exercises dealt with preparing and responding to a PHE or if they dealt with aspects of recovery or response. A few materials self-identified as testing health security or testing for contingency, and so warranted extra categories. The materials identified looked at aspects of health security across different geographical levels (international, national, regional and facility); they more frequently examined the regional/district (43%) and facility levels (31.4%) (figure 3). The most frequent hazard type (37%) simulated was biological which included pandemic influenza and other emerging infections (figure 4).

Figure 2

Simulation Exercise (SimEx) materials by scope (n=51).

Figure 3

Simulation Exercise (SimEx) material by geographic or administrative level (n=51).

Figure 4

Simulation Exercise (SimEx) materials by types of hazards (n=51). CBRNe, Chemical, Biological, Radiological, Nuclear and Explosive.

Though often not the explicit focus, all reviewed materials (n=51) tested at least one building block or an aspect of a health system building block (figure 5). A functional approach was apparent, though functions were generally tested in a narrow way both within and across relevant building blocks, rather than integrated across all relevant health system building blocks. IPC was the most commonly tested service delivery function with adherence to case management guidelines and standard operating procedures (SOP) being a frequent focus of testing.8 The quality of health services provided in the context of the response or measures of the maintenance of essential health services were lacking. Figure 5 and table 3 highlight examples of building block functions and the specific functions identified as tested in the material.

Figure 5

Simulation Exercise (SimEx) focusing on health system building blocks (n=51) (some materials cover more than one building block).

Table 3

Examples of health system functions identified in SimEx material

Service delivery	Health workforce	Medicines and technologies
+ IPC	+ Surge capacity	+ Access to medicines and technologies
+ Case management	− Training/knowledge	− Access mechanisms
+ Standard operating procedures (SOP)		− Supply chain
− Quality of care
− System integration
− Core functions
Financing	Leadership a nd g overnance	Health i nformation s ystems
+ Financing	+ Governance	+ Communication
+ Support material	+ Plans, SOPs, roles	− Surveillance systems
− Access mechanisms	− Alignment between health systems and health security	− Risk communication

Symbol + indicates presence of function in the set of SimEx materials.

Symbol − indicates absence of function in the set of SimEx materials.

IPC, infection prevention and control; SimEx, Simulation Exercise.

Surge capacity was the most frequently tested function of health workforce, though a narrow approach was often apparent. Exercises tested aspects of surge capacity such as staffing, however, this was not addressed within other building blocks in relation to the invariable increased demands on services and supplies. A narrow approach to health information systems was also apparent, with three materials (out of 51) focusing on surveillance systems and only limited aspects of risk communication tested. Only one identified material tested the triggering of a response plan using a surveillance system.12 Leadership and governance (from a security perspective) was the most frequently addressed building block, with many materials testing aspects of this building block such as response plans and roles and responsibilities of responders. However, alignment with national structures and guidelines was not apparent, nor was consideration given to decision in risk identification. While access to medicines and supplies was frequently alluded to, testing of access to mechanisms or supply chain resilience was infrequent. Financing was alluded to in only 17% of materials (n=51), but testing of mechanisms to access funding in the event of PHEs was not identified.

Tabletop exercise/discussion was the most frequent approach identified (figure 6). They are less expensive and faster to execute, particularly when considering large groups of stakeholders. The limitation with this approach is limitation in scope to adequately simulate PHEs and test the individual and integrative aspects of health systems.

Figure 6

Approaches in the conduct of Simulation Exercise (SimEx) (n=51). TTX, tabletop exercise.

SimEx consideration of health system functionality and system underpinning

Eight materials were identified for further analyses on the functionality/system aspect of the six building blocks, using a set of questions (box 1). Most materials tested activation of appropriate emergency response mechanisms and structures within the respective administrative levels tested, for example, a facility-level exercise testing activation of all appropriate response mechanisms within the health facility. Some materials tested activation across different levels within the system, for example, a health facility responding to an outbreak activating regional or national response systems. Emergency response systems were generally assumed to have been activated, without testing system triggering. As was apparent with the building block analysis, healthcare functions were generally not tested in an integrated way. For example, a mass dispensing drill had no regard for the parallel response structures with which they would have to integrate during a response.13 Similarly, a facility response to a sarin attack failed to integrate with national response agencies.13 14

Reporting of alignment with national command and control structures or response plans was limited as was evidence of consideration of the impact of the PHE on other health system-level facilities or management structures. Materials to test the impact of emergencies on primary healthcare (PHC) or its response were extremely limited and community resilience materials, where identified, failed to link with health systems.15–17

The materials were limited in their focus on testing the quality of services delivered in the context of response to a PHE. Where present, measures tended to focus on clinical aspects of care rather than system and process measures. Measures to test the maintenance of essential services from a quality perspective were not identified in any of the eight selected materials.

Limitations of the review

The application of the health system framework to review SimEx materials introduced a degree of subjectivity. In reality, it is the same health system that provides routine healthcare, emergency-specific healthcare and response to a shock impacting public health. This was addressed through discussion as well as with input from health system and security experts in WHO. Similarly, the approach to the analyses conducted required sufficient detail in the identified materials in relation to exercises or objectives, which led to the exclusion of a number of relevant materials as they were not present either with sufficient details or as a package (written narrative, scenario, injects and postexercise report). Both the academic literature and institutional materials were analysed using the same approach. This likely led to an underestimation of effect in relation to academic literature as not all that was tested may have been reported, and an overestimation in relation to institutional materials as objectives may not have been applied effectively within exercises.

A publication bias also likely exists in relation to health security exercise reports and materials given the sensitive nature of findings as well as in relation to materials developed and delivered by private companies. While multiple institutions and organisations (n=31) known to be involved in SimEx were contacted to identify unpublished materials, it is likely that there remains a pool of materials not made available as they were considered proprietary and/or sensitive. However, the objectives of the exercise are based on publicly available materials that are accessible and can be used as global resources, as such access-restricted materials fall beyond the scope of this review.

In general, materials identified were from countries with developed capacities for emergency preparedness and response—materials from other settings may not have percolated the literature and may not have an institutional home where materials are placed. However, the inclusion of institutions known to support SimEx within low income settings is likely to reduce the impact of the bias towards high-income countries within the findings of the work. The exclusion of non-English materials has the potential to introduce a cultural bias, although materials in the review included materials from most continents (North America, Europe, Africa, Asia and Australia). The extent of any cultural bias is therefore limited, and unlikely to affect the findings of the work.

Despite these limitations, there is no reason to suspect that materials not included in this review differ systematically from those included in such a way as to negate the findings.

Discussion

While all materials identified tested aspects of health systems, there was limited evidence of an integrated health system approach. Health system building blocks were touched on from a preparedness perspective, and often tested in a fragmented and isolated way without addressing interlinkages.

IPC was frequently tested in the context of response to a pandemic or other emerging infection, with a focus on governance structures and adherence to case management and SOPs. IPC is central to the response to any emerging infection as was highlighted within the EVD outbreak in West Africa where poor IPC practices contributed to significant health worker transmission, leading to reductions in response as well as essential healthcare delivery.2 In a similar instance, a lack of strict adherence to IPC guidelines was associated with ongoing healthcare facility transmission during the severe acute respiratory syndrome outbreak in both Toronto and Taiwan in 2003.18 An integrated approach to IPC as a critical function of quality health service delivery in these SimEx materials is thus required.

The availability and timeliness of emergency financing is critical in determining the timeliness and effectiveness of coordinated efforts in any emergency response. Despite this, limited consideration was given to the supply of resources required to meet the surge in demand, including staff, diagnostics, medicines or personal protective equipment, and to accessing the financing necessary to meet these demands.19 It is therefore of utmost importance that the rapid mobilisation of financing should be regularly tested in SimEx, taking into account the response required as well as consideration of maintaining quality essential health services.

Health information systems, though often alluded to, were rarely tested in a robust and integrated way, with a lack of focus on surveillance systems in particular. The only material identified that tested activation of an emergency plan using syndromic surveillance failed to trigger activation of the response plan.12 The pivotal role played by data and information systems in routine healthcare delivery is even greater in the context of emergencies. A strong and reliable health information system ensures understanding of the epidemiology of disease, and is critical in coordination, communication and management of response efforts. Testing this functionality should be an integral part of SimEx conducted at any level.

Consideration of the integration of preparedness and response across health system levels was lacking, with services and facilities tending to test their own response capabilities in isolation. PHC, despite being the likely first point of care in many PHEs, was rarely considered in the SimEx materials reviewed either in terms of supporting preparedness capacity or in terms of the effect of the PHE on PHC.15 PHC plays a central role in surveillance which was highlighted within the EVD outbreak in West Africa, where failure to identify EVD when it first presented within the community in Guinea led to a significant delay in response with a concomitant lack of containment. Despite evidence from EVD West Africa that community linkages can support health system response in the face of PHEs, the current global pool of SimEx materials were unlikely to link with community aspects and where community preparedness materials existed they rarely linked with health systems.16 17

The importance of integrating private healthcare facilities into emergency preparedness and response capabilities was highlighted by the recent dengue outbreak in Khyber Pakhtunkhwa in 2017. A private health facility, unfamiliar with emergency response protocols, failed to activate emergency response mechanisms in a timely manner. This, alongside a lack of case management guidelines, contributed to prolonged community transmission.20 No evidence of integration of private healthcare facilities into health system preparedness and response was identified within the current pool of global SimEx materials.

While there is international consensus of the need to focus on quality in healthcare, the focus on the quality of services delivered in the context of a PHE was lacking.21 22 Where materials did focus on service quality, they tended to focus on clinical aspects of care. System and process measures were only identified in drills that tested mass dispensing capabilities in the context of a pandemic or biological attack. These capabilities represented parallel response structures and their linkages with appropriate health system structures were not tested, nor was the communication required to divert individuals from normal health services tested.23 24

No materials identified included measures to test the maintenance of or quality of essential health services during the response to a PHE. Evidence from West Africa shows that the indirect mortality and morbidity associated with discontinuity of health services as well as poor quality health services was significant across the three countries, with the biggest impact on maternal and child health.6 The current global pool of SimEx materials are limited in its ability to test health service resilience alongside preparedness and response. This in turn limits the opportunity to practically bridge health security and health systems at different administrative levels. Special attention is required in using SimEx approaches to drive sustainability of investment in health security preparedness or disease-specific programmes to proactively position available scarce resources into sector-wide development of health systems for all public health hazards per IHR (2005).

Conclusion

SimExercises are a well-practised method of testing and promoting emergency preparedness and response for local, national and global health threats. While much experience exists globally in this area, the proprietary nature of some of the materials creates a missed opportunity for the sharing of knowledge across global health security and preparedness communities. It would be in the interest of the global community to develop a mechanism to support sharing of lessons learnt that respects the integrity of private organisations involved in SimEx development and delivery.

The effectiveness of SimEx has been demonstrated in identifying gaps in emergency response plans, skills and associated resources.11 25 The lack of an integrated health system perspective in the current global pool of SimEx materials limits their ability to support health system functionality and strengthening in the context of PHEs. This, along with their lack of focus on the quality of response and the maintenance of quality essential health service functions, means that they do not have the capacity to support health system resilience. The incorporation of a health system perspective into SimEx materials has the potential to enhance health system strengthening and the development of resilience alongside emergency response and health security capabilities.

An integrated approach to SimEx including health security, emergency preparedness and health systems is required to address the gap identified by this review. As a result of these findings, an off-the-shelf SimEx package that addresses health system aspects within the context of response to a PHE is being developed, which will be freely available for all countries, particularly low-income countries. This could be further supported through the cross involvement in ongoing exercises and after-action reviews to enhance connectivity and support the development of shared ownership of improvement recommendations. These new-generation SimEx materials could be collated in a global repository that could be accessed by national authorities. Such a collaborative approach would allow the leveraging of the considerable expertise in SimEx present within health security and emergency preparedness sectors. Lessons learnt from this integrated approach will allow health systems to be built better, function better, which will ultimately lead to the protection and improvement of human life.