Health system resilience: a literature review of empirical research.

The concept of health system resilience has gained popularity in the global health discourse, featuring in UN policies, academic articles and conferences. While substantial effort has gone into the conceptualization of health system resilience, there has been no review of how the concept has been operationalized in empirical studies. We conducted an empirical review in three databases using systematic methods. Findings were synthesized using descriptive quantitative analysis and by mapping aims, findings, underlying concepts and measurement approaches according to the resilience definition by Blanchet et al. We identified 71 empirical studies on health system resilience from 2008 to 2019, with an increase in literature in recent years (62% of studies published since 2017). Most studies addressed a specific crisis or challenge (82%), most notably infectious disease outbreaks (20%), natural disasters (15%) and climate change (11%). A large proportion of studies focused on service delivery (48%), while other health system building blocks were side-lined. The studies differed in terms of their disciplinary tradition and conceptual background, which was reflected in the variety of concepts and measurement approaches used. Despite extensive theoretical work on the domains which constitute health system resilience, we found that most of the empirical literature only addressed particular aspects related to absorptive and adaptive capacities, with legitimacy of institutions and transformative resilience seldom addressed. Qualitative and mixed methods research captured a broader range of resilience domains than quantitative research. The review shows that the way in which resilience is currently applied in the empirical literature does not match its theoretical foundations. In order to do justice to the complexities of the resilience concept, knowledge from both quantitative and qualitative research traditions should be integrated in a comprehensive assessment framework. Only then will the theoretical 'resilience idea' be able to prove its usefulness for the research community.

Key Messages

The way in which resilience is currently applied in the empirical literature does not match its theoretical foundations.

In order to do justice to the complexities of the resilience concept, knowledge from both quantitative and qualitative research traditions should be integrated in a comprehensive assessment framework.

Introduction

The word ‘resilience’ origins from the Latin prefix ‘re-’ (back) and the verb ‘salire’ (to jump, leap). In science, it has long been used by engineering and material science to describe the ability of a material to absorb energy without losing its original form or characteristics (Hollnagel 2009). Over time, different disciplines adopted and adapted the term, adding different interpretations and facets to it: In ecology, resilience describes the persistence of ecological systems and measures a system’s ability to absorb changes of variables and maintain relationships between different populations (Holling 1973). In psychology, resilience is understood as the individual human capability to cope with crises, losses or hardships without negative consequences (Tugade and Fredrickson 2004).

In the last decade, the concept of resilience has also gained popularity in global public health. This development is reflected by major UN frameworks adopted in the last decade: The 2005–15 Hyogo Framework for Action (UNISDR, 2005) was subtitled ‘Building the Resilience of Nations and Communities to Disasters’. Its successor, the 2015–30 Sendai Framework for Disaster Risk Reduction (UNISDR, 2015), increases the focus on health in the disaster preparedness discourse and correspondingly calls for health resilience. Various sustainable development goals point to resilience as a target (1.5: ‘resilience of the poor’, 2.4 ‘resilient agricultural practices’, 11b ‘resilience to disasters’; Bahadur ; UNISDR 2015) In a 2016 editorial of Bulletin of the World Health Organisation (WHO), health system resilience is named as a critical concept for global health, in the same vein as health system strengthening, universal health coverage and health security (Kutzin and Sparkes 2016).

The shifting conceptualization of health system resilience

While definitions and concepts of health systems resilience differ substantially throughout the literature, all have a common core: they regard resilience as the degree of change a system can undergo while maintaining its functionality. The concept of resilience was introduced to the health systems literature from the ecological sciences through an increased understanding of health systems as complex adaptive systems (Blanchet and James 2013). In this context, the idea of resilience, defined as ‘a measure of the amount of change a system can experience while maintaining the same controls on structure and function’ (Blanchet and James 2013), can act as a useful tool to help us understand health system dynamics. The ecological idea that strategies to enhance resilience can be absorptive, adaptive or transformative depending on the impact and intensity of the crisis has been particularly impactful in the health system resilience discourse.

Popularized further during the Ebola crisis, health system resilience underwent a conceptual shift; from a mere ‘system’ capacity to recognizing the contribution of individuals and their agency within that system and acknowledging the wider social, economic and political context in which responses occur. Critics argued that the application of the resilience concept—as a ‘top-down’ approach—obscured important factors which prevented an adequate response to the Ebola crisis. They emphasized instead the importance of ‘understanding and reducing local power disparities, building the trustworthiness of health actors […] both between and during crisis’ which improves the ‘everyday functioning of the health system’ (Martineau 2016). In response to these criticisms, Barasa proposed the idea of ‘everyday resilience’, emphasizing in particular the importance of the capacities and resources available to individuals faced with delivering health services every day. Everyday resilience may especially be of relevance, they argued, in low- and middle-income countries where managers may ‘routinely face structural and policy instability, such as changes in governance structures, payment delays, abrupt and imposed policy directives […], unstable authority delegations, unpredictable staff and […] changing patient and community expectations’ (Barasa ).

Similarly, Blanchet proposed a new model of understanding health systems resilience which focuses not just on the outcome of the resilience process (i.e. absorptive, adaptive and transformative capacities), but also on the underlying management capacities of the system and its actors to response to change: knowledge, uncertainties, interdependence and legitimacy (Box 1). These operational dimensions are interlinked with each other and together characterize the management of resilience in health systems (Figure 1). While these two more recent conceptualizations of resilience can be understood as different in terms of taking a ‘top-down’ (Blanchet ) and ‘bottom-up’ (Barasa ) approach, they both acknowledge the importance of the context in which the resilience process takes place and the agency of actors involved, and thus represent two sides of the same coin.

Figure 1

Conceptual overview of health system resilience, adapted from Blanchet .

Box 1 Resilience domains used in conceptual analysis of studies, as defined by Blanchet

Management capacities:

Knowledge—‘Capacity to collect, integrate and analyse different forms of knowledge and information’

Uncertainties—‘Ability to anticipate and cope with uncertainties and surprises’

Interdependence—‘Capacity to manage interdependence: to engage effectively with and handle multiple- and cross-scale dynamics’

Legitimacy—‘Capacity to build or develop legitimate institutions that are socially accepted and contextually adapted’

Three levels of resilience:

Absorptive capacity—‘capacity of a health system to continue to deliver the same level (quantity, quality and equity) of basic healthcare services and protection to populations despite the shock using the same level of resources and capacities’

Adaptive capacity—‘capacity of the health system actors to deliver the same level of healthcare services with fewer and/ or different resources, which requires making organisational adaptations’

Transformative capacity—‘the ability of health system actors to transform the functions and structure of the health system to respond to a changing environment’

Conceptual influences from other fields

In addition to the conceptualization of resilience outlined above, other disciplinary fields have influenced the discourse on health system resilience, most notably the disaster management and healthcare quality literature.

In the disaster management sciences, resilience discussions were initially focused on the maintenance of infrastructure, functionality of health care facilities and continued service delivery (Crowe ; Balbus ; Cimellaro ) operationalizing resilience as ‘capability of a health system to mitigate the impact of major external disruptions on its ability to meet the needs of the population during the disaster’ (Crowe ). However, experiences of Hurricane Catrina in the USA shifted the dominant discourse in the disaster management literature to the concept of community resilience (Wulff ; Olu 2017). Community resilience proposes that the key to a good disaster response lies in communities, and their ability to ‘prepare, respond, and recover’ from major events through a range of measures including increased social connectedness, adaptive health and social systems and emergency preparedness planning (Wulff ).

A further prominent influence on the health system resilience discussions has been the concept of ‘resilience engineering’ or ‘health care resilience’, emerging from the healthcare quality literature. This approach, developed as a critique to traditional views of healthcare safety as an ‘absence of failures’, defines safety as the ‘ability to succeed under varying conditions’ (Hollnagel ). It thus focuses on nurturing the everyday functioning of healthcare teams and facilities to strengthen resilience and reduce clinical mistakes. A recent review on the topic has found that this approach has garnered significant attention in both the primary and secondary literature since its emergence around 2012 (Ellis ).

The need for a review of the empirical literature

Existing literature reviews have been conducted on the theoretical conceptualization of health system resilience (Turenne ) and the factors contributing to resilient health systems (Barasa ). The concept of resilience has also been extensively discussed outside the health sector (Tanner ). While grasping the theoretical background of the concept is certainly crucial, understanding how theory is translated into evidence is equally important for assessing the usefulness of the ‘resilience idea’ for the research community. However, so far there has been no critical appraisal of how the concept of health system resilience has been operationalized and applied in the empirical literature.

We thus conducted an empirical review of health system literature in order to better understand how the resilience concept has been operationalized in empirical studies. Within this research aim, we address three specific sub-questions: (1) What are the key aspects (methodological approach, geographic focus, health system building block addressed and crisis/challenge discussed) of research on health system resilience and how have these changed over time? (2) What concepts and frameworks on health system resilience have been used to operationalize resilience in the health systems literature? (3) What is the scope of empirical research on health system resilience within current definitions of the concept? We thus provide an overview of the existing empirical literature on health systems research which can be used to further develop the concept and inform its operationalization in future studies.

Methodology

We conducted a review of empirical literature, following systematic review methodology in line with the understanding brought forward by Moher . This included a systematic literature search, and a rigorous and systematic data screening and extraction process (Peters ).

Searches were conducted in Medline, Social Science Citation Index and CINAHL (Cumulative Index to Nursing and Allied Health Literature) using Resilien* AND a health system related terms (see Box 2).

Box 2 Search terms

Search terms:

((((((((secondary health care [mh]) OR primary health care [mh]) OR health services [mh]) OR delivery of health care [mh]) OR health services research [mh])) OR ((((((((((((““health system””) OR ““health systems””) OR ““health care system””) OR ““health care systems””) OR ““health care””) OR ““health care sector””) OR ““health care sectors””) OR ““health service””) OR ““health services””) OR ““service delivery””) OR ““health care service””) OR ““health care services””))) AND Resilien*

The searches were conducted on 18 October 2019 and were limited to articles published since 2008 in English or German language to keep the extent of the review feasible. The search produced 6136 publications for screening after the removal of 794 duplicates [see Figure 2 for the PRISMA flow diagram in line with Moher ].

Figure 2

Prisma flow diagram.

Due to the high number of items, we used a three-stage screening process, eliminating non-relevant articles at the stage of title-, abstract- and full text-screening. Items were excluded if they did not report primary data, or were concerned with individual/psychological resilience including resilience of healthcare providers (e.g. nurses, physicians), resilience in the non-health space (e.g. social resilience, resilience of urban environments and resilience of biological systems), community resilience without link to health systems or articles that were concerned neither with health systems nor resilience. We also excluded articles which were concerned with health system resilience, but only used the term as a ‘buzzword’, without further definition, discussion or operationalization of the concept. As the research objective was to understand the application and use of resilience in health system research, items with any research design, geographic scope and health system focus were included.

After the abstract-screening stage, 517 references remained, with another 444 references excluded after screening full texts (see Figure 2). Both abstract- and full text-screening were carried out by the first and second author with joint synthesis until consensus was reached. Two further articles were from the reference lists of the literature reviews by Barasa and Turenne met the inclusion criteria for the present study and were included in the review. The remaining articles were divided into two categories: (1) those papers which specifically assessed health system resilience by including this as a specific research objective or applying a framework allowing for the operationalization of health system resilience (‘key papers’) and (2) articles reporting research which led to a discussion of health system resilience or how to achieve health system resilience.

Data extraction was carried out by the first and second author using Microsoft Excel. To answer the first research question on key aspects of the empirical health systems literature, data on type of research (primary/secondary research), discipline of the first author, the health system building block studied [according to World Health Organization (2010)], the type of crisis or conflict studied, study location (country, continent, low-/middle-/high-income country), the organizational level being studied (e.g. global, national or regional) and type of data used were extracted from all identified studies.

To answer the second and third research objectives, only those studies directly measuring or assessing health system resilience (‘key papers’) were analysed. In order to evaluate the use of existing empirical frameworks in the empirical literature (second objective), information on frameworks used was extracted if these guided either the data collection or analysis process, or both. To further extract the scope of empirical research in terms of aspects or elements of the concept being addressed (third objective), we were guided by the conceptual framework of Blanchet . We used this framework because it captures the various ways in which resilience is used in the empirical literature: it describes both the management capacities essential for a resilient system (management capacities: knowledge, uncertainties, interdependence, legitimacy) as well as those describing the outcome (three levels of resilience: absorptive, adaptive and transformative capacities). It thus is able to capture a broad range of research on post ex ante and ex post (Béné ) aspects of the resilience process. Research articles were classified within this framework using the definitions listed in Box 2.

We synthesized the findings by combining a narrative synthesis with descriptive quantitative analysis of key aspects addressed. We further tabulated and mapped aims, findings, underlying concepts and measurement approaches according to the resilience definition by Blanchet . Indicators used to measure aspects of resilience in quantitative and mixed methods studies were also extracted and mapped according to their respective resilience domain and the level of data collection (national, organizational, staff or population/patient level).

Results

A total of 71 articles met our inclusion criteria, comprising 40 research papers specifically measuring or addressing health system resilience and 31 discussing health system resilience using empirical research (Figure 2, see Supplementary file for full list of studies).

Quantitative synthesis and mapping of empirical literature in health system resilience

The literature was found to be fairly evenly distributed across continents: Africa (n = 18; 25%), Europe (n = 18; 25%), Asia (n = 15; 21%), North America (n = 15; 21%) and Australia (n = 2; 3%), with four studies reporting data across continents. The exception was South America, where no empirical papers were found. The majority of research was conducted in high-income countries (n = 37; 52%), with 18 studies (25%) in middle-income countries and 13 studies (18%) in low-income countries. We found an increase in literature in recent years (62% of studies published since 2017).

The majority of research (n = 58; 82%) addressed a specific crisis or challenge. Overall, infectious disease outbreaks was the most frequently addressed challenge (n = 14; 20%), followed by natural disasters (n = 11; 15%) and climate change (n = 8; 11%). Other challenges were conflicts (n = 4; 6%), migration (n = 4; 6%), financial crises (n = 2; 3%) and terrorist attacks (n = 1; 1%). Several articles addressed chronic, non-crisis-related challenges (n = 12; 17%): changes in team composition (n = 1; 1%), human error (n = 5; 7%), everyday resilience (n = 3; 4%) and structural change (n = 2; 3%). While non-crisis-related challenges and climate-related studies dominated the early records from 2008 to 2014, over time, the diversity of addressed challenges has grown embracing financial crises from 2013, infectious disease outbreaks from 2015 triggered by the Ebola epidemic and migration from 2017 (see Figure 3).

Figure 3.

Identified literature on health system resilience (N = 71) organized by type of challenge and year (2008–19).

In terms of health system building blocks addressed, a large proportion of studies (n = 34; 48%) focused on service delivery, while 14 (20%) did not focus on a particular health system building block but took a general perspective. Other building blocks addressed frequently include leadership and governance (n = 9; 13%) and health workforce (n = 8; 11%), while health information systems (n = 4; 6%), medicines and access to medicines (n = 2; 3%) and health system financing (n = 1; 1%) are addressed less frequently.

Overall, the empirical studies identified differed in terms of their disciplinary tradition or conceptual background. Studies from the public health sciences tended to converge in three groups: (1) quantitative studies focusing on service delivery, making use of service utilization indicators provide an easily accessible measure to assess resilience before, during and after a crisis (Paterson ; Gizelis ; Sochas ; Kozuki ; Ray-Bennett ), (2) qualitative studies focusing on the health workforce, influenced by ideas of ‘everyday resilience’ and addressing the contributions of social connectedness and leadership on health system resilience (Mash ; Witter ; Raven ; Brooke-Sumner ; Thude ), and (3) studies taking a broad perspective of health system resilience, looking at multiple health system building blocks or aspects of a health system to assess resiliency (Ager ; Ammar ; Fukuma ; Ling ; Meyer ; Watts ).

However, influences from outside the public health sciences could also be identified in the empirical health system resilience literature. As a relatively distinct influence, the disciplines of engineering and architecture have contributed empirical research assessing the infrastructure and thermal resilience of healthcare facilities and structures (Lomas ; Iddon ; Short ; Dippenaar and Bezuidenhout 2019). A further relatively distinct influence has been the contribution of specific checklists to assess facility and organizational resilience from the fields of disaster management and emergency preparedness (Paterson et al. 2014; Zhong , 2015; Dobalian ; Khan ; Meyer ). Also from the field of disaster management, but perhaps more intertwined with resilience in the way it has been conceptualized in the health systems literature, are studies assessing community resilience and its relationship with service delivery during a crisis (O’Sullivan ; Andrew ; Toner ; Alonge ; Cohen ). Finally, hailing from the tradition of medical sciences concerned with patient safety and quality of care, concepts of ‘health care resilience’ or ‘resilience engineering’ have also influenced the empirical literature on health system resilience (Brattheim ; Franklin ; Falegnami ; Merandi ; Patriarca ). While study object of these studies is also the health workforce, the focus is placed on the analysis of work processes and the avoidance of medical errors to maintain functionality of services.

Methodological analysis of key empirical papers

We identified 40 high-relevance empirical studies specifically assessing health system resilience. Fifteen articles used a quantitative methodology (Table 1), nine articles applied mixed methods (Table 2) and a further 16 used qualitative methods (Table 3). Given the distinction between articles in terms of their thematic focus described above, we present articles in six thematic areas: assessing national-level health system resilience in the context of a specific crisis (n = 8; 20%), assessing health service delivery (n = 10; 25%), addressing health workforce issues (n = 7; 18%), taking a community resilience perspective (n = 3; 7%), looking at infrastructure and thermal resilience (n = 3; 7%) and developing emergency preparedness checklists and assessment tools (n = 9; 23%).

Table 1

Overview of aims, methods, concepts used and dimensions of resilience addressed by quantitative research papers (n = 15)

Author	Year	Country	Research objective	Key findings	Study design and data source	Operationalization of resilience	Conceptual framework used	Management capacity dimensions	Resilience outcome dimensions
Assessing national-level health system resilience in context of a specific crisis
Fukuma et al.	2017	Japan	To assess population-level health indicator changes following the Great East Japan Earthquake and discuss redesign to enhance health systems' responsiveness and resilience	Age-adjusted all-cause mortality and some disease specific mortality rates rose more in affected prefectures than the average national rates. Disaster revealed strengths and deficiencies in responsiveness and resilience of health system	Case study using routine data sources	Population indicators, Health system indicators, Health outcome indicators	N/A	N/A	Absorptive
Watts et al.	2018	Global	To track a series of indicators of progress, publishing an annual ‘health check’, from now until 2030, on the state of the climate, progress made in meeting global commitments under the Paris Agreement, and adapting and mitigating to climate change	The public, and the health systems they depend upon, are ill-prepared to manage the health impacts of climate change	National survey completed by 101 national ministries of health	Six Indicators developed as part of the Lancet Countdown initiative	Several UN frameworks used	Knowledge Uncertainties	Adaptive
Assessing health service delivery in context of a specific crisis
Radcliff et al.	2018	USA	To present ‘measures of ambulatory care recovery and resilience that rely on routine appointment records’	‘Most clinics in affected areas achieved appointment completion percentages that matched or exceeded prestorm levels within 2 weeks of the storm’.	Administative data from Veterans Affairs clinics	Percentage of completed appointments before and after a disaster	N/A	N/A	Absorptive
Simonetti et al.	2018	USA	‘To understand the impacts of emergency situations on blood availability and the resiliency of the US blood supply system’.	‘The absence of blood shortage in both emergency scenarios highlighted the resilience of the inter-regional system to meet the potential associated blood demand’.	Modelling study using administrative data on blood collections and utilization	Blood stocks available at regional and national level	N/A	N/A	Absorptive
Sochas et al.	2017	Sierra Leone	To quantify the extent of the drop in utilization of essential reproductive, maternal and neonatal health services in Sierra Leone during the Ebola outbreak and to model the implication of the decrease in utilization in terms of excess maternal and neonatal deaths	Use of essential reproductive health services decreased as a result of Ebola. This decrease translates to 3600 additional maternal, neonatal and stillbirth deaths in 2014–15 meaning the indirect mortality effects of the crisis may be as important as direct mortality.	Modelling study using routine utilization data to compare projected utilization without crisis to real utilization	Antenatal health services utilization data	N/A	N/A	Absorptive
Taking a community resilience perspective
Andrew et al.	2016	Thailand	To test the impact of two competing hypotheses - bonding and bridging- on enhancing organisational resiliency during the Thailand floods of 2011	Resilience was found to be associated with bridging effect, rural location and private and NGO status rather than public sector organizations	Structured interviews with key informants	Four items on the robustness, resourcefulness, redundancy and rapidity of the organization during the crisis	Framework to assess seismic resilience of communities (Bruneau et al. 2003)	Interdependence	Absorptive
Cohen et al.	2019	Israel	To explore the relationship between the public’s confidence in the availability of healthcare services during and following emergencies, and community resilience	‘Confidence in continuity of health services during a state of emergency was found to be positively correlated with community resilience’	Household survey	Conjoint Community Resilience Assessment Measurement (CCRAM) tool	CCRAM model (Leykin et al. 2013)	Uncertainties Interdependence Legitimacy	N/A
Health workforce issues
Falegnami et al.	2018	Italy	‘To assess the anaesthesia professionals' organizational resilient performance with respect to different daily work conditions’.	The questionnaire shows the potential to assess proxy measures of resilience, despite being complex and time‐consuming.	Survey of anaesthesiologists	Resilience Assessment Grid, adapted using analytic hierarchy process method	Four cornerstones of resilience (Hollnagel et al. 2006)	Knowledge Uncertainties	N/A
Infrastructure and Thermal Resilience
Iddon et al.	2015	UK	To assess the influence of hospital ward design on resilience to health waves and develop model for predicting inside temperatures	Nightingale wards showed remarkable resilience to hot weather. Distributed lag models are a promising method for forecasting inside temperatures	Distributed lag models developed from measured temperatures	Temperature maintenance	N/A	N/A	Absorption
Lomas et al.	2012	United Kingdom	To estimate the resilience of a representative building type, the ‘Nightingale wards’	Nightingale wards demonstrate relative resilience. Modest refurbishment such as insulation, shading, improved natural ventilation and removal of suspended ceilings increase resilience.	Descriptive statistics and calibrated dynamic thermal model developed from measured temperatures	Temperature maintenance	N/A	N/A	Absorption Adaptation
Short et al.	2015	UK	To model temperature for Rosie Maternity Hospital in Cambridge and develop and compare adaptive interventions	The existing building is unable to shed heat so that recommended maximum internal temperatures are reached in relatively mild external conditions. Adaptations to the building will be required to maintain resilience to increasing temperatures.	Multizone thermal dynamic model developed from measured temperatures	Temperature maintenance	N/A	N/A	Absorption Adaptation
Development of preparedness checklists and assessment tools
Dobalian et al.	2016	USA	To develop a hospital preparedness tool for six domains or ‘mission areas’	Tool serves as comprehensive assessment for hospital preparedness	Structured assessment of 140 Veteran Affair Hospitals, two-stage confirmatory factor analysis	Six critical mission areas: programme management, incident management, safety and security, resiliency and continuity, medical surge, support to external requirements	N/A	Uncertainties Intrdependence	N/A
Goncalves et al.	2019	Spain	To adapt and validate the short-form version of the Benchmark Resilience Tool into Spanish language and to explore its relationship with safety climate	The instrument fulfils the psychometric criteria to evaluate resilience in healthcare and nuclear organizations in Spain	Survey of workers in the healthcare and nuclear energy sectors	Short-form version of the Benchmark Resilience Tool (BRT-13B)	Four Cornerstones of Resilience (Hollnagel et al. 2006)	Knowledge Uncertainties Interdependence	Absorption
Zhong et al.	2014a	China	To validate a framework of key indicators of hospital resilience	Identification of a four-factor structure of hospital resilience: Emergency medical response capability, disaster management mechanisms, hospital infrastructural safety and disaster resources with good internal consistency	Survey among 41 tertiary hospitals in China; factor analysis	Eight key domains: hospital safety standard and procedures, emergency command, communication and cooperation system, disaster plan, disaster resource stockpile and logistics management, emergency staff capability, emergency services and surge capability, training and drills, and recovery and adaptation strategies	Four criteria of disaster resilience (robustness, resourcefulness, redundancy and rapidness) (Zhong et al. 2014b)	Uncertainties	N/A
Zhong et al.	2015	China	To develop a framework of key indicators of hospital resilience	Framework identified a comprehensive set of indicators for hospital resilience, 60 of 75 proposed measures reached consensus.	Modified Delphi consultation of experts; Likert scale evaluation of proposed measures

Table 2

Overview of aims, methods, concepts used and dimensions of resilience addressed by mixed methods research papers (n = 9)

Author	Year	Country	Research objective	Key findings	Study design and data source	Operationalization of resilience	Conceptual framework used	Management capacity dimensions	Resilience outcome dimensions
Assessing national-level health system resilience in context of a specific crisis
Ammar et al.	2016	Lebanon	‘To assess the resilience of the Lebanese health system in the face of an acute and severe crisis and in the context of political instability’.	Institutions sustained performance and even improved during the crisis	Case study approach with descriptive statistics of routine data	Indicators on human resources, financing, governance, service provision, utilization, expenditure, morbidity/mortality and prevention of outbreaks	input–process–out- put/outcome model of a health system to measure capacities and performance	Knowledge, Uncertainties, Interdependence, Legitimacy	Absorptive, Adaptive, Transformative
Orru et al.	2018	Estonia	To clarify the factors determining the effectiveness of the Estonian health system in assessing and managing the health risks of climate change	The health effects of climate change have not been mainstreamed into policy.	Document review; expert interviews; population survey	Indicators in five key areas: Policies and programmes; Responses and protective measures; Monitoring and information; Admin. capacity and culture; Issue salience and contextual drivers	WHO Operational Framework for Building Climate Resilient Health Systems (World Health Organization 2015)	Knowledge Uncertainties Interdependencies Legitimacy	N/A
Thomas et al.	2013	Ireland	To develop a framework for health system resilience in economic crises and to apply to Irish health system	Irish health system performed well for adaptive resilience, with mixed evidence for adaptive and transformatory resilience.	Quantitative indicators calculated from gov. documents; Semi-structured interviews	Indicators in for three elements: Financial, adaptive and transformatory resilience	Own framework	N/A	Adaptation Transformation
Assessing health service delivery in context of a specific crisis
Gizelis et al.	2017	Liberia	To assess impact of Ebola epidemic on maternal delivery services	Decline in the use of public hospital deliveries, increase of deliveries in private facilities and equal levels of home births. Private sector played critical role during outbreak.	Population surveys Semi-structured interviews Focus group discussions	Maternity service utilization	N/A	Interdependence Legitimacy	Absorption Adaptation
Kozuki et al.	2018	South Sudan	To document the operations of an Integrated Community Case Management programme during an acute crisis and to assess the programme’s ability to continue operations.	community health workers continued to provide treatment for childhood illnesses during an acute emergency and service provision recovered faster to pre-crisis levels than the formal health sector.	Interviews and focus groups with key stakeholders; routine programme data	Process evaluation of response to crisis, including quant. indicators: reporting, contact, treatment, supervision & referral rates	N/A	N/A	Absorption
Ray-Bennett et al.	2019	Bangladesh	To study the reproductive health challenges at the facility and community levels during the 2016 flood in Belkuchi, Bangladesh	Major challenges of a lack of services and a shortage of medicines, as well as inadequate equipment and insufficient trained health workers were found during both the dry and wet season	Structured facility assessments; Structured interviews with patients	Structured facility assessment tool reviewing: Human resources; Care management; Service delivery performance; Logistics and medical equipment; Essential drugs; Needs assessment for care training Structured interviews covering reproductive health knowledge, utilisation of services and relevant morbidities	N/A	Uncertainties	Absorption
Witter et al.	2017	Uganda, Sierra Leone, Zimbabwe, Cambodia	To analyse the impact of different kinds of shocks on health staff (their vulnerabilities)—but also how they coped (their adaptive capacity).	The impact of shocks and coping strategies are similar between conflict/post-conflict and epidemic contexts—particularly in relation to physical threats and psychosocial threats—while all contexts create challenges for working conditions and remuneration	Surveys with health workers; human resource data; document review; qualitative interviews	Challenges faced in the job and during crisis as well as coping strategies	N/A	N/A	Absorption Adaptation
Development of preparedness checklists and assessment tools
Paterson et al.	2014	Canada	Development of a toolkit to assess health care facility resiliency to climate change	‘The toolkit helps health care facility officials identify gaps in climate change preparedness, direct allocation of adaptation resources and inform strategic planning to increase resiliency to climate change’.	Literature review, workshops	Indicators in two areas: (1) Emergency management and strengthening health care services; (2) Climate-proofing and greening operations	N/A	Knowledge Uncertainties Interdependence	N/A

Table 3

Overview of aims, methods, concepts used and dimensions of resilience addressed by qualitative research papers (n = 16)

Author	Year	Country	Research objective	Key findings	Study design and data source	Conceptual framework used	Management capacity dimensions	Resilience outcome dimensions
Assessing national-level health system resilience in context of a specific crisis
Ager et al.	2015	Nigeria	To identify key pathways of threat to provision, response and adaption for health service resilience in Nigeria in context of Boko Haram	Transport restrictions, health worker migration and suspension of external programmes identified as threat to provision. Political will, indigenous staff commitment and policy changes supported health system recovery/function.	Structured Interviews, systems dynamics analysis with group model building approach	UK Government Humanitarian Policy (DfID 2011)	N/A	Absorption Adaptation Transformation
Alameddine et al.	2019	Lebanon; Jordan	To assess the ‘validity and utility of a capacity-oriented resilience framework […] in Lebanon and Jordan in the context of the Syrian crisis’.	‘We find that UNRWA systems in Lebanon and Jordan were broadly resilient, deploying diverse strategies to address health challenges and friction between host and refugee populations’.	Semi-structured interviews with health professionals in primary care and management functions	‘capacities’ framework (Blanchet et al. 2017)	Uncertainties Interdependence Legitimacy	Absorption Adaptation Transformation
Ling et al.	2017	Liberia	‘To understand how a health system adapts to crisis and how the priorities of different heath system actors influence this response’.	Although the Ebola epidemic stimulated some positive adaptations in Liberia's health system, building a resilient health system will require longer-term investments and sustained attention	Thematic analysis of semi-structured interviews and focus group discussions	‘resilience index’ framework (Kruk et al. 2015)	Knowledge Uncertainties Interdependence Legitimacy	Absorption Adaptation Transformation
Assessing health service delivery in context of a specific crisis
Landeg et al.	2019	United Kingdom	‘To assess the health care system impacts associated with the December 2013 east coast flooding in Boston, Lincolnshire, in order to gain an insight into the capacity of the health care sector to respond to high-impact weather’.	‘The health care sector appears to have limited capacity to respond to weather-related impacts and is therefore unprepared for the risks associated with a future changing climate’.	Semi-structured interviews with key decision-makers;Document analysis	N/A	Knowledge Uncertainties Interdependence	Absorption Adaptation
Ridde et al.	2016	Burkina Faso	To describe the management of the Ouagadougou Terrorist attack in January 2016 from the standpoint of health system resilience.	Identified strengths were an emergency response plan which had been put in place and available blood bank and psychological services. Challenges included the development, application and coordination of framework documents for financial, material and human resources.	Observations and expert interviews structured as anecdotal event report	‘Resilience index’ framework (Kruk et al. 2015)	Knowledge Uncertainties Interdependence Legitimacy	Absorption Adaptation
Back et al.	2017	United Kingdom	‘To examine escalation policy in theory and practice, using resilient health care principles to identify opportunities for improving the way escalation is planned and managed’.	Under pressure it may be difficult to dynamically reconfigure resources, such as staff and equipment and lead to informal management of processes not specified in the policies.	Policy analysis Observation	Concepts for Applying Resilience Engineering (CARE) model of resilient healthcare (Anderson et al. 2016)	Uncertainties	Adaptation
Errett et al.	2018	Canada	‘To identify maritime transportation disruption impacts on available health care supplies and workers necessary to deliver hospital-based acute health care in geographically isolated communities post-disaster’	Critical vulnerabilities to care delivery include ‘lack of information about the existing supply chain, lack of formal plans and agreements, and limited local supply storage and workforce capacity’.	Semi-structured key informant interviews	N/A	Knowledge Uncertainties	N/A
Taking a community resilience perspective
Alonge et al.	2019	Liberia	‘To understand key factors that constitute community resilience and their role in responding to the EVD outbreak in Liberia’.	‘Efforts to systematically build responsible leadership and social capital at community level, including those that strengthen bonds in communities and trust across key actors in the health systems, are needed to address health shocks like EVD outbreaks’.	Key informant interviews and a national stakeholder meeting	N/A	Legitimacy	Absorption Adaptation Transformation
Health workforce issues
Gilson et al.	2017	Kenya; South Africa	To compare experiences from district health systems in Kenya and South Africa order to reveal patterns and insights for everyday resilience	Stable governance structures and adequate resources influence everyday resilience, however empowerment of leaders, mindful staff engagement and social networks also appear important.	Case study methodology: synthesis of document reviews, interviews, group discussions and observations	Vulnerability reduction framework (Béné et al. 2012)Everyday resilience (Barasa et al. 2017)	Knowledge Uncertainties Interdependence	Absorption Adaptation Transformation
Raven et al.	2018	Nepal; Sierra Leone	To assess how health workers cope in times of crisis and how they can best be enabled to continue their work.	‘In both contexts, health workers demonstrated considerable resilience in continuing to provide services despite limited support’.	In-depth interviews with health workers (Sierra Leone); Observation, semi-structured interviews with health workers & management (Nepal)	N/A	Knowledge Uncertainties Legitimacy	Absorption Adaptation
Russo et al.	2016	Portugal	To ‘explore physicians’ perceptions of the changes brought on by the [economic] crisis and associated austerity measures to the market for medical services, as well as to their working routines, remuneration and intention to leave the sector’.	The economic crisis brought considerable changes for the health system, however insights to existence of resilience merged	Semi-structured interviews with physicians	N/A	N/A	Absorption Adaptation Transformation
Thude et al.	2019	Denmark	‘To understand how the staff at the two wards with challenged leader teams coped with everyday work and whether the way in which the staff handled the challenges was resilient’.	‘The staff at both wards were handling the everyday work in a resilient way. […] To increase the resilience in an organisation, leaders should acknowledge the need to establish strong emotional ties among staff and at the same time ensure role structures that make sense in the everyday work’.	Semi-structured interviews with healthcare staff	N/A	N/A	Absorption
Development of preparedness checklists and assessment tools
Toner et al.	2017	USA	To use experiences from communities affected by Hurricane Sandy 2012 for developing a checklist outlining action steps for assessing and strengthening communities' health sector resilience	Description of a conceptual map of health sector resilience, with key findings organized in eight themes. Identification of recommended actions for improvement of health sector resilience at local level	key informant interviews; focus groups; literature review	N/A	Uncertainties Interdependence	Absorption Adaptation
O'Sullivan et al.	2013	Canada	To explore the complexity of disasters at the micro level and to determine levers for action to facilitate collaborative action and promote health among high risk population	‘Promoting population health in disaster context requires shifting from risk management to resilience, […] from command and control models to collaboration’.	Community-based participatory research design with focus groups	Resilient communities framework (Norris et al. 2008)Functional needs framework (Kailes and Enders 2007)	Knowledge Uncertainties Interdependence Legitimacy	N/A
Khan et al.	2018	Canada	‘To describe the essential elements of a resilient public health system and how the elements interact as a complex adaptive system’.	Eleven essential elements for public health emergency preparedness were identified, and a conceptual framework developed with ethics and values at its core.	Focus groups using Structured Interview Matrix facilitation technique	N/A	Knowledge Uncertainties Interdependence Legitimacy	N/A
Meyer et al.	2018	USA	To identify and integrate lessons from response to the EVB epidemic into an actionable checklist.	‘Health care facilities shouldered much of the response, and even those facilities with designated treatment units had to adapt in real time’. Experiences can help inform future response.	Semi-structures key informant interviews	N/A	Knowledge Uncertainties Interdependence Legitimacy	Absorption Adaptation

Assessing national-level health system resilience in context of a specific crisis

Of the eight studies which assessed an entire, national health system in the context of a particular crisis, two studies took a purely quantitative approach: Fukuma assessed Japan’s health system responsiveness and resilience after the Great East Japan Earthquake and Watts assessed the resilience of 101 health systems in the context of climate change. Fukuma operationalized resilience by using composite routine data indicators during the time of crisis, including: service utilization, cause-specific mortality rates incl. suicides, number of hospitals, health expenditures, human resources and immunization coverage. Watts assessed resilience by surveying for the presence of specific policy efforts and strategies in the context of climate change at a national level.

Three further studies assessed the resilience of a health system at the country level using a mixed methods approach. Ammar studied the Lebanese health system in the context of the Syrian refugee crisis using a case study approach. Orru assessed the ways in which the Estonian health system was able to assess and manage the health risks of climate change using a combination of document review, expert interviews and population survey data as applied to the WHO Operational Framework for Building Climate Resilient Health Systems (World Health Organization 2015). Thomas assessed the performance of the Irish health system in the face of the economic crisis by applying quantitative indicators developed from their own resilience framework to government documents and supplementing these with semi-structured interviews.

Finally, three qualitative studies considered national health system resilience. Ager assessed key barriers to the provision of responsive service in the context of Boko Haram in Nigeria, Alameddine assessed the resilience of Lebanon and Jordan’s health systems in the context of the Syrian crisis and Ling assessed the resilience of Liberia’s health system during the Ebola crisis. All three studies used semi-structured interviews with health professionals and other key health stakeholders for data collection, with Ling complementing these with focus group discussions.

Out of these eight studies, all studies except one (Fukuma ) applied a specific conceptual framework to study resilience. However, the frameworks used in the other studies vary, including frameworks developed by international or development agencies, such as the World Health Organization or the United Kingdom’s Department for International Development (Ager ; Orru ; Watts ), general health system frameworks (Ammar ) and resilience frameworks developed in the academic literature (Thomas ; Ling ; Alameddine ).

Assessing resilience of health service delivery

Ten studies focused on the resilience of health service delivery. Six studies assessed the delivery of emergency services, three focused on the delivery of maternal health services, while one considered the continuity of a community health worker programme.

Two quantitative studies from the USA take a specific look at the delivery of emergency services: Radcliff analyse ambulatory care measures during and after a storm, while Simonetti model the potential of the US blood supply system during an emergency. Both make use of available administrative data, with Radcliff relying in utilization data from Veterans Affairs clinics and Simonetti using data on the national availability of blood stocks. The provision of emergency services during a crisis is also explored in four qualitative studies. Two of these assess service provision in the context of a particular crisis: Ridde describe the emergency response to the Ouagadougou Terrorist attack in Burkina Faso, using a mixture of observations and expert interviews as their data source and structuring insights around Kruk ‘resilience indicators’ framework. Landeg assess the emergency response to localized flooding in the UK using semi-structured interviews with decision-makers and document analysis. Finally, two qualitative studies explore the functionality of emergency service processes: while Back use policy analysis and observation to examine escalation policies in UK hospitals, Errett use semi-structured interviews with key informants to identify the impact of disruption of maritime transportation on the provision of emergency services during a disaster.

Being the only purely quantitative study to do so, Sochas analysed the utilization of reproductive, maternal and neonatal health services in Sierra Leone in the context of the Ebola crisis using antenatal health service utilisation data. Gizelis also assessed the impact of the Ebola epidemic on maternity delivery services, using a mixed methods approach by complementing maternity service utilization data from population surveys with semi-structured interviews and focus group discussions. Ray-Bennett looked at the provision of reproductive health services in the context of flooding in Bangladesh, applying a structured facility assessment tool complemented by structured interviews with patients.

Kozuki use a process evaluation methodology to document the ability of an integrated community case management programme to continue operation during the active conflict of 2013 and 2014 in South Sudan. The authors use routine programme data, including reporting, supervision, contact, treatment and referral rates, as well as interviews and focus groups with key stakeholders to evaluate the programme’s resiliency.

Only two of these studies (Ridde ; Back ), both using qualitative methodologies, apply a specific framework of health system resilience. All quantitative studies and one mixed methods study focus on the absorptive capacities of service delivery, while the other studies address a more varied set of resilience dimensions.

Health workforce issues

A total of seven studies were identified which address aspects of health workforce resilience. These include studies both from the tradition of ‘resilience engineering’, as well as research influenced by the concept of ‘everyday resilience’.

One quantitative and one mixed methods study were conducted in the field of resilience engineering and safety research. Falegnami surveyed the resilience of anaesthesia professionals in different work conditions in Italy using the four cornerstones of resilience framework (Hollnagel 2009). In the same setting, Patriarca applied the functional resonance analysis method to explore the potential of the tool in enhancing the resilience of anaesthesia practices, drawing on documentary studies, interviews, observations and patient pathway modelling to do so.

Three studies considered health workforce issues on the context of a specific crisis. Applying a mixed methods approach, Witter explored the impact of shocks on the health workforce across different contexts in Uganda, Sierra Leone, Zimbabwe and Cambodia, with a particular focus on vulnerabilities and coping strategies employed. The authors employed a mixture of methods for analysis, including surveys, human resource data, document review and qualitative interviews. Also taking a cross-national perspective, Raven conducted observations and in-depth interviews with healthcare workers and management in Sierra Leone during the time of the Ebola crisis and in Nepal during a major earthquake to explore coping strategies of staff in both settings. In Portugal, Russo explored physician’s perceptions of the changes in their work environment during the economic crisis in semi-structured interviews.

Finally, two qualitative studies take an ‘everyday resilience’ perspective to understand the ability of health workers in dealing with everyday challenges. Comparing experiences in Kenya and South Africa, Gilson synthesize information from documents, interviews, group discussions and observations to understand factors influencing everyday resilience of staff. In Denmark, Thude conducted semi-structured interviews with healthcare staff to explore the resilience of the workforce faced with challenges in their work environment, including changing leadership structures.

Only two of these studies (Gilson ; Falegnami ) make use of an explicit resilience framework in their analysis. The dimensions assessed in individual studies varies: while Gilson and Raven explore a broad range of management capacities and resilience outcomes, the other five studies focus on only one of these two aspects, with three studies restricted in their analysis to a single outcome dimension (Witter ; Patriarca ; Thude ).

Taking a community resilience perspective

Three studies approached health system resilience from a community perspective. Cohen quantitatively analyse the relationship between community resilience and the public’s confidence in the availability of healthcare services during emergency situations in Israel. Data for this study were conducted using the conjoint community resilience assessment measurement tool (Leykin ) in a household survey. Alonge apply a qualitative approach to understand the relationship between community resilience and health system resilience. Combining information from key informant interviews and a national stakeholder meeting, they look at the contribution of responsible leadership and social capital into the resilience of the health system during the Ebola outbreak in Liberia. Finally, Andrew take a slightly different approach to the issue of community resilience, by focusing on the resilience of community organizations involved with the relief efforts in the aftermath of the Thailand floods in 2011. Applying Bruneau framework on the seismic resilience of communities, the authors quantitatively assess whether the bonding or the bridging effect made a larger contribution on the ability of organizations to deliver essential services after the crisis.

Both quantitative community resilience studies made use of an explicit framework for their analysis, while the qualitative study did not. While one study (Cohen ) focused entirely on dimensions of resilience management capacities, the other two studies explored a mix of management capacities and outcomes.

Infrastructure and thermal resilience

Three studies assessed the infrastructure and thermal resilience of hospitals, taking a purely quantitative approach. Resilience in this context is understood as the capability of buildings to withstand extreme conditions such as heat or earthquakes. Iddon , Lomas and Short assessed the thermal resilience for specific building styles of wards in the UK in order to ensure climate change resiliency. None of these studies used specific conceptual frameworks for their analysis. In terms of the dimensions of resilience addressed, they focused entirely on dimensions of outcome, rather than management capacities. All three studies considered ways in which hospital infrastructure was able to absorb temperature changes, with two studies additionally assessing the potential for adaptation in response to these changes.

Development of preparedness checklists and assessment tools

A total of nine articles described the development of checklists to prepare for future catastrophic events or tools with which such preparedness can be measured. These have been developed at different levels: six studies focused on healthcare facilities and hospitals, two studies considered communities, while one study developed a conceptual framework at the national level.

Four articles described the quantitative development of checklists or measurement tools for assessing resilience of healthcare facilities. Dobalian developed a general hospital preparedness tool, while Zhong developed a framework for measuring hospital resilience and applied it to 41 tertiary care hospitals in a province in China (Zhong ). Goncalves adapted and validated the short-form version of the Benchmark resilience tool for assessing the resilience of healthcare organizations. Using a mixed methods approach, Paterson developed a toolkit for assessing the resiliency of healthcare facilities in the context of climate change. The methods for development differ: while Zhong and Paterson , respectively, used a Delphi consultation and workshops for an expert evaluation of proposed domains, Dobalian , Goncalves and Zhong ) used psychometric assessments to assess validity and reliability. One of the instruments was operationalized as a survey of workers (Goncalves ), while the other three carried out assessments at the organizational level—either by external evaluation (Dobalian ), as a survey completed by managers of the facility (Zhong , 2015) or as a toolkit for facilities aiming to improve their climate resiliency (Paterson ). Finally, Meyer conduct semi-structured interviews with key informants involved in the Ebola response in the USA to develop an actionable checklist to enable preparedness for future responses.

Two further papers used a qualitative approach to develop checklist for enhancing community resilience in a health system context. O’Sullivan ) identify levers to promote community resilience for health during disasters using a community-based participatory research approach. Toner used experiences from Hurricane Sandy collected through key informant interviews and focus groups to develop a checklist for assessing and strengthening communities’ health sector resilience.

Finally, Khan conducted focus groups to develop a framework comprising of essential elements of a resilient public health system in during emergencies, using the lens of complex adaptive health systems. They discuss the importance of recognizing the interconnectedness of actors and processes during an emergency response, acknowledging that these dimensions, while crucial, are particularly difficult to measure and quantify.

Many of these studies understandably did not use a specific resilience framework, as part of the research aim was to develop key dimensions of resilience in a particular context. However, three studies did use frameworks to guide the selection of their proposed dimensions (Zhong , 2015; Goncalves ) or the development of topics for discussion in focus groups (O’Sullivan ). Checklists tended to focus on measuring the management capacities of facilities, organizations and systems, with a noticeable trend towards a more diverse set of dimensions among the qualitative studies. Only two studies (O’Sullivan ; Khan ) considered assessment of the system’s ability for absorption and adaptation.

Conceptual analysis of key empirical studies

Conceptual frameworks used

Across the empirical studies, a specific framework for assessing resilience was used by four quantitative studies, two mixed methods studies and seven qualitative studies. The types and disciplinary origins of the frameworks differed widely. Of the concepts developed in the health systems resilience discourse, the ‘resilience index’ framework (Kruk ), ‘resilience capacities’ framework (Blanchet ) and ‘everyday resilience’ framework (Barasa ) were used. From the resilience engineering discourse, the Concepts for Applying Resilience Engineering (CARE) model (Anderson ) and the Four Cornerstones of Resilience framework (Hollnagel 2009) were applied. Notably, three frameworks from the area of community resilience were used: CCRAM model (Leykin ), framework to assess seismic resilience of communities (Bruneau ) and the resilient communities framework (Norris ). Other frameworks used included the UK government’s humanitarian policy (DfID 2011) and the WHO Operational Framework for Building Climate Resilient Health Systems (World Health Organization 2015). Only two frameworks (Hollnagel 2009; Kruk ) were used twice, all other studies used distinctive frameworks for their analysis.

Dimensions of resilience addressed

We used the framework formulated by Blanchet as an analytical lens allows for a more in-depth analysis of the content and dimensions of resilience addressed across the empirical papers using the definitions of management capacities and levels of resilience provided in Box 1. Across the empirical papers, 12 studies focused exclusively on resilience domains in the ex ante ‘management capacities’ side of Blanchet et al.’s resilience definition, while 14 studies focused exclusively on absorptive, adaptive or transformative levels of the resilience process. Fourteen studies considered both management capacities and resilience levels. Qualitative studies more often considered both management capacities and resilience levels, while quantitative studies more often exclusively focused on one of the two (Figure 4a). Among the management capacities, the dimension of ‘uncertainty’ was most frequently assessed by all types of research, followed by dimensions of ‘interdependence’, ‘knowledge’ and ‘legitimacy’, in that order (Figure 4b).

Figure 4

(a–c) Domains of resilience addressed by key papers (n = 40), by research methodology.

Among the ex post resilience levels, ‘absorptive capacities’ was most frequently addressed across research types, although qualitative research explored ‘absorptive capacities’ and ‘adaptive capacities’ to an equal extent (Figure 4c). Only a limited number of quantitative and mixed methods studies considered the ‘adaptive capacities’ and ‘transformative capacities’ dimensions of health system resilience.

Looking across management capacities and resilience levels, qualitative research was able to address a much broader range of dimensions than quantitative research, with individual studies often exploring multiple dimensions of the resilience concept (Figure 4b and 4c).

Quantitative indicators used

A total of 24 studies used quantitative indicators to measure different aspects of the resilience concept, with several studies using multiple indicators across multiple domains of responsiveness (Table 4). The reported indicators were collected using different data collection strategies, including the use of routine data, observational data and primary survey data. The indicators further differed in the level at which data were collected, spanning national, organizational, staff and patient/population levels. Across the ‘management capacities’ domains, several indicators at different levels of data collection addressed the domains of knowledge, uncertainties and interdependence. However, only two indicators, both collected at population level, captured the legitimacy dimension. Across the ‘levels of resilience’ domains, several studies used indicators across different levels of data collection for the ‘absorption’ domain. However, only three indicators were used for the ‘adaptation’ domain, collected at national and organizational level, while no indicators were identified for the ‘transformation’ domain.

Table 4

Resilience indicators used in quantitative and mixed methods studies (n = 24), by resilience domain and level of data collection

			Level of data collection
		National level	Organizational level	Staff level	Patient/ Population level
Management Capacities	Knowledge	Presence of effective surveillance systems (Ammar et al. 2016; Orru et al. 2018)a Lancet Countdown Survey Item 2.1 ‘adaptation planning and assessment’ (Watts et al. 2018)b	General resiliency indicators for health care facilities (Paterson et al. 2014)a: ‘Builds and enhances climate change knowledge capacity as it relates to hazards of concern for the health care facility’ As well as several knowledge indicators under different emergency scenarios	Short-form version of the Benchmark Resilience Tool Question 1 (Goncalves et al. 2019)b Resilience Assessment Grid Items relating to the ‘ability to monitor’ (Falegnami et al. 2018)b
	Uncertainties	Presence of strategic response policies and plans (Ammar et al. 2016; Orru et al. 2018)a Lancet Countdown Survey Item 2.2 ‘climate information services for health’ (Watts et al. 2018)b	Structured hospital assessment MA2: ‘Incident Management’, MA 3: ‘Occupant Safety’, MA4: ‘Resiliency and Continuity Operations’, MA5: ‘Medical Surge’ (Dobalian et al. 2016)a General resiliency indicators for health care facilities: several indicators related to planning and response policies under different emergency scenarios. (Paterson et al. 2014)a Indicators of hospital resilience (Zhong et al. 2014a, 2015)b	Short-form version of the Benchmark Resilience Tool, Questions 2, 5 and 9 (Goncalves et al. 2019)b Resilience Assessment Grid items relating to ‘international guidelines’, ‘internal procedures’, ‘availability of resources in expected and unexpected complications’, ‘participation into update of procedures and protocols’ (Falegnami et al. 2018)b	Conjoint Community Resilience Assessment Measurement (CCRAM) tool Items 3, 8, 17 and 24 (Cohen et al. 2019)b
	Interdependence	Changes in funding from (international) donors (Ammar et al. 2016)a Presence of multi-sectoral health strategies and partnerships (Ammar et al. 2016)a	Structured hospital assessment MA6: ‘Support to External Requirements’ (Dobalian et al. 2016)a General resiliency indicators for health care facilities (Paterson et al. 2014)a: ‘Builds climate change adaptive capacity through partnerships and by securing mutual support’ As well as several indicators related to interdependence under different emergency scenarios Strength of collaboration between organisations or businesses during a disaster (Andrew et al. 2016) b	Short-form version of the Benchmark Resilience Tool Question 4 (Goncalves et al. 2019) b	Conjoint Community Resilience Assessment Measurement (CCRAM) tool (Cohen et al. 2019)b
	Legitimacy	N/A	N/A	N/A	Trust in institutional efficacy and issue saliency (Eurobarometer survey; Orru et al. 2018)a Conjoint Community Resilience Assessment Measurement (CCRAM) tool Items 1, 6, 15, 19, 21 and 23 (Cohen et al. 2019)b
Levels of Resilience	Absorption	[R] Changes in population-level health indicators (Ammar et al. 2016; Fukuma et al. 2017)a: Morbidity (Case notification rates, outbreaks of new infections) Mortality (Infant Mortality, Under-5 Mortality, Maternal Mortality Ratio, Cause-specific mortality rates) Change in service utilisation indicators(Ammar et al. 2016; Gizelis et al. 2017; Fukuma et al. 2017; Sochas et al. 2017) aChange in service coverage indicators (e.g. vaccination)(Ammar et al. 2016)a Change in availability of medical supplies and human resources(Simonetti et al. 2018; Fukuma et al. 2017; Ammar et al. 2016)a Changes in spending on health and contracts with health facilities (Ammar et al. 2016; Fukuma et al. 2017)a Operationality of health facilities and programmes (i.e. no temporary or permanent health facility or programme closures) (Ammar et al. 2016; Fukuma et al. 2017)a	Change in service utilisation indicators (Radcliff et al. 2018)a Change in availability of medical supplies and human resources(Witter et al. 2017; Ray-Bennett et al. 2019) a Service quality indicators (Kozuki et al. 2018; Ray-Bennett et al. 2019)a Models assessing impact of changes in air temperatures on temperatures of health facilities. (Lomas et al. 2012; Iddon et al. 2015; Short et al. 2015)a Modelling of patient pathways through medical processes under different scenarios (Functional Resonance Analysis Method) (Patriarca et al. 2018)a Four survey items measuring organizational resiliency during a disaster (Andrew et al. 2016)b: Ability to carry out routine tasks and help victims and communities cope Resourcefulness in meeting needs of the victims and their communities Ability to overcome operational disruptions Rapidity of providing victims with resources	Short-form version of the Benchmark Resilience Tool Questions 3 and 7 (Goncalves et al. 2019)b	Health service utilisation during a disaster and reasons for non-utilisation (Ray-Bennett et al. 2019)b
	Adaptation	Indicators for assessing adaptive financial resilience (Thomas et al. 2013)a: Reduction of Unit costs (salaries, wages, fees) Increase in system productivity (Average length of stay, proportion of day cases in acute care) Reduction in staffing with no commensurate reduction in service. Protection of services (no loss of entitlements or rationing by volume) Achievement of stated targets. Lancet Countdown Survey Items 2.3 ‘adaptation delivery and implementation’ and 2.4 ‘spending on adaptation for health and health-related activities’ (Watts et al. 2018)b	Modelling studies effect of different adaptive scenarios health facility temperatures (Lomas et al. 2012; Short et al. 2015)a	N/A	N/A
	Transformation	N/A	N/A	N/A	N/A

Routine data, document review or observation;.

Survey data.

Presence of effective surveillance systems (Ammar ; Orru )

Lancet Countdown Survey Item 2.1 ‘adaptation planning and assessment’ (Watts )

General resiliency indicators for health care facilities (Paterson ):

Short-form version of the Benchmark Resilience Tool Question 1 (Goncalves )

Resilience Assessment Grid Items relating to the ‘ability to monitor’ (Falegnami )

Presence of strategic response policies and plans (Ammar ; Orru )

Lancet Countdown Survey Item 2.2 ‘climate information services for health’ (Watts )

Structured hospital assessment MA2: ‘Incident Management’, MA 3: ‘Occupant Safety’, MA4: ‘Resiliency and Continuity Operations’, MA5: ‘Medical Surge’ (Dobalian )

General resiliency indicators for health care facilities: several indicators related to planning and response policies under different emergency scenarios. (Paterson )

Indicators of hospital resilience (Zhong , 2015)

Short-form version of the Benchmark Resilience Tool, Questions 2, 5 and 9 (Goncalves )

Resilience Assessment Grid items relating to ‘international guidelines’, ‘internal procedures’, ‘availability of resources in expected and unexpected complications’, ‘participation into update of procedures and protocols’ (Falegnami )

Changes in funding from (international) donors (Ammar )

Presence of multi-sectoral health strategies and partnerships (Ammar )

Structured hospital assessment MA6: ‘Support to External Requirements’ (Dobalian )

General resiliency indicators for health care facilities (Paterson ):

Strength of collaboration between organisations or businesses during a disaster (Andrew ) b

Trust in institutional efficacy and issue saliency (Eurobarometer survey; Orru )

Conjoint Community Resilience Assessment Measurement (CCRAM) tool Items 1, 6, 15, 19, 21 and 23 (Cohen )

Change in service utilisation indicators (Radcliff )

Change in availability of medical supplies and human resources(Witter ; Ray-Bennett ) a

Service quality indicators (Kozuki ; Ray-Bennett )

Models assessing impact of changes in air temperatures on temperatures of health facilities. (Lomas ; Iddon ; Short )

Modelling of patient pathways through medical processes under different scenarios (Functional Resonance Analysis Method) (Patriarca )

Four survey items measuring organizational resiliency during a disaster (Andrew ):

Discussion

The concept of health system resilience has soared in popularity in the health system field over the last years, not just in the theoretical or political discourse but also as an object of empirical inquiry. Its application has been incredibly diverse, with research from different disciplines applying the concepts in different healthcare sectors and in various settings. This diversity is not itself problematic. However, this review has demonstrated that empirical studies fundamentally differ in the way that resilience is understood in a health system context.

In terms of the content of the studies, much empirical research focuses on service delivery, health workforce or governance issues, whereas resilience of other health system building blocks is either barely studied, such as health financing, or only studied in high-income countries, as is the case of health information systems. This shows a distinct gap between the concepts and the operationalization of resilience in the context of health system research. If research on health system resilience is to live up to recent comprehensive definitions, the focus has to widen: all building blocks are interlinked and essential for well-functioning health systems, and should therefore not be analysed in singularity, but be considered jointly when assessing health system resilience.

Furthermore, despite much theoretical work on the dimensions which constitute health system resilience, we found that most of the empirical literature only addressed particular aspects. Applying the dimensions outlined by Blanchet , we found that the importance of developing legitimate institutions appears to be neglected in empirical research. This is particularly concerning given that a lack of in healthcare institutions has recently emerged as one of the key barriers to the continued functioning of the health system, e.g. in the context of the Ebola outbreak (Kittelsen and Keating 2019). The ability of health systems to demonstrate transformative capacities has been similarly under-evaluated, especially in quantitative research. Very few empirical studies took an approach to resilience that takes into account the various nuances in the conceptualization of the term which have recently emerged. This trend appeared to be particularly pronounced in those studies with a quantitative or mixed methods approach.

Thus, there is a mismatch between the conceptual models of health system resilience and the way resilience is understood and applied in empirical research both in terms of the breadth of health system factors considered and in terms of the resilience dimensions which are taken into account. Part of the issue may be that the empirical literature assessed in this review comes from a broad range of disciplines, with differing traditions of how ‘resilience’ is understood. While different traditions can offer unique and potentially complementary perspectives on the topic of resilience, this underlines the importance of more clarity in the empirical literature about which concepts and definitions are applied, and how these are then operationalized.

However, only very few empirical studies make use of an explicit conceptual framework for collection or analysis of data, thus not linking research objectives to the rich theoretical body of work on how resilience can be understood in a health system context. Arguably, those studies assessing resilience at a national level were most cognizant of using conceptual frameworks for their analysis. Our review showed that these studies were best able to capture the multiple dimensions of health system resilience. While several other studies aimed to measure health system resilience, they subsequently operationalized this concept in a very narrow way, e.g. by measuring only health service utilization, infrastructure resilience or emergency preparedness. Encouraging the use of an explicit framework for health system resilience could help to strengthen the links between the conceptualization and the operationalization of resilience, thus improving our understanding of health system resilience in different contexts and settings.

Our review further demonstrates that qualitative articles tend to employ a more comprehensive approach to the resilience concept than quantitative studies, which are often limited by availability of data and indicators to few aspects of resilience. The mismatch between concepts and research, therefore, appears to lie not in a lack of appreciation for the complexities of the resilience concept, but rather in a lack of measurable indices which reflect this complexity. While the proposed resilience index (Kruk ) specifies a list of potentially measurable indicators, so far these have only been operationalized in qualitative research. Similarly, the ‘resilience capacities’ framework specified by Blanchet , and the ‘everyday resilience’ framework by Barasa have been operationalized exclusively in qualitative research. All identified quantitative studies have utilized frameworks originating in discourses tangential to the health system resilience discourse.

Yet so far there has been no discussion about which aspects of the health system resilience frameworks are actually measurable. Within the ‘resilience capacities’ framework, the identified studies demonstrate that it is possible to measure ‘absorptive’ aspects by comparing levels of service provision and utilization in different circumstances. However, this is more challenging for ‘adaptive’ and ‘transformative’ aspects. Understanding whether a health system has truly transformed itself in response to a challenge needs to take into account multiple contextual factors and thus lends itself more naturally to be answered by qualitative methods and policy analysis, but also to complexity science. Equally, studies were able to quantitatively assess the presence or absence of preparedness plans to deal with uncertainties and data collection mechanisms for an improved knowledge of potential challenges, but quantifying the ability to handle cross-scale dynamics and develop legitimate institutions proved to be more difficult to capture. Incidentally, the identified studies developing resilience checklists and measurement tools all took a very narrow perspective of resilience by focusing on single healthcare facilities and organizations.

The key question in the development of a comprehensive resilience index, or a measure that allows for effective combination of quantitative and qualitative aspects, becomes whether the requirements to create a comparable measurement tool can be reconciled with the very broad and comprehensive definition of resilience which has emerged from an understanding of health systems as complex adaptive systems. According to Haldane the resilience concept ‘should […] not be prescriptive, but have breadth and flexibility, recognize complexity, consider shocks and cumulative stresses, attempt to deal with disruptions and anticipate future failures’. It appears that, so far, the qualitative literature has been more successful in translating such a comprehensive framework into research practice, while quantitative studies have been limited both by theoretical models and a lack of appropriate data with which to measure resilience. Thus a key task for future researchers in the resilience field will be not only how the resilience concept can be operationalized, but—acknowledging that quantitative assessment of resilience in its entirety is illusionary—determine how measureable aspects can be combined with qualitative aspects in a way that allows for an assessment of health system resilience as a dynamic, complex phenomenon. Thus further research is required for the development of an operational framework on health system resilience which seamlessly integrates both qualitative and quantitative evidence; knowledge from existing guidelines on integrating quantitative and qualitative knowledge, e.g. in the realm of assessing the effectiveness of complex interventions, could be utilized for this purpose (Noyes ).

Our review adds to the existing conceptual review by Turenne , who argue that the concept of health system resilience is still in infancy. We demonstrate the implications of this conceptual immaturity on existing empirical research: while the qualitative literature has explored the notion of health system resilience in its broad definition, the quantitative literature has been limited by the lack of clearly defined characteristics, preconditions and limits of the concept.

Our review makes a substantial contribution to the health systems research literature by analysing the operationalization of the health system resilience concept in empirical studies. Due to our inclusive search and broad inclusion criteria, we were able to consider a broad range of relevant articles from multiple disciplines and thus demonstrate the influence of other disciplines in the health systems research field. However, as search terms were geared to finding articles which specifically referred to the resilience concept, we may have missed empirical studies which operationalized aspects of resilience, but used different terminology. Further research could specifically identify such studies by using elements of the resilience definition instead of merely using the term itself. This could also help to better gain an understanding of how the concept of resilience overlaps with other health systems concepts such as health system strengthening or health system responsiveness and map potential synergies in assessment. We also did not include secondary research or grey literature in our review, which may provide further useful information on the operationalization of the resilience concept. Further research is needed to combine and integrate knowledge from these diverse sources in a comprehensive assessment framework.

A further limitation of our study is the initial exclusion of items based on titles, which was necessary due to the sheer number of results. This may have excluded several studies in associated disciplines, such as those relating to community resilience, which are of importance to the health systems resilience discourse. Findings of our review should be complemented by reviews of the resilience concept in other disciplines to check for congruence.

Conclusion

The health systems research community has made substantial advances in the conceptualization of health system resilience and its potential for the analysis of health systems in changing environments. However, the empirical literature has not yet caught up with the complexities of the concept: there is a mismatch between the nuances and the breadth of the concept at a theoretical level and the way it has been operationalized in empirical studies. In order to do justice to the complexities of the resilience concept, knowledge from both quantitative and qualitative research traditions should be integrated in a way that resilience as a complex, adaptive phenomenon. Only once a comprehensive assessment framework has been defined and applied across different research contexts will the theoretical ‘resilience idea’ be able to more convincingly prove its usefulness for the research community.

Click here for additional data file.