Are preparedness indices reflective of pandemic preparedness? A COVID-19 reality check.

The paper contributes to the body of knowledge working towards enhancing the understanding of crisis and disaster preparedness and effective response, via the lens of the ongoing global pandemic and responding to the questions: do the current measures for pandemic preparedness reflect preparedness adequately, and what does pandemic preparedness mean? We analysed how the reported cumulative mortality rates, during the spring of 2020 and in the 60 days after the date of a country's first COVID-19 related death, compared to the expected preparedness rank according to the existing global preparedness indices (IHR and GHSI) on a country level. We found, at country level, that the health-related outcomes from the first wave of the pandemic were primarily negatively correlated with the expected preparedness. We contend that our results indicate a need to investigate further development and enhancement of the preparedness indices.

Introduction

The ongoing SARS-CoV-2 pandemic represents a global crisis of unprecedented proportions. It has left governments scrambling in response to contain the virus that causes the Corona Virus Disease 2019 (COVID-19) to keep their populations safe. The COVID-19 pandemic has caused extensive morbidity and mortality as well as social, political and economic upheaval [1,2].

In March 2020, the World Health Organisation declared a public health emergency of international concern (PHEIC) [3,4]. On September 29, 2020, information published by WHO, received from national authorities, reported 33,039,504 confirmed cases of COVID-19, including 996,384 deaths in 235 countries, areas or territories [5]. At the time of writing, the COVID-19 pandemic has so far turned out to be a PHEIC of indeterminate duration, scope and effect.

Over the years, the world has experienced several pandemic outbreaks; the Spanish flu in 1918–1920 and more recently in the 2000s, the severe acute respiratory syndrome (SARS) epidemic, the H1N1 pandemic influenza and multiple outbreaks of the Ebola virus disease. The risk of epidemics and pandemics is well known and mitigating actions have been taken globally to prepare for such eventualities [6]. However, the COVID-19 pandemic has proven to be different from previous pandemics in many ways, notably in its high infection numbers and global spread. Therefore, although previous work has investigated pandemic preparedness [[7], [8], [9]], it is important to understand if, and how, the sub-themes of the expected preparedness are reflected in the response. In this work, we hope to narrow down and clarify some of the research avenues with respect to the understanding of pandemic preparedness.

The authors recognise that the COVID-19 pandemic is still unfolding and that the complete impact will remain unknown for some time to come, in all sectors and certainly in the research community. However, we offer an analysis based on the first wave of the pandemic in the [northern hemisphere] spring of 2020 in combination with the accepted characterisations of preparedness and effective response as a way to manage a pandemic. COVID-19 has highlighted shortcomings and exposed vulnerabilities in the preparedness landscape; while some ask the question why aren't we prepared for this pandemic?, we think the more beneficial questions to ask in rethinking pandemic preparedness are: do the current measures for pandemic preparedness reflect preparedness adequately?, what does pandemic preparedness mean?, and - what have we omitted in measuring pandemic preparedness?

The study follows an assumption and hypothesises that existing preparedness indices will, as stated in the aims of the indices, reflect the capacities of the response and the impact (measured as mortality) of a pandemic event. The paper provides a contribution to the body of knowledge working towards enhancing the understanding of crisis and disaster preparedness and response, via the lens of an ongoing global pandemic.

The rest of this paper is laid out as follows. We start with an overview of disaster management and pandemic preparedness and response specifically. After that, we outline the method used for this work. We then present our findings and discuss these findings from a pandemic preparedness and effective response perspective, from which we draw our conclusions and suggestions for future work.

Main article: pandemic preparedness within the disaster management framework

Disaster management

This section contains an exploration of the relevant literature around disaster and pandemic management including preparedness and response. It should be noted that the study of disaster as well as pandemic preparedness and response are both inter- and multi-disciplinary; the topics bring together the subjects of risk, crisis, disaster, economics, logistics, leadership, sociology, psychology and public health, among others.

For many decades now, an important instrument used to capture and explain the diverse activities associated with disaster management is the disaster management cycle, usually shown as having four interrelated phases; reduction/mitigation, readiness/preparedness, response and recovery [10,11]. The phases refer to the activities that take place at each stage; mitigation refers to arrangements to remove or reduce factors which may bring about disasters while preparedness involves planning for and developing response capabilities. According to UNISDR [12]; response is immediately post-event and involves rescue as well as restoration of essential services. Recovery is the phase in which the immediate needs of the affected community have been met and longer-term activities to restore society and infrastructure take place. The entirety of the disaster management cycle also includes the crafting of public policies and creation of plans that are relevant for each phase.

The genesis and evolution of the disaster management cycle have been contributed to by various disciplines, among them civil defence, geography, sociology, psychology, public administration, engineering and development studies [13,14]. Neal [15] asserts that studies on the different stages of disaster can be traced back to the 1930s and that the use of typing and classification, by both practitioners and academics, of different stages of disaster management was to enable understanding of their different fields of practice and discipline and to improve their response to disaster events.

As well, over time, the disaster management community has adopted a whole-of-community and whole-of-society approach to disaster management [16,17]. This is, in part, characterised by a culture of preparedness, advocating for all segments of a society to do their part in disaster risk reduction and preparedness.

Latterly, a landmark agreement, the Sendai Framework for Disaster Risk Reduction (SFDRR), was adopted by 187 United Nations member states in 2015 [18]. The Sendai Framework is voluntary, directs more attention to the management of risk and not just the management of disasters and aims to reduce disaster losses in lives, livelihoods and health through a series of agreed actions. The goals and priority areas for action within the Sendai Framework focus on the results of Disaster Risk Reduction (DRR) achieved through determined Disaster Risk Management (DRM) actions. These DRM actions highlight the departure from overly emphasising response and recovery to managing disaster risk with attention to reduction (mitigation), readiness (preparedness), response and recovery [19].

Importantly, the Sendai Framework, while pointing to the implementation of an all-hazards approach to managing disaster risk, contains multiple specific references to health, including linkages to epidemics and pandemics and references to the International Health Regulations [20]. The Framework highlights both biological hazards (e.g., epidemics and pandemics) and natural hazards (e.g., cyclones and earthquakes) as key areas for disaster risk management. Specifically included in the aims and actions of the Sendai Framework -

“The substantial reduction of disaster risk and losses in lives, livelihoods and health and in the economic, physical, social, cultural and environmental assets of persons, businesses, communities and countries … through the implementation of integrated and inclusive economic, structural, legal, social, health, cultural, educational, environmental, technological, political and institutional measures that prevent and reduce hazard exposure and vulnerability to disaster, increase preparedness for response and recovery and thus strengthen resilience” [18], p7

The Framework further outlines voluntary commitments specific to health including enhancing the preparedness and resilience of national health systems through training and capacity development of health personnel (including in understanding risks and using disaster risk in their work), the integration of disaster risk management into health care provision at all levels as well as the implementation of IHR (2005).

In 2016, the International Conference on the Implementation of the Health Aspects of the Sendai Framework for Disaster Risk Reduction 2015–2030 – The Bangkok Principles - recommended measures aimed at assisting countries to prevent and/or reduce the risk of health emergencies, e.g., pandemics, that have the potential for large-scale loss of life and social and economic impact. The Bangkok Principles press for coherence between disaster and health risk management through inter-operable, multi-sectoral approaches [21]. UNISDR [22] asserts that -

“The Principles build on the commonality between the two realms, such as the shared need for risk assessment, surveillance and early warning systems, resilient infrastructure, and coordinated incident management. This strengthened coherence is needed as the inter-connected and transboundary nature of hazard risk increases”

Regardless of definition, it can be deduced that there is agreement among practitioners, policymakers, academics and other stakeholders that planning and preparedness prior to a disaster event are essential to minimising the risks and any resulting damages. Furthermore, the capacity to understand the scale and scope of an event as well as communicating, coordinating and working together in response are touted as critical factors in successful disaster management.

Concept of preparedness

Before going into COVID-19 preparedness, we start by striving to understand and define disasters and preparedness.

The United Nations International Strategy for Disaster Reduction, UNISDR, defines disaster as –

“a serious disruption of the functioning of a community or a society involving widespread human, material, economic or environmental losses and impacts, which exceeds the ability of the affected community or society to cope using its own resources” [12].

This is the definition also used by WHO. In addition, WHO defines a public health emergency as -

“ …. an occurrence or imminent threat of an illness or health condition, caused by bio terrorism, epidemic or pandemic disease, or (a) novel and highly fatal infectious agent or biological toxin, that poses a substantial risk of a significant number of human facilities or incidents or permanent or long-term disability (WHO/DCD, 2001). The declaration of a state of public health emergency permits the governor to suspend state regulations, change the functions of state agencies.” [6]

WHO utilises the UNISDR and OCHA-WFP characterisations of preparedness –

“Activities and measures taken in advance to ensure effective response to the impact of hazards, including the issuance of timely and effective early warnings and the temporary evacuation of people and property from threatened locations (ISDR) 2. Pre-disaster activities, including an overall strategy, policies, and institutional and management structures, that are geared to helping at-risk communities safeguard their lives and assets by being alert to hazards and taking appropriate action in the face of an imminent threat or the actual onset of a disaster (OCHA-WFP).” [6].

Lastly, a pandemic is defined as -

“an epidemic occurring worldwide, or over a very wide area, crossing international boundaries and usually affecting a large number of people” [23]

The Covid-19 pandemic clearly fulfils the definitions of a public health emergency as well as a disaster, as it has caused excess deaths in several countries and caused huge economic losses globally [24].

Pandemic preparedness and response

Several authors have written on the difficulty of defining what constitutes public health preparedness (see for example [[25], [26], [27]]. Khan et al [28] propose a public health preparedness framework that includes 11 interrelated aspects covering preparedness, response and recovery. These aspects include leadership, collaborative networks, planning and community engagement. Khan et al. further point out that public health emergency preparedness is complex.

Globally, countries around the world have spent decades preparing for the scenario that COVID-19 has brought about. The International Health Regulations (2005) are the global legal framework agreed to by 195 countries (as at 2019), including all WHO member states, to work together towards global health security. These regulations outline an individual country's responsibilities and WHO's role in the declaration and management of a PHEIC. The IHR went into effect in 2007 and WHO has the coordinating function.

At the 64th World Health Assembly, a review of IHR (2005), specific to the H1N1 pandemic of 2009 was presented [29]. The review found that -

“The world is ill prepared to respond to a severe influenza pandemic or to any similarly global, sustained and threatening public-health emergency”

On account of what the IHR sets out to do, it is fair to say that there was an intention of shared or co-responsibility among the countries involved as a way of reducing the burden on any single entity. Importantly, WHO also recognises the cross-border and transnational nature of global health, which requires actions at both micro (national) and macro (global) levels. More recently, after a large-scale outbreak of the Ebola virus disease of 2014–2016 in West Africa, expert deliberations led to guidance and recommendations for pandemic preparedness which underscored strengthening standards and capacities to determine national epidemic and pandemic preparedness levels [30].

To understand national pandemic preparedness, there are various instruments that have been developed. The previously described IHR can be used to rank a country's pandemic preparedness with countries asked to self-report their IHR compliance annually using the online International Health Regulations Monitoring Tool (IHRMT). Countries self-report on 8 core capacities (legislation policy, coordination, surveillance, response, preparedness, risk communication, human resource capacity and laboratory) and 4 hazards (zoonotic events, food safety events, chemical events, radiation emergencies) with the findings compiled in the States Parties Annual Report (SPAR).1

The Global Health Security Index, GHSI, is a collaboration between the Nuclear Threat Initiative (NTI) and the Johns Hopkins Center for Health Security (JHU) and was developed in collaboration with The Economist Intelligence Unit (EIU) [31]. The GHSI has 34 categories across 6 indicators and as at 2019 had benchmarked health security capabilities across the 195 States Parties to the IHR. GHSI attempts to provide objective and measurable individual country compliance with IHR, using a combination of data from previous pandemics and information from international health systems’ information as a way to improve global health security and the international capability to address infectious disease outbreaks [31]. Crucially, unlike the self-reported IHR which may be subject to self-reporting bias, GHSI is non-voluntary.

A third instrument, the Epidemic Preparedness Index (EPI), is said to complement the IHR's JEE by providing a holistic view of preparedness and is constructed to support comparative risk assessment between countries measuring relative epidemic and pandemic preparedness across 188 countries [32]. Lastly, the INFORM Epidemic Risk Index assesses the risk of countries to epidemic outbreak, which would exceed the national capacity to respond to the crisis [33]. Table 1 shows a summary of the contents of the IHR, GHSI, EPI and INFORM indices.

Table 1

Comparison of the more commonly used pandemic preparedness indices globally.

	GHS Index	IHR monitoring framework	EPIa	INFORM epidemic risk
Purpose	Assesses and benchmarks health security capabilities	Set of legal instruments designed to ensure and improve the capacity to prevent, detect, assess, notify, and respond to public health risks and acute events	Measures a country's capacity to detect and respond to infectious disease events	Risk assessment tool for humanitarian crisis, disasters and epidemics
Categories	Prevention of the emergence, early detection and reporting, rapid response and mitigation of spread, sufficient and robust health sector, commitments to national improvements, overall risk environment	Legislation, coordination, surveillance, response, preparedness, risk communication, human resources, laboratory, points of entry, zoonosis, food safety, chemical, radio-nuclear	Public health infrastructure, physical infrastructure, institutional capacity, economic resources	Hazard and exposure, vulnerability, lack of coping capacity
Top performers	United States, UK, Netherland, Australia, Canada, Thailand, Scandinavia and South Korea	Canada, United States, Singapore, Norway, Cuba, South Korea, Malaysia, Japan and China	European countries, United States, Canada and Australia	Scandinavia, Benelux countries and UK, Singapore and Canada

EPI and INFORM have not been accounted for in the empirical part of this study, but are portrayed in Table 1 for comparison.

These preparedness instruments rank countries on how well a country should handle and manage a pandemic. The aim of the preparedness rank is that it reflects capabilities the country has to respond effectively to a disaster. Confronted with COVID-19, this study strives to conduct a “reality check” on preparedness and put to test the theoretical level and rank of a country's preparedness.

Method

Some of the basic public health goals of any pandemic response are to minimise transmission and to reduce loss of life [34]. For this review study, we analysed the pandemic impact (see [35]) as reported cumulative mortality rates in the 60 days after the date of a county's first COVID-19 related death compared to the expected preparedness measured by the existing preparedness indices (IHR and GHSI) at a country level. We did not use EPI as there were no raw data available and the model clustered countries into five groups which is not detailed enough for the purposes of this paper. We did not use the INFORM Epidemic Risk Index either since it is mainly concerned with detailing national vulnerabilities and risks as pertains to pandemic preparedness. Lastly, data for both IHR and GHSI are collected for the 195 States Parties to IHR, across similar and complementary categories, enabling better comparison of rankings.

We used this country specific time period (first death + 60 days) to ensure comparability of results across different countries and to reflect particularly over the theoretical preparedness, acute response and also impact with mortality as a proxy. This approach controls for the fact that the spread of the pandemic happened at a different point in time in different countries. Additionally, the approach excludes countries where the time interval we use, 60 days, had not passed, at the time data were collected, since the first reported COVID-19 related death.

Our use of the COVID-19 mortality rates is a proxy indication of how successful a pandemic response was for the time period in this paper. The University of Washington's Institute for Health Metrics and Evaluation (IHME) describes the use of the number of deaths per day as -

“the best indicator of the progression of the pandemic, although there is generally a 17- to 21-day lag between infection and deaths.” [36]

The COVID-19 mortality data used in this work were collected from Johns Hopkins University [37]. The death time-series in each country begins from the first COVID-19 death and ends 60 days later, but latest at end of May 2020. Here, we applied the total COVID-19 deaths per million people. The data were downloaded June 1, 2020 [38] and reflect the COVID-19 situation in the [northern hemisphere] spring of 2020. The time frame includes what could be considered the first wave2 of the COVID-19 pandemic in most of the European countries.

It should be noted that death tolls have their problems regarding the reporting procedures and categorisations of cause of death as being COVID-19 because testing policies and death rate data differ between countries. For example, some countries only report COVID-19 deaths that occur in hospitals – COVID-19 deaths at home may not be recorded. Some countries only report deaths with a positive COVID-19 test – untested individuals may not be included. Still, we see the reported mortality rate as the most reliable proxy for the acute response and the effectiveness of the initial response to the pandemic. Other potential indicators for effective response that would have directly tested the indices, could have been COVID-19 cases or health care surge capacity [39]. Mortality data are more reliable, compared to the number of infected people since the latter can be affected by sampling bias due to the different testing protocols and capacity in each country [40]. To empirically study the relevance of the two indices, IHR and GHSI, we compared country rankings in the indices to the number of COVID-19 deaths per capita. First, we analysed the IHR and GHSI data and ranked all the countries based on the indicators. GHSI had built-in rankings, in the case of the IHR the ranking is based on average of indicators scores. Finally, we performed a scatter plot analysis and calculated correlations with the indices and their individual components and COVID-19 mortality. We calculated the correlations between the overall IHR index as well as between the sub-themes and actual individual components of the index and COVID-19 mortality. To examine the significance of the correlation, we performed a Pearson correlation test and explored the resulting correlation coefficients and p-values.

Results

In Table 2, Table 3 , the top 10 performing countries of the indices are illustrated (for reasons of space, only the 10 countries with the highest COVID-19 mortality data for each index are shown here. See appendix 1 for the complete rankings). In general, the countries seem to rank quite similarly in both indices. However, some differences exist. In the IHR, Canada, US, Singapore, Norway and Cuba hold the highest ranks, while in the GHSI US, UK, Netherlands, Australia and Canada are ranked the highest. Canada and the US are ranked amongst the top ten in both indices. Table 2, Table 3 also portray the number of deaths/1 million people for the first 60 days after the first reported COVD-19 death. The average number of deaths/1 million people for the first 60 days after the first reported COVID-19 death was 47 deaths/1 million people for all countries in IHR and GHSI, with a minimum of 1 death and a max of 747 deaths/1 million people (see appendix 1).

Table 2

Top 10 ranked International Health Regulation (IHR) countries and COVID-19 mortality.

Country	International Health Regulations (IHR) Rank	Cumulative reported deaths/1 M first 60 days
Canada	1	121
United States	2	184
Singapore	3	4
Norway	3	41
Cuba	5	7
Saudi Arabia	6	11
Malaysia	7	3
South Korea	8	5
Japan	9	1
China	10	2

Table 3

Top 10 ranked Global Health Security Index (GHSI) countries and COVID-19 mortality.

Country	Global Health Security Index (GHSI) Rank	Cumulative reported deaths/1 M first 60 days
United States	1	184
United Kingdom	2	433
Netherlands	3	302
Australia	4	4
Canada	5	121
Thailand	6	1
Sweden	7	319
Denmark	8	93
South Korea	9	5
Finland	10	55

Scatter plots were used to observe the relationship between national rankings in IHR and GHSI against COVID-19 mortality data. The scatter plot for GHSI national rank against mortality data is presented in Fig. 1 below. Different geographical areas of the world are distinguished based on colour, e.g., yellow dots represent South Asian countries. The scatter plot shows a slight indication that the better ranked countries seem to have higher mortality rates than the countries with lower ranks. However, it is important to note that the mortality data were extracted on June 1st and some of the regions had not yet peaked in terms of mortality at the time (e.g., Latin America).

Fig. 1

Global Health Security Index (GHSI) country rank plotted against COVID-19 deaths per 1 M people.

At the time of writing, countries situated on the African continent did not report high COVID-19 mortality rates in the [northern hemisphere] spring of 2020. The countries with highest mortality rates during spring 2020 are mostly situated in North America and Europe (Ecuador as an exception). The countries with the highest mortality rates (Belgium, Spain, Italy, UK) can be seen to be ranked among the top 40 best prepared in the GHSI index ranking and among the 70 best in the IHR ranking (see Fig. 2 ).

Fig. 2

International Health Regulations (IHR) country rank plotted against COVID-19 deaths per 1 M people.

To test the hypothesis that higher ranked countries also have higher mortality rates, we performed a Pearson correlation test. According to the test, GHSI rank had a negative correlation with mortality (p < 0.001). IHR was also negatively correlated, with a p-value 0.02.

To summarise the results in Fig. 1, Fig. 2, the health-related outcomes from the first wave were primarily negatively correlated with the expected pandemic preparedness capacities and capabilities as measured by the existing IHR and GHSI preparedness indices on a country level. This demonstrates that the higher a country's overall ranking in both the IHR and/or GHSI lists, the more COVID-19 deaths per capita in the time period we used.

Furthermore, we analysed the correlations between individual questions across the six categories in GHSI and COVID-19 country mortality data. These findings are presented in Table 4, Table 5 . For the analysis, we used the 40 countries with the highest mortality data. With this subset of countries, we identified sub-indicator items which had the highest positive and negative correlation with mortality. Table 4 contains the 10 most negatively correlated sub-indicator items against COVID-19 mortality data while Table 5 contains the 10 most positively correlated sub-indicator items against COVID-19 mortality data for the 40 countries.

Table 4

Ten most negatively correlated Global Health Security index (GHSI) sub-indicator items against COVID-19 mortality data for 40 countries with highest mortality data in GHSI.

Sub-indicator	Category	Indicator	Specific question/item	GHSI scoring/scale	Pearson correlation	P-value
4.6.2a	4. Health System - Sufficient & Robust Health System To Treat The Sick & Protect Health Workers	Capacity to test and approve new medical countermeasures	Is there a government agency responsible for approving new medical countermeasures (MCM) for humans?	Yes = 1No = 0	−0.463	0.003**
5.5.3b	5. Compliance with International Standards - Commitments To Improving NationalCapacity, Financing And Adherence To Norms	Financing	Is there evidence that the country has, in the past three years, either invested finances (from donors or national budget) or provided technical support either to • Support other countries to improve capacity to address epidemic threats?• Improve the country's domestic capacity to address epidemic threats?Needs to meet at least one of the criteria to be scored a 1 on this measure.	Yes = 1No = 0	−0.250	0.119
3.1.1c	3. Rapid Response - Rapid Response To And Mitigation Of The Spread Of An Epidemic	Emergency preparedness and response planning	If this plan is in place, does it include considerations for pediatric and/or other vulnerablepopulations?	Yes = 1No/no plan in place = 0	−0.250	0.120
5.5.2a	5. Compliance with International Standards - Commitments To Improving NationalCapacity, Financing And Adherence To Norms	Financing	Is there a publicly identified special emergency public financing mechanism and funds which the country can access in theface of a public health emergency (such as through a dedicated national reserve fund, anestablished agreement with the World Bankpandemic financing facility/other multilateralemergency funding mechanism, or otherpathway identified through a public health or state of emergency act)?	Yes = 1No = 0	−0.242	0.133
6.2.3a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Socio-economic resilience	Poverty headcount ratio at $1.90 a day (2011PPP) (% of population)	Yes = 1No = 0	−0.184	0.255
2.3.2a	2. Detection & Reporting - Early Detection & Reporting For Epidemics Of Potential International Concern	Epidemiology workforce	Is there public evidence that the country has at least 1 trained field epidemiologist per 200,000 people?	Yes = 1No = 0	−0.180	0.266
2.1.2a	2. Detection & Reporting - Early Detection & Reporting For Epidemics Of Potential International Concern	Laboratory systems	Does the country participate in a regional or international laboratory network?	Yes = 1No = 0	−0.177	0.274
4.5.1a	4. Health System - Sufficient & Robust Health System To Treat The Sick & Protect Health Workers	Infection control practices and availability of equipment	Has the country published a publicly available plan, strategy, or similar document to addresspersonal protective equipment (PPE) supplyissues for both routine national use and during a public health emergency?	Yes = 1No = 0	−0.138	0.396
1.2.1c	1. Prevention - Prevention Of The Emergence Or Release Of Pathogens	Zoonotic disease	Is there a department, agency, or similar unit dedicated to zoonotic disease that functionsacross ministries?	Yes = 1No = 0	−0.130	0.424
3.6.4a	3. Rapid Response - Rapid Response To And Mitigation Of The Spread Of An Epidemic	Access to communications infrastructure	Percentage point gap between males and females whose home has access to theInternet	Yes = 1No = 0	−0.122	0.453

Table 5

Ten most positively correlated Global Health Security index (GHSI) sub-indicator items against COVID-19 mortality data for 40 countries with highest mortality data in GHSI.

Sub-indicator	Category	Indicator	Specific question/item	GHSI scoring/scale	Pearson correlation	P-value
4.3.1c	4. Health System - Sufficient & Robust Health System To Treat The Sick & Protect Health Workers	Healthcare access	Out-of-pocket health expenditures per capita, purchasing power parity (PPP; current international $)	31.5–2325.7	0.473	0.002**
6.5.1b	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Public health vulnerabilities	Healthcare Access and Quality (HAQ) Index frontier score	35–96.6	0.459	0.003**
1.2.4a	1. Prevention - Prevention Of The Emergence Or Release Of Pathogens	Zoonotic diseases	Number of veterinarians per 100,000 people	0–229	0.458	0.003**
6.5.1a	6. Risk Environment - Overall Risk EnvironmentAnd Country Vulnerability To Biological Threats	Public health vulnerabilities	Total life expectancy (years)	62.47–89.4	0.423	0.007**
6.3.1a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Infrastructure adequacy	What is the risk that the road network will prove inadequate to meet needs?	1, 2, 3, 4	0.414	0.008**
6.4.3a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Environmental risks	What is the risk that the economy will suffer a major disruption owing to a natural disaster?	1, 2, 3, 4	0.400	0.011*
6.2.2a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Socio-economic resilience	United Nations Development Programme (UNDP) Gender Inequality Index score	0.39–0.96	0.400	0.011*
6.1.1a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Political and security risk	Government effectiveness (EIU score)	1, 2, 3, 4	0.385	0.014*
3.6.1a	6. Rapid Response - Rapid Response To And Mitigation Of The Spread Of An Epidemic	Access to communications infrastructure	Percentage of households with Internet	31–98	0.373	0.018*
6.5.3a	6. Risk Environment - Overall Risk Environment And Country Vulnerability To Biological Threats	Public health vulnerabilities	Domestic general government health expenditure per capita (PPP)	56–8078	0.368	0.020*

For the sub-indicators of the GHSI listed in Table 4, the more negative the correlation value, the lower the mortality numbers. This seems to suggest that the better prepared the country is in these respects, the less death the country has experienced during the first phase of COVID-19. Thus, for the sub-indicators listed in Table 4, the preparedness index seems to function as it was intended to. For example, if the country responded that they have a publicly identified special emergency public financing mechanism in place and at least one (1) trained field epidemiologist per 200,000 people this correlated with fewer deaths. Nevertheless, these correlations are not statistically significant (besides the first question) and are spread across all six of the GHSI categories.

The individual GHSI sub-indicator items that have a positive correlation with death numbers are listed in Table 5, positive correlation means that the country ranks similarly both in terms of the preparedness index as well as in mortality. Unlike for Table 4 with sub-indicators across all the GHSI categories, it can be noted that most sub-indicators in Table 5 are from category 6 of GHSI (overall risk environment and country vulnerability to biological threats). There are questions related to economics and health care financing e.g., out-of-pocket health expenditures per capita, government health expenditure per capita, as well as risk that the economy will suffer. For the subcategory of “overall risk environment and country vulnerability to biological treats” of the GHSI, the index did not function as intended.

Discussion: COVID-19 response – infection prevention and control: containment and mitigation

In the early stages of the COVID-19 pandemic, much was unknown about this novel contagion. This meant that what may theoretically have seemed to be effective ways of responding to the outbreak may have been inadequate or not enough as authorities were faced with a novel crisis, which they had limited knowledge of [41]. According to Boin and t'Hart [42]; in certain types of crises the outcome is fairly predictable, e.g., those related to floods, but the outcome of all crises is not immediately evident. Our study shows that countries that were assumed to be well prepared for an epidemic outbreak suffered some of the worst health outcomes and impact from COVID-19 in the [northern hemisphere] spring of 2020, as measured by mortality data.

At the time of this writing, there was still no widespread treatment for COVID-19 and the vaccines were in the initial phases of roll-out. This left non-pharmaceutical interventions as ways of responding to COVID-19. Information from the OECD country policy tracker [43] and from the CoronaNet Project [44] demonstrated that nearly all governments employed some form of policy response tools to the COVID-19 pandemic encompassing health, economic, social and medical interventions [45]. Countries utilised both containment and mitigation actions in response to COVID-19. Some of the COVID-19 non-pharmaceutical intervention and prevention measures included avoiding exposure to the virus by maintaining personal hygiene, i.e., washing hands regularly, maintaining physical distance, not gathering in large groups, e.g., in workplaces, wearing face masks and self-isolating/quarantining if ill or suspected to be ill. The director of the University of Washington's Institute for Health Metrics and Evaluation (IHME) put forward that -

“the science is clear and the evidence irrefutable: mask-wearing, social distancing, and limits to social gatherings are vital to helping prevent transmission of the virus” [46]

One way of contextualising this is that individual countries across the globe were dealing with a similar problem, at the same time and with access to almost the same information. We also note that for pandemic preparedness, both IHR and GHSI call for concerted actions across national borders and yet the initial individual country responses to COVID-19 were dissimilar and disparate.

Furthermore, it is important to note that if different countries have different scores in comparison to each other, on each of IHR and GHSI, that this points to pandemic preparedness levels being fairly divergent globally with different countries at different levels of preparedness. This divergence is concerning in part because it has implications for any response to a pandemic that transcends borders and calls for international cooperation. Also, it is apparent from the results of the study that ranking high in IHR and GHSI has not led to a good result in terms of COVID-19 mortality. The authors would thus like to ask that if a country's rank on the IHR and GHSI is not reflected in the actual response and impact of a pandemic, what are the ranks reflections of?

One explanation could be found in the limitations of the methodology: using country rankings in the IHR and GHSI indices ignores the demographic, cultural and political difference between countries, and it may well be that the differences between the countries that rank in the, say, top 20–40 are negligible, since they are all high income developed countries. There might also be a bias in terms of countries ranking better, having more developed reporting structures in place. However, if this were the case, it is even more surprising to observe such large differences in the mortality figures between the top-ranking countries.

The existing preparedness indices may have been constructed based on lessons learned from previous epidemics which have either been suppressed (e.g., SARS), have had a lower mortality (e.g., H1N1 of 2009), or have transmitted with a slower pace (e.g., HIV). It may also be that the preparedness indices have not been updated for our modern societies especially in consideration of extensive [air] travel. However, even if their theoretical preparedness did not save the European countries from severe outcomes, the countries and responders should not see this as a directive not to improve preparedness but rather to discuss how preparedness can be better understood and improved for future pandemic response. Therefore, we contend that our results indicate a need to investigate further development and enhancement in the preparedness indices.

The GHSI, for example, seems to include all preparedness relevant categories, e.g., early detection and reporting, rapid response and mitigation of spread, sufficient and robust health sector, commitments to national improvements and overall risk environment. While the IHR also includes aspects of governance and regulation in categories of legislation, coordination, surveillance and risk communication, as well as some aspects of supply, e.g., human resources and laboratory capacity.

From the results in the previous section, a look at Table 4 shows the negative correlations for specific GHSI sub-indicators to COVID-19 death rates do not follow a specific pattern. The negative correlations can be identified across all six categories of GHSI. By proxy, this may be a signal that there are opportunities for review and enhancement across all areas of GHSI. Table 5, on the other hand, with positive sub-indicator correlations to death numbers, has 80% of indicators from category 6 of GHSI, overall risk environment.3 This could be taken to mean that joint preparedness, at the international – transboundary - level, is inadequate and incomplete, as evidenced by the somewhat uneven international response to COVID-19. For instance Boin [47], refers to transboundary crises and writes that contemporary society is ill-prepared to deal with such crises.

The preparedness indices and improved preparedness could include aspects of collaboration across nations, both for coordination as well as for sharing of resources and information (data sharing), the ability and speed of political decision making and ability to execute decisions, public adherence to official guidelines and instructions, as well as a country's potential to ramp up capacity (service and supply). For instance, particularly rapid decision making and the quicker start of social distancing measures seem to have had an impact in the outcome of the first wave of COVID-19 [48]. Additionally, both IHR and GHSI emphasise international cooperation, however, it was shown during the initial response phases that there was difficulty in international cooperation as well as disagreements between experts. These seemingly individual, yet interconnected, country interests and social phenomena do not seem to have been adequately accounted for in IHR and GHSI for pandemic preparedness. Furthermore, challenges in coordinating communication to the public may have contributed to a fragmented response. For example, the conflicting information among experts with respect to the use of masks and to availability of masks resulted in mixed messages, especially for the public.

Furthermore, both IHR and GHSI assess health system capacity encompassing health facilities, personnel, equipment and access. However, research shows that there was a health system overload in some regions in Europe during spring 2020 (see, e.g., Nuñez-Gil et al [49]). The overload resulted in the reduced ability to handle the large number of COVID-19 patients as well as the reduced ability to handle non-COVID-19 conditions which then contributed to additional mortality [50]. Thus, the normal state healthcare capacity is not a sufficient indicator for the preparedness of a country. Rather, we propose that the capability to quickly ramp up capacity for surge demand seems to be important in responding to pandemics. The usage of masks and PPE also seemed to have had a role in the response of COVID-19 [51]. Therefore, capacity to ramp up supply, either in terms of production or procurement of PPE could be included into existing preparedness indices.

Taken together, our findings support the work of Boin [47] on the ability of authorities' response to novel crises and transboundary crises as well as Moon's [45] comment regarding the policy and governance challenges brought about by the COVID-19 pandemic. As well, Fukuyama [52], contends that “it takes a state”, i.e., able state institutions, effective leadership and citizens’ trust in the government to effectively respond to the challenge presented by the COVID-19 pandemic. Indeed, it can be deduced from indications in both academic and non-academic sources on the COVID-19 response that these are areas requiring further research for pandemic preparedness and response.

Conclusions

At the beginning of this paper, we asked do the current measures for pandemic preparedness reflect preparedness adequately?, what does pandemic preparedness mean?, and what have we omitted in measuring pandemic preparedness? The COVID-19 pandemic has revealed that it remains unclear how best to measure countries’ capacity to respond effectively to severe pandemic threats. The cause of the disparity that exists between the IHR and GHSI overall scores and COVID-19 death rates is difficult to explain without further investigating pandemic preparedness.

We found that the health-related outcomes from the first wave (in the northern hemisphere spring of 2020) were primarily negatively correlated with the expected preparedness measured by the existing preparedness indices on a country level. Put another way, the countries with better preparedness did not have better health outcomes in the first wave as measured by the number of COVID-19 deaths. For this pandemic, national level health preparedness rankings were not an indicator of how well a country handled the pandemic. In fact, we ask the question, did the health rankings lead to a false sense of confidence among countries or was COVID-19 impossible to prepare for, is the COVID-19 pandemic much different from previous pandemics? We contend that COVID-19 was not novel and was not unexpected, before COVID-19 there was the Spanish Flu of 1918, SARS in 2002 and MERS in 2012 among others.

In answering the question what have we omitted in measuring pandemic preparedness? academics and policymakers should consider what other elements should be considered in preparing better for the next pandemic. The COVID-19 pandemic has highlighted weaknesses in the global health system, specifically in pandemic preparedness and response. The world is increasingly interconnected and given that infectious diseases know no borders, assessing the capacity of countries to manage infectious disease outbreaks individually is insufficient. Our economic, social and political links now transcend the categories in the indices and include cross-border pathogenic contexts. The pandemic has shown that the world is highly interconnected and assigning a pandemic preparedness number to singular countries needs to be re-thought.

A quantitative approach to ranking country health preparedness, i.e., IHR and GHSI, is beneficial for dealing with a large dataset, (195+ countries) as well as for a quick snapshot of where each country stands in relation to an aggregate score. However, the current quantitative measures do not capture social phenomena, e.g., individual country cultural/communal characteristics, type of political leadership, pace of decision making or even size of national budget/economy or the meaning behind such phenomena.

Our findings show that while IHR and the GHSI rank pandemic preparedness from the perspective of national level processes and mechanisms, they do not sufficiently highlight the potential significance that sub-national pandemic preparedness and response capabilities, citizen practice (e.g., distancing and wearing masks), mis- and dis-information (e.g., in risk communication), the tension between health, economic and social policies for response, [political] leadership (e.g., consistent), and even trust (e.g., in leadership decisions) can shape response outcomes. As COVID-19 transcended national borders, these softer elements, coupled with some of the harder quantitative elements, seemed to be more significant predictors of outcomes. Future research should investigate these elements in relation to pandemic preparedness and response.

Lastly, it is clear that COVID-19 is not just a health issue, we propose that other avenues for prospective research should include investigating a more global approach to pandemic prevention and preparedness, as alluded to in IHR (2005), as well as investigation into factors that would contribute to integrated global pandemic preparedness and response. A pathogen may originate in one country, however, in-country efforts alone may be insufficient and lead to a fragmented global response. WHO itself advised that without regional or global leadership and collaboration on formal pandemic planning, preparedness may veer even further in different directions across the world.

We are aware that our study considers only the initial period of the global COVID-19 pandemic. In future research, repeating the study with data from a longer time period when the total death toll is known, would improve the robustness of our results.

Availability of data

Public.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Country	IHR ranking	GHSI ranking	Deaths/1 million people, first 60 days at threshold
Canada	1	5	121
United States	2	1	184
Singapore	3	24	4
Norway	3	16	41
Cuba	5	110	7
Saudi Arabia	6	47	11
Malaysia	7	18	3
South Korea	8	9	5
Japan	9	21	1
China	10	51	2
Russia	11	63	25
Australia	12	4	4
New Zealand	12	35	5
Egypt	14	87	5
Cyprus	16	77	19
Netherlands	17	3	302
Thailand	18	6	1
Venezuela	18	176	0
Indonesia	20	30	4
Finland	20	10	55
Germany	20	14	88
Morocco	23	68	5
India	24	57	2
El Salvador	26	65	7
Mexico	27	28	44
Sweden	29	7	319
Nicaragua	30	73	5
Portugal	31	20	118
United Arab Emirates	32	56	24
Brazil	33	22	74
Czech Republic	34	42	29
Macedonia	35	90	53
Italy	36	31	415
Bahrain	37	88	7
South Africa	37	34	8
Belarus	39	108	25
Oman	42	73	8
United Kingdom	43	2	433
Switzerland	44	13	171
Armenia	45	44	29
Denmark	46	8	93
Spain	47	15	544
Uruguay	48	81	6
Colombia	49	65	12
Turkey	51	40	49
Cote d'Ivoire	52	105	1
Latvia	53	17	13
Guyana	54	137	5
Luxembourg	56	67	165
Costa Rica	57	62	2
Chile	60	27	31
France	61	11	241
Uzbekistan	61	116	0
Kuwait	63	59	54
Iraq	64	167	3
Philippines	65	53	1
Belgium	67	19	747
Azerbaijan	68	117	4
Jamaica	69	147	3
Sri Lanka	71	120	0
Kazakhstan	74	83	2
Bahamas	75	142	28
Paraguay	76	103	2
Liberia	77	111	6
Greece	78	37	14
Lebanon	78	73	4
Bolivia	80	102	25
Moldova	81	78	52
Hungary	83	35	46
Panama	84	68	55
Iran	85	97	61
Angola	86	170	0
Lithuania	87	33	22
Slovenia	89	12	50
Romania	90	60	60
Ecuador	91	45	132
Niger	92	132	3
Poland	93	32	21
Ireland	94	23	295
Algeria	96	173	12
Zimbabwe	96	92	0
Iceland	99	58	29
Trinidad and Tobago	100	99	6
Ghana	101	105	1
Jordan	102	80	1
Suriname	104	100	2
Bangladesh	104	113	2
Qatar	107	82	10
Mauritius	107	114	8
Argentina	111	25	6
Bulgaria	112	61	13
Tanzania	113	101	0
Croatia	115	38	24
Austria	118	26	69
Estonia	121	29	48
Brunei	122	128	5
Peru	122	49	85
Israel	125	54	32
Honduras	129	156	18
Zambia	132	152	0
Democratic Republic of Congo	133	161	1
Cape Verde	134	146	5
Libya	135	168	1
Sudan	138	163	2
Syria	140	188	0
Myanmar	144	72	0
Kenya	144	55	1
Togo	146	129	2
Tunisia	148	122	4
Dominican Republic	150	91	41
Bosnia and Herzegovina	151	79	41
Montenegro	153	68	14
Guatemala	154	125	2
Cameroon	159	115	6
Gabon	160	186	5
Nigeria	162	96	1
Kyrgyzstan	163	47	2
Ukraine	164	94	12
Pakistan	168	105	4
Albania	169	39	11
Burkina Faso	170	145	2
Senegal	172	95	3
Serbia	173	41	34
Mali	176	147	4
Botswana	178	139	0
Afghanistan	180	130	5
Gambia	185	117	0
Mauritania	187	157	5
Congo	191	173	3