An axiomatic approach towards pandemic performance indicators.

During a pandemic, each country (or region) is characterized by a status matrix indicating its positive cases, hospitalizations and deaths. A pandemic performance indicator is a real-valued mapping from the set of status matrices to the set of non-negative real numbers, whereby lower values stand for better performance. We show that four axioms together characterize a family of indicators arising from a weighted average of the incidence rate, morbidity rate and mortality rate. We use these indicators to evaluate the impact of COVID-19 in major countries worldwide.

Introduction

The COVID-19 pandemic has shaken the world massively. As the crisis unfolded, competing narratives about the impact of the disease arose. They each focused on different indicators regarding the performance of countries or regions. International or regional comparisons are constantly made at almost any level. In the case of a pandemic, they are necessary, for instance, to properly determine triage or the allocation of relief funds. But even comparing the simplest data (such as those capturing the basic effects of a pandemic) can be complex. Thus, there is an obvious need to obtain normative foundations for indicators that might play a focal role to make those comparisons. That is the aim of our paper.

We provide a stylized framework in which each country/region is described by a status matrix. Each row in the matrix represents each individual from that country/region, who is described by a triple referring to three dichotomous dimensions with respect to a pandemic: infection, hospitalization and death. More precisely, an individual is described as if not affected by the pandemic at any level; as if tested positive (infected), but neither hospitalized nor dead; as if tested positive and hospitalized (allegedly because being a serious or critical case), but not dead; and as if died from the pandemic (thus, after testing positive and, most likely, being hospitalized too).1 A (pandemic performance) indicator is simply a mapping associating to each status matrix a non-negative number, with the convention that the lower the value the better. Instances of focal indicators are the incidence rate, morbidity rate (proxied by the relative number of hospitalized individuals), mortality rate, or combinations of them.2

Pandemic performance can be understood in different ways. For instance, one might take a flow approach, being interested in how a country is dealing with the pandemic at a given moment of time (or over a short period of time, such as the last two weeks, for instance). Alternatively, one might take a stock approach being interested in how a country has dealt with the pandemic since it began. Furthermore, one might take a single country approach, being interested in how a single country is coping with the pandemic (e.g., whether the policies it has put into place are working), or, alternatively, a multiple countries approach, being interested in how different countries compare to one another. Depending on the chosen perspective, different indicators and normative assumptions will be relevant. For instance, according to the flow approach the number of people infected or hospitalized in a given time period will need to be taken into account, but for the stock approach we might want to know how many people have immunity (as a result of previous infections or a result of vaccinations). If we are comparing countries, we must be confident that our variables are measured in a similar way in the countries we are comparing. As it will become clear later in our empirical application, we shall endorse here a multiple countries flow approach. Our analysis will thus have obvious implications, as well as limitations.

In order to obtain normative foundations for some of the indicators mentioned above, we pursue the axiomatic approach, which is standard in normative economics (e.g., Thomson, 2001). That is, we formalize axioms reflecting some ethical or operational principles and explore their implications. Our starting point is the principle of impartiality, with a long tradition in the theory of justice (e.g., Moreno-Ternero and Roemer, 2006) stating that ethically irrelevant aspects should be excluded from our analysis. The basic formulation of such a principle is typically the axiom of anonymity, which in our setting says that a permutation of the rows in the status matrix does not change the value of the indicator. That is, the name of individuals does not matter in the evaluation.3 We then move to another fundamental requirement in distributive justice, which is to set meaningful lower or upper bounds (e.g., Thomson, 2011). In our setting, this is reflected by the axiom of perfect status, which states that a zero status matrix (i.e., a country with no deceases, hospitalizations or infections from the pandemic) yields the lowest possible value of the indicator (namely, zero).4 More generally, we can consider the generic principle saying that if the underlying data of a problem change in a specific way, the solution should change accordingly. This is typically reflected by a monotonicity axiom, which also have a long tradition of use in economics (e.g., Kalai and Smorodinsky, 1975, Roemer, 1986). In our setting, the axiom states that if the health condition of an individual in the society worsens (from infected to hospitalized, or from hospitalized to deceased), then the indicator cannot decrease. Finally, we consider a decomposability axiom relating the performance of a population with the performance of subpopulations. The axiom is akin to those namesakes used in the literature on income inequality measurement (e.g., Bourguignon, 1979), poverty measurement (e.g., Foster et al., 1984) or income mobility measurement (e.g., Fields and Ok, 1996). It also implies another standard axiom in axiomatic work, known as replication invariance (e.g., Blackorby et al., 1996).

Our main result states that the combination of the four axioms stated above characterizes a family of indicators arising from a weighted average of the incidence rate, morbidity rate and mortality rate. The weights are unspecified, beyond being increasingly ordered and positive.5 To specify them, one would need to consider additional axioms, to be combined with the above four. For instance, considering axioms that would state the independence with respect to two of the three dimensions in our setting. As a matter of fact, following that approach we obtain characterization results for the incidence rate, morbidity rate and mortality rate as independent indicators. Alternatively, we can take the dominance approach, popularized in the literature on income inequality (e.g., Sen and Foster, 1997). For instance, for each pair of countries, we could obtain all possible weights for which the corresponding indicators within the family provide a consistent ranking for that pair of countries.

We also consider a composition axiom relating the performance of a population with the performance of the resulting population after adding one individual. The (variable-population) axiom is reminiscent of the so-called “path independence” axiom for choice functions (e.g., Plott, 1973). It also has a relative in the theory of axiomatic bargaining: the so-called “step-by-step negotiations” axiom introduced by Kalai (1977). The same principle has also been frequently used in other related contexts (e.g., Young, 1988, Moulin, 2000, Moulin and Stong, 2002, Moreno-Ternero and Roemer, 2012, Martínez and Sánchez-Soriano, 2021). We show that replacing the decomposability axiom in our main result with the composition axiom just described, we characterize another family of indicators arising from a weighted average of the total (instead of relative) incidence, morbidity and mortality, instead of their rates. As with the main result, the weights are unspecified, beyond being increasingly ordered and positive. The independence axioms mentioned above would permit to obtain characterization results for the absolute incidence, absolute morbidity and absolute mortality as independent indicators.

Our work aligns with the tradition of axiomatic work in economics that can be traced back to the 1950’s (e.g., Nash, 1950, Arrow, 1951, Shapley, 1953). The last seven decades have witnessed numerous applications of the axiomatic approach to the measurement of somewhat unconventional concepts, such as polarization (e.g., Esteban and Ray, 1994), economic insecurity (e.g., Bossert and D’Ambrosio, 2013), poverty reduction failure (e.g., Chakravarty and D’Ambrosio, 2013) or resilience (e.g., Asheim et al., 2020).

Closer to home, Hougaard et al. (2013) explore the implications of normative principles for the evaluation of population health.6 They characterize focal population health evaluation functions, such as the unweighted aggregation of quality-adjusted life years (QALYs) or healthy years equivalents (HYEs) and generalizations of the two. Although the settings are different, our axioms of perfect status and monotonicity are reminiscent of their axioms of perfect health superiority and positive lifetime desirability, respectively.

Obviously, our work also aligns with the recent efforts to assess various effects of the COVID-19 pandemic, in what has been dubbed epidemic-related economics (e.g., Murray, 2020). These efforts range from designing optimal targeted lockdowns (e.g., Acemoglu et al., 2021, Alvarez et al., 2021) to estimating the economic benefits of tests (e.g., Atkeson et al., 2020), or the impact of indoor face mask mandates and other non-pharmaceutical interventions (e.g., Karaivanova et al., 2021, as well as to explore procedural rationality and persistent behavior in human responses during pandemic outbreaks (e.g., Dasgupta et al., 2020, Dasgupta et al., 2021).7

The rest of the paper is organized as follows. We present our model, axioms and main indicators in Section 2. We provide the characterization results in Section 3. Section 4 is devoted to an empirical application for the case of COVID-19. Finally, Section 5 concludes. For a smooth passage, all proofs have been deferred to an appendix.

The model

Let be the set of individuals in a region/country. Let denote the set of dichotomous dimensions that will characterize each individual with respect to a pandemic: case, hospitalization, and death. More precisely, for each , let denote ’s health status. We write if is not affected by the pandemic whatsoever, if is a case (i.e., tested positive), if is a serious case, which required hospitalization, and if died because of the pandemic. A status matrix is thus a matrix , in which each row denotes an individual and each column a dimension, and such that, for each , . For each , is simply ’th row at . For each , let denote ’th column at . We denote (overall number of cases), (overall number of hospitalizations), and (overall number of deaths).

Let be the domain of all possible matrix statuses of (society) . We shall also consider a variable-population generalization of the model. Then, there is a set of potential individuals, which are indexed by the natural numbers . Let be the set of finite subsets of , with generic element . We denote by the class of all possible matrix statuses with variable population.

An indicator is a real-valued function such that if and only if the status is at least as good as the status .

Indicators

We now list some natural instances of indicators. The first three are quite simple and evaluate the health status of a country by the number of cases, hospitalizations, and deaths per capita, respectively.

Incidence rate. For each ,

Morbidity rate. For each ,

Mortality rate. For each ,

The next indicator is the arithmetic mean of the three previous indicators.

Average performance. For each ,

Finally, the indicator referring to the fatalities among infected people.

Fatality ratio. For each ,

Axioms

We now list axioms for indicators, reflecting some ethical or operational principles. The first axiom is a standard principle of impartiality, requiring that permutations of rows in a status matrix do not alter the indicator. It therefore implies that the name of individuals should not be relevant for the evaluation.

Anonymity. For each , where is a permutation of the rows of .

The second axiom states that if a country is in perfect health (no cases, no hospitalizations, and no deaths) then the indicator must be zero (the lowest possible value).

Perfect status. For each , if then .

The third axiom states that if the health condition of an individual has worsened (from infected to hospitalized or from hospitalized to deceased), then the indicator cannot decrease.

Monotonicity. For each pair , if there exists such that

( and ) or ( and ),

for each and for each ,

then .

The fourth axiom is the following. Suppose the original population is partitioned in two (disjoint) groups of individuals and (e.g., two regions of a country). Decomposability states that the performance of the original population must be a (size)-weighted average of the performances of subgroups and .8

Decomposability. Let such that . For each and each , where is the matrix resulting from concatenating above (by rows).

A somewhat related axiom states that if a new individual joins the group, then the indicator of the new status matrix coincides with the indicator of the original matrix plus the indicator for the (one-row) matrix corresponding to that individual.

Agent composition. For each , and each , let denote the resulting matrix from concatenating as an additional row to . Then, .

Note that this axiom implies that the indicator is weakly increasing in the size of its population (i.e., adding people can only drive the indicator up). As a matter of fact, as it will be clearer from the characterizations result below, this axiom will be instrumental to characterize absolute indicators, as opposed to relative indicators. A normative justification to use the former instead of the latter would be, for instance, caring about absolute numbers from the standpoint of providing aid.

We conclude our inventory with three independence axioms.

First, the axiom that indicates independence with respect to deaths and hospitalizations.

Case relevance. For each pair , if then .

Similarly, the next axiom indicates independence with respect to deaths and cases.

Hospitalization relevance. For each pair , if then .

Finally, the next axiom indicates independence with respect to hospitalizations and cases.

Death relevance. For each pair , if then .

Characterizations

The main result of our paper is the characterization arising from combining the first four axioms introduced above. It actually yields a generalization of the average performance indicator, when (positive and ordered) weights are given to each of the three dimensions.9

An indicator satisfies anonymity, perfect status, monotonicity and decomposability if and only if there exist such that, , and for each ,

Our next result is a characterization of specific members of the family of indicators from the previous result, obtained upon replacing monotonicity by each of the independence axioms listed above.

The following statements hold:

An indicator satisfies anonymity, perfect status, decomposability, and case relevance if and only if it is a linear transformation of the incidence rate indicator, i.e., there exists such that, for each ,

An indicator satisfies anonymity, perfect status, decomposability, and hospitalization relevance if and only if it is a linear transformation of the morbidity rate indicator, i.e., there exists such that, for each ,

An indicator satisfies anonymity, perfect status, decomposability, and death relevance if and only if it is a linear transformation of the mortality rate indicator, i.e., there exists such that, for each ,

The previous results characterize relative indicators, in which the population of each country was taken into account. But attention has also been paid to absolute indicators (see, for instance, the data provided by the Johns Hopkins coronavirus research centre). It turns out we can provide parallel characterizations to those above upon simply replacing the decomposability axiom by the composition axiom. More precisely, the next result is the characterization arising from combining anonymity, perfect status, monotonicity and agent composition. It actually yields an aggregation of the performance indicators across dimensions, when (positive and ordered) weights are given to each of the three dimensions.

An indicator satisfies anonymity, perfect status, monotonicity and agent composition if and only if there exist such that, , and for each ,

Our last result is a characterization of specific members of the family of indicators from the previous result, again replacing monotonicity by each of the independence axioms listed above.

The following statements hold:

An indicator satisfies anonymity, perfect status, agent composition and death relevance if and only if it is a linear transformation of the number of deaths, i.e., there exists such that, for each ,

An indicator satisfies anonymity, perfect status, agent composition and hospitalization relevance if and only if it is a linear transformation of the number of hospitalizations, i.e., there exists such that, for each ,

An indicator satisfies anonymity, perfect status, agent composition and case relevance if and only if it is a linear transformation of the number of infections, i.e., there exists such that, for each ,

Application: Countries under COVID-19

We now apply some of the indicators characterized above to COVID-19. We use data from Our World in Data, which provides daily deaths, hospitalizations, and infections for 32 somewhat homogeneous countries.10 Our analysis is based on weekly observations of the three variables within the period from March 3, 2020 to January 17, 2022.

We shall consider three indicators (within the family characterized in Theorem 1). One is the death rate (thus, arising when ). Another is the average performance indicator (thus, arising when ), which assumes that all three variables (deaths, hospitalizations, and infections per capita) are equally relevant to assess the performance of each country with respect to COVID-19. And the third one is the weighted average performance indicator, when weights are geometric (, and ). Formally, let denote the status matrix of country at a given date. Then,

We acknowledge that there are certain limitations in our empirical application. To begin with, incidence, hospitalization and mortality might not be measured in exactly the same way in all countries. Likewise, incidence is influenced by the amount of testing (and even by the type of tests) which might also differ among countries. Finally, hospitalization numbers may depend upon the capacity of the hospital system and the availability of hospitals. Nevertheless, with all those caveats (and also due to the fact that countries within our sample are relatively homogeneous), we do believe our empirical exercise might still be a good application for our theoretical model analyzed above. And it will also allow us to derive interesting insights.

Tables 1, 2, Table 3, Table 4 illustrate the application of the previous indicators (with the scaling mentioned above) on four specific dates: April 6th, 2020; January 11th, 2021, June 7th, 2021, and January 17th, 2022. The first two dates and the last one roughly correspond to peaks in the distribution, whereas the third one to a dip in the distribution. Columns 2 to 4 in Tables 1, 2, Table 3, Table 4 gather the values provided by the indicators , , and , respectively, to each of the countries from the list. Columns 5 to 7 yield the countries’ relative positions in the corresponding rankings. The higher the position in the ranking, the lower the indicator and, therefore, the better the performance of the nation with respect to the pandemic.

Table 1

Evaluations and relative positions of countries according to , , and , on April 6th, 2020. For ease of exposition, all numbers within columns 2, 3 and 4 are multiplied by .

	Indicator			Ranking
Country	I^1	I^2	I^3	I^1	I^2	I^3
Austria	1.6	34.3	93.9	16	15	17
Belgium	18.5	152.6	575.6	29	29	29
Bulgaria	0.1	5.5	17.3	3	2	5
Canada	1.7	29.8	68.1	18	14	13
Croatia	0.2	10.4	13.4	4	6	3
Cyprus	0.1	26.6	44.2	2	12	8
Czechia	0.7	17.6	38.9	9	9	7
Denmark	1.6	40.1	86.6	17	17	16
Estonia	0.8	27.5	71.1	11	13	15
Finland	1.0	22.8	50.7	12	11	11
France	9.4	144.6	421.2	26	28	28
Hungary	0.7	16.9	54.6	10	8	12
Iceland	1.1	70.1	118.6	14	22	18
Ireland	3.5	113.9	217.3	20	27	23
Israel	0.6	38.1	69.9	8	16	14
Italy	6.6	105.1	363.8	25	25	27
Latvia	0.2	8.6	18.2	5	4	6
Lithuania	0.2	9.0	12.3	6	5	2
Luxembourg	4.7	110.0	271.3	22	26	25
Netherlands	5.7	66.8	199.7	24	20	22
Norway	1.0	21.0	50.4	13	10	10
Poland	0.4	8.1	16.3	7	3	4
Portugal	2.1	65.8	131.1	19	19	19
Slovakia	0.0	4.7	5.0	1	1	1
Slovenia	1.2	16.3	49.5	15	7	9
Spain	9.8	85.0	231.6	28	23	24
Sweden	4.9	60.9	192.1	23	18	21
United Kingdom	9.4	87.2	320.4	27	24	26
United States	4.5	70.0	137.1	21	21	20

Table 2

Evaluations and relative positions of countries according to , , and , on January 11th, 2021. For ease of exposition, all numbers within columns 2, 3 and 4 are multiplied by .

	Indicator			Ranking
Country	I^1	I^2	I^3	I^1	I^2	I^3
Austria	4.0	171.8	301.9	14	10	8
Belgium	3.1	144.9	240.5	6	6	5
Bulgaria	5.2	107.9	350.2	15	4	11
Canada	2.7	141.3	219.6	5	5	4
Croatia	6.1	174.8	407.5	19	11	14
Cyprus	2.2	225.5	326.1	4	14	9
Czechia	11.4	618.3	987.8	28	26	28
Denmark	3.5	148.8	245.2	10	7	6
Estonia	3.2	303.3	444.5	8	17	17
Finland	0.6	34.0	48.7	2	2	2
France	3.8	230.3	397.5	13	15	13
Hungary	7.2	155.7	410.3	22	8	15
Iceland	0.0	20.8	29.7	1	1	1
Ireland	5.3	544.9	731.2	16	25	22
Israel	3.6	678.0	805.8	11	28	24
Italy	5.7	221.9	434.9	17	13	16
Latvia	6.9	391.8	670.4	20	20	20
Lithuania	9.2	391.5	750.9	25	19	23
Luxembourg	3.5	157.2	251.6	9	9	7
Netherlands	3.8	246.1	334.1	12	16	10
Norway	0.8	61.8	82.6	3	3	3
Poland	5.7	181.2	396.3	18	12	12
Portugal	10.4	703.8	989.4	27	29	29
Slovakia	10.2	627.8	949.5	26	27	27
Slovenia	8.8	541.6	850.8	23	24	25
Spain	3.1	470.8	625.0	7	21	19
Sweden	8.8	369.8	579.8	24	18	18
United Kingdom	11.5	541.1	878.7	29	23	26
United States	7.0	504.5	723.3	21	22	21

Table 3

Evaluations and relative positions of countries according to , , and , on June 7th, 2021. For ease of exposition, all numbers within columns 2, 3 and 4 are multiplied by .

	Indicator			Ranking
Country	I^1	I^2	I^3	I^1	I^2	I^3
Austria	0.2	24.3	39.9	12	8	7
Belgium	0.5	55.1	83.2	13	20	22
Bulgaria	1.2	48.7	171.2	27	17	29
Canada	0.5	29.8	51.0	16	11	10
Croatia	1.1	37.5	90.3	26	15	23
Cyprus	1.1	51.1	81.5	25	19	21
Czechia	0.6	17.5	33.1	18	6	5
Denmark	0.1	60.3	68.1	7	23	17
Estonia	0.2	34.6	52.7	11	12	13
Finland	0.1	10.5	11.7	4	3	3
France	0.6	61.8	129.5	17	25	26
Hungary	0.5	13.9	37.5	15	4	6
Iceland	0.0	3.4	4.0	2	2	1
Ireland	0.0	47.6	51.7	2	16	12
Israel	0.1	1.9	6.2	9	1	2
Italy	0.8	29.4	65.2	23	10	16
Latvia	2.3	82.3	161.1	29	28	28
Lithuania	1.2	65.6	132.4	28	26	27
Luxembourg	0.0	37.4	43.9	2	14	8
Netherlands	0.2	59.9	71.8	10	22	18
Norway	0.1	23.3	26.6	5	7	4
Poland	1.1	14.1	51.1	24	5	11
Portugal	0.1	50.3	61.2	8	18	15
Slovakia	0.6	26.4	48.1	19	9	9
Slovenia	0.8	60.4	96.6	22	24	24
Spain	0.7	82.4	108.7	20	29	25
Sweden	0.5	58.4	80.2	14	21	20
United Kingdom	0.1	73.5	79.6	6	27	19
United States	0.7	35.2	59.6	21	13	14

Table 4

Evaluations and relative positions of countries according to , , and , on January 17th, 2022. For ease of exposition, all numbers within columns 2, 3 and 4 are multiplied by .

	Indicator			Ranking
Country	I^1	I^2	I^3	I^1	I^2	I^3
Austria	0.8	1742.7	1789.8	5	15	14
Belgium	1.4	2487.6	2578.7	9	23	23
Bulgaria	8.4	948.5	1301.3	29	6	8
Canada	2.7	436.1	559.4	17	1	1
Croatia	7.1	1467.9	1574.1	28	10	10
Cyprus	3.5	1520.9	1660.3	19	11	11
Czechia	1.7	1457.2	1529.3	11	9	9
Denmark	2.0	4602.4	4673.6	12	29	29
Estonia	2.0	1862.0	1963.3	13	16	17
Finland	0.4	981.8	1024.9	3	8	5
France	2.5	3783.8	3944.0	15	27	27
Hungary	6.1	806.4	980.5	26	3	4
Iceland	0.0	2609.0	2622.6	1	24	24
Ireland	1.0	871.8	942.2	7	5	3
Israel	0.8	4546.7	4620.0	6	28	28
Italy	4.0	2054.7	2220.1	23	20	20
Latvia	3.5	1735.3	1881.6	20	14	15
Lithuania	3.9	1551.4	1719.9	22	13	12
Luxembourg	1.4	2254.4	2309.7	8	22	22
Netherlands	0.3	1875.7	1895.7	2	17	16
Norway	0.6	2021.2	2043.2	4	19	18
Poland	4.1	583.7	755.2	24	2	2
Portugal	2.6	3335.4	3434.2	16	25	25
Slovakia	6.3	867.7	1051.8	27	4	6
Slovenia	3.6	3399.5	3537.3	21	26	26
Spain	2.1	1925.0	2062.2	14	18	19
Sweden	1.6	2217.8	2286.7	10	21	21
United Kingdom	2.8	974.0	1095.0	18	7	7
United States	4.3	1535.9	1736.1	25	12	13

On April 6th, 2020 (the first date we consider, Table 1), we can see that, for instance, Slovakia is at the top of the three rankings and Belgium at the bottom of the three rankings. Bulgaria, Croatia, Latvia and Lithuania are also consistently at the top of the rankings, but with some small differences among each of them. On the other hand, France, Italy, Spain and the United Kingdom are also consistently at the bottom of the rankings, but also with some small differences among each of them. More notable differences occur for some countries in middle positions. For instance, Iceland appears 14th in the first ranking whereas 22nd in the second ranking (and 18th in the third). Similarly, Slovenia appears 15th in the first ranking (right after Iceland), whereas 7th in the second ranking (and 9th in the third).

On January 11th, 2021 (the second date we consider, Table 2), we can see that, for instance, Iceland is at the top of the three rankings, but there is no single country at the bottom of the three rankings (Belgium happens to be much better than on the previous date, with top positions in each ranking on this date). Finland, Norway and Canada are also consistently at the top of the rankings (with very small differences). On the other hand, Czechia, Portugal and (again) the United Kingdom are also consistently at the bottom of the rankings, with some small differences among each of them. Notable differences occur, for instance, for Israel, which appears 11th in the first ranking whereas 28th in the second ranking, and 24th in the third. Similarly, Spain appears 7th in the first ranking whereas 21st in the second ranking, and 19th in the third. On the other hand, Hungary appears 22nd in the first ranking (thus, 11 positions below Israel), whereas 8th in the second ranking (thus, 20 positions above Israel) and 15th in the third.

On June 6th, 2021 (the third date we consider, Table 3), we can see that, for instance, Iceland continues to be at the top of the three rankings (tied in the first with Ireland and Luxembourg, and below Israel in the second) and Latvia is, essentially, at the bottom of all of them (Spain is slightly worse in the second ranking and Bulgaria is slightly worse in the third ranking). Finland and Norway are also consistently at the top of the rankings (with some small differences). There are more differences at the bottom of the rankings, but countries such as Bulgaria, France, Lithuania and Spain appear therein for some of them. Israel now does much better (9th at the first ranking 1st in the second ranking and 2nd in the third). The United Kingdom still performs poorly in the second ranking (27th, thus almost at the very bottom) but much better in the first ranking (6th). Notable differences also occur, for instance, for Poland, which appears 24th in the first ranking whereas 5th in the second ranking, and 11th in the third. On the other hand, Denmark appears 7th in the first ranking whereas 23rd in the second ranking, and 17th in the third. Finally, Ireland is at the top of the first ranking (tied with Iceland and Luxembourg), whereas 16th in the second ranking (thus, 14 positions below Iceland) and 12th in the third.

Finally, on January 17th, 2022 (the fourth date we consider, Table 4), we obtain striking differences. For instance, Iceland is also at the top of the first ranking, but moves all the way down to the 24th position in the other two rankings. Somewhat similarly, Israel is 6th at the first ranking and 28th in each of the other two, and Denmark is 12th at the first ranking and 29th (last one) in each of the other two. A completely opposite behavior is shown, for instance, by Bulgaria (ranking at the very bottom with the first indicator, whereas being 6th and 8th, respectively, with the other two), Slovakia (27th in the first ranking, whereas being 4th and 6th, respectively, with the other two) and Poland (24th in the first ranking, whereas being 2nd with the other two). Slovenia is, essentially, at the bottom of all of them (Bulgaria is slightly worse in the third ranking). Finland and Ireland are consistently at the top of the rankings (with some small differences). As for the bottom of the rankings, beyond the cases already highlighted above, countries such as France, Portugal and Italy appear therein for some of them. The United Kingdom performs now somewhat poorly in the first ranking (18th) but much better in the other two rankings (7th). Notable differences also occur, for instance, for Canada, which appears 17th in the first ranking whereas first in the other two. On the other hand, The Netherlands appears 2nd in the first ranking whereas 17th in the second ranking, and 16th in the third.

We can also derive dominance relations from our empirical application. For instance, if we focus on the last date (January 17th, 2022) the UK dominates the US in the three dimensions. Namely, the UK exhibits lower incidence, morbidity, and mortality rates than the US.11 Thus, each and every indicator within the family characterized in Theorem 1 will rank the UK above the US (as it indeed happens with , and , as shown in the last two lines of Table 4). If we take, for instance, Israel and the US, dominance does not occur. As shown in Table 4, Israel is ranked much better than the US with , but much worse with the other two indicators. This is a consequence of the fact that the mortality rate (as of January 17th, 2022) is much lower in Israel than in the US, whereas the incidence rate is much lower in the US than in Israel.12 Thus, one cannot expect that all indicators within our family rank both countries in the same way.

Fig. 1, Fig. 2 go beyond by showing the evolution of the three indicators for a selected group of representative countries. In Fig. 1 we gather the countries that had the last two indicators sky-rocketing in the most recent dates (during the aftermath of the Omicron variant explosion, with infected cases at all-time record high worldwide). This pattern is also shared, although to a lower extent, in the countries gathered at Fig. 2.

Fig. 1

Evolution of the indicators for a group of representative countries.

Fig. 2

Evolution of the indicators for a group of representative countries.

We observe from Fig. 1, Fig. 2 that some countries essentially replicate the performance with the three indicators. This is, for instance, the case of Czechia (with, maybe, the exception of the peak in early 2021, that is more drastically shown with and than with , as well as the current trends of and , not reflected yet by ). For other countries, differences are obvious. For instance, in the case of Ireland (and, to some extent, United Kingdom too) the curves are rather different. In particular, the first peak is much more evident with than with and . The opposite occurs for the other peaks. As for the trends, although the intervals at which the three curves increase and decrease are very similar for most of the countries, there are some exceptions too. For instance, in the case of France, decreases from December 2020 to April 2021, whereas and increase in the same period. Somewhat similarly, most countries show and increasing in recent dates, but some of them show decreasing.

Another common feature of all countries is that death rates reached their maximum in earlier dates (in spite of the record high incidence in more recent dates). Countries such as Belgium, France or Ireland reached it in the first wave; others such as Portugal, Slovenia, the UK and the US did it in the second wave. Only a country such as Denmark seems to be approaching now its peak (so far achieved in the second wave). This feature might be reflecting (among other things) the high vaccination rates in all these countries as of early 2022 (in contrast to what happened in earlier dates, when previous waves occurred).

Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7, Fig. 8, Fig. 9, Fig. 10, Fig. 11 provide scatterplots based on the pairwise combinations of the three indicators. Fig. 3, Fig. 4, Fig. 5 focus on the first date we consider (March 9th, 2020), whereas Fig. 6, Fig. 7, Fig. 8 focus on the last date we consider (January 17th, 2022). They each provide a direct visual ranking of the countries’ performance and also visualize the correlation (positive or negative, depending on the case) between each pair of indicators. Figs. 9 to 11 provide the evolution of the correlation between each pair of indicators within the time window we consider.

Fig. 3

Correlation between and on March 9th, 2020.

Fig. 4

Correlation between and on March 9th, 2020.

Fig. 5

Correlation between and on March 9th, 2020.

Fig. 6

Correlation between and on January 17th, 2022.

Fig. 7

Correlation between and on January 17th, 2022.

Fig. 8

Correlation between and on January 17th, 2022.

Fig. 9

Evolution of the correlation between and .

Fig. 10

Evolution of the correlation between and .

Fig. 11

Evolution of the correlation between and .

As argued in the introduction, we took a multiple countries flow approach for our analysis. This has obvious implications for the empirical application just presented, as well as limitations. In particular, we do not fully address the dynamics of the pandemic as our indicators are static. Nevertheless, we can indeed obtain some dynamic features. In general, deaths follow hospitalizations and hospitalizations follow cases. Shifts over time in our characterized indicators are therefore to be expected. In order to capture this, we have actually exhibited the changes in the ranking of the countries (obtained from the indicators) over time. For instance, as mentioned above, the most recent wave (driven by the Omicron variant) is highlighting striking differences between the last two indicators and the first one. It might be possible that the wave is not being reflected in death records yet. But it also seems safe to argue that, typically, death levels will not be comparable to those in previous waves thanks to the high rates of vaccinated population in most of the countries in our sample.

Finally, we acknowledge that the performance of countries does not only reflect different policies, but also different background conditions (such as the age structure of the population, population density, or the ease with which frontiers can be closed, to name a few). We have not included those conditions in our application because it is far from obvious to select among them.

Discussion

We have provided in this paper normative foundations for popular indicators of pandemic performance. Our main result shows that four axioms, formalizing principles of impartiality, monotonicity, decomposability and meaningful bounds, characterize a family of indicators arising from weighted linear combinations of three focal indicators; namely, incidence rate, morbidity rate, and mortality rate. Those focal indicators can be characterized themselves, upon replacing one of the axioms from our list by another (stronger) independence axiom. We also characterize the counterpart indicators arising from shifting the focus from relative to absolute terms, upon replacing the decomposability axiom by a composition axiom. Epidemiologists have looked at more sophisticated indicators, such as the positivity rate or the excess mortality. It is left for further research to obtain normative foundations for those indicators too.

The axioms we consider in our analysis have a long tradition of use within normative economics. We also believe they have simple and intuitive interpretations (although, obviously, some are more controversial than others). We refer to a wide variety of experimental contributions testing empirically some of the principles (or related ideas) over which our axioms rely in related contexts (e.g., Kobus and Milos, 2012; Turpcu et al., 2012). As mentioned above, our main result characterizes a weighted sum of the incidence rate, morbidity rate, and mortality rate. Recently, Mitra and Ozbek (2021) have precisely provided an axiomatic analysis of the weighted sum criterion using a general choice framework.13 In their setting, the primitive of the analysis is the preference order (i.e., a complete transitive binary relation) of a decision maker over a set of alternatives, where each alternative represents a vector in the finite dimensional Euclidean space . If one considers the set of alternatives all admissible vectors representing incidence rate, morbidity rate, and mortality rate, respectively, then our family of indicators characterized in Theorem 1 would be similar to the weighted sum characterized by Mitra and Ozbek (2021). In our analysis, the set of alternatives is, instead, the set of status matrices, which allows for both absolute and relative considerations. An interesting aspect that the two analyses share is that we directly construct the vector of weights from the indicator itself, much in the same way as they construct it from the preference order, which makes them useful for economic applications. Nevertheless, our monotonicity axiom imposes some conditions on the weights we construct, which is not imposed by the counterpart monotonicity axiom they consider.

One way to read our main result is that if one endorses a list of basic axioms for the evaluation of pandemic performance (regarding status matrices) then one must endorse a certain member of the family of indicators arising from a weighted sum of the incidence rate, morbidity rate, and mortality rate. Now, the family is wide enough to accommodate diverse views, depending on the specific weights one considers. Nevertheless, if a country dominates (outperforms) another on all three dimensions (i.e., it shows lower incidence morbidity and mortality rates than the other), then each indicator from our family (i.e., for all possible weights) will yield a lower value to the former country than to the latter. If dominance does not occur, then we might be able to identify the subset of indicators within our family (i.e., the possible weights) for which one country always obtains a lower value than another within a given pair.

We have illustrated our theoretical analysis applying some of the characterized indicators to real data on COVID-19. The main lesson we can derive from this illustration is seemingly obvious: the choice of indicator matters! Not only quantitatively, but also qualitatively.14 Countries might perform much better, or much worse, depending on the indicator we use. Some others have a more consistent behavior with respect to all the indicators we consider.

To conclude, we acknowledge that our analysis has only focused on health outcomes to obtain the performance of countries with respect to the pandemic. One might argue that our stylized model (whose prior is the so-called status matrices) is simpler than desired to properly address the well-being consequences of a pandemic. For instance, there might be compelling reasons to include inequality aspects (between socioeconomic groups and/or age groups), which have played an important role in the policy debate in many countries. It is left for further research to extend our analysis to include economic outcomes too, even possibly accounting for the restrictions (light, moderate or severe) that they have instituted on commerce and social interactions. Efforts along these lines have already been made in the media.15 Understanding the extent to which health and economic goals are mutually reinforcing, and the extent to which they conflict, seems to be crucial nowadays (e.g., Jamison et al., 2020).