Structural identifiability and observability of compartmental models of the COVID-19 pandemic.

The recent coronavirus disease (COVID-19) outbreak has dramatically increased the public awareness and appreciation of the utility of dynamic models. At the same time, the dissemination of contradictory model predictions has highlighted their limitations. If some parameters and/or state variables of a model cannot be determined from output measurements, its ability to yield correct insights - as well as the possibility of controlling the system - may be compromised. Epidemic dynamics are commonly analysed using compartmental models, and many variations of such models have been used for analysing and predicting the evolution of the COVID-19 pandemic. In this paper we survey the different models proposed in the literature, assembling a list of 36 model structures and assessing their ability to provide reliable information. We address the problem using the control theoretic concepts of structural identifiability and observability. Since some parameters can vary during the course of an epidemic, we consider both the constant and time-varying parameter assumptions. We analyse the structural identifiability and observability of all of the models, considering all plausible choices of outputs and time-varying parameters, which leads us to analyse 255 different model versions. We classify the models according to their structural identifiability and observability under the different assumptions and discuss the implications of the results. We also illustrate with an example several alternative ways of remedying the lack of observability of a model. Our analyses provide guidelines for choosing the most informative model for each purpose, taking into account the available knowledge and measurements.

Introduction

The current coronavirus disease (COVID-19) pandemic, caused by the SARS-CoV-2 virus, continues to wreak unparallelled havoc across the world. Public health authorities can use mathematical models to answer critical questions related with the dynamics of an epidemic (severity and time course of infected people), its impact on the healthcare system, and the design and effectiveness of different interventions (Adam, 2020, Currie et al., 2020, Li, 2018, Lofgren et al., 2014). Mathematical modelling of infectious diseases has a long history (Brauer et al., 2008, Martcheva, 2015). Modelling efforts are particularly important in the context of COVID-19 because its dynamics can be particularly complex and counter-intuitive due to the uncertainty in the transmission mechanisms, possible seasonal variation in both susceptibility and transmission, and their variation within subpopulations (Cobey, 2020). The media has given extensive coverage to analyses and forecasts using COVID-19 models, with increased attention to cases of conflicting conclusions, giving the impression that epidemiological models are unreliable or flawed. However, a closer look reveals that these modelling studies were following different approaches, handling uncertainty differently, and ultimately addressing different questions on different time-scales (Holmdahl & Buckee, 2020).

Broadly speaking, data-driven models (using statistical regression or machine learning) can be used for short-term forecasts (one or a few weeks). Mechanistic models based on assumptions about transmission and immunity try to mimic how the virus spreads, and can be used to formalize current knowledge and explore long-term outcomes of the pandemic and the effectiveness of different interventions. However, the accuracy of mechanistic models is constrained by the uncertainties in our knowledge, which creates uncertainties in model parameters and even in the model structure (Holmdahl & Buckee, 2020). Further, the uncertainty in the COVID-19 data and the exponential spread of the virus amplify the uncertainty in the predictions. Predictability studies (Scarpino & Petri, 2019) seek the characterization of the fundamental limits to outbreak prediction and their impact on decision-making. Despite the vast literature on mathematical epidemiology in general, and modelling of COVID-19 in particular, comparatively few authors have considered the predictability of infectious disease outbreaks (Castro et al., 2020, Scarpino and Petri, 2019). Uncertainty quantification (Arriola & Hyman, 2009) is an interconnected concept that is also key for the reliability of a model, and that has received similarly scant attention (Faranda et al., 2020, Raimúndez et al., 2020).

In addition to predictability and uncertainty quantification approaches, identifiability is a related property whose absence can severely limit the usefulness of a mechanistic model (Chowell, 2017). A model is identifiable if we can determine the values of its parameters from knowledge of its inputs and outputs. Likewise, the related control theoretic property of observability describes if we can infer the model states from knowledge of its inputs and outputs. If a model is non-identifiable (or non-observable) different sets of parameters (or states) can produce the same predictions or fit to data. The implications can be enormous: in the context of the COVID-19 outbreak in Wuhan, non-identifiability in model calibrations was identified as the main reason for wide variations in model predictions (Roda, Varughese, Han, & Li, 2020).

Reliable models can be used in combination with optimization and optimal control methods to find the best intervention strategies, such as lock-downs with minimum economic impact (Acemoglu et al., 2020, Alvarez et al., 2020). Further, they can be used to explore the feasibility of model-based real-time control of the pandemic (Casella, 2020, Köhler et al., 2020). However, using calibrated models with non-identifiability or non-observability issues can result in bad or even dangerous intervention and control strategies.

It is common to distinguish between structural and practical identifiability. Structural non-identifiability may be due to the model and measurement (input–output) structure. Practical non-identifiability is due to lack of information in the considered data-sets. Non-identifiability results in incorrect parameter estimates and bad uncertainty quantification (Chowell, 2017, Kao and Eisenberg, 2018), i.e. a misleading calibrated model which should not be used to analyse epidemiological data, test hypothesis, or design interventions. The structural identifiability of several epidemic mechanistic models has been studied e.g. in Brouwer et al., 2019, Chapman and Evans, 2009, Eisenberg et al., 2013, Evans et al., 2002, Evans et al., 2005, Prague et al., 2020. Other recent studies have mostly focused on practical identifiability, such as Alahmadi et al., 2020, Chowell, 2017, Kao and Eisenberg, 2018, Roosa and Chowell, 2019, Tuncer and Le, 2018.

Since checking structural identifiability can be hard, one might think that directly checking practical identifiability could be a better alternative. However, it has been shown Janzén et al. (2016) that this practice entails at least two dangers: (i) checking practical identifiability can exhibit numerical issues that lead to wrong results, and (ii) if structural identifiability is unknown, the source of uncertainty in the parameter estimates can be either the data or the model structure, so it will not be clear how to improve the situation. In contrast, if we can verify structural identifiability, we can then focus on the data uncertainty to resolve possible practical identifiability issues. In other words, practical identifiability analysis cannot be used to deduce structural identifiability, and ignoring the latter can lead to modelling artefacts.

In this paper we assess the structural identifiability and observability of a large set of COVID-19 mechanistic models described by deterministic ordinary differential equations, derived by different authors using the compartmental modelling framework (Brauer, 2008). Compartmental models are widely used in epidemiology because they are tractable and powerful despite their simplicity. We collect 36 different compartmental models, of which we consider several variations, making up a total of 255 different model versions. Our aim is to characterize their ability to provide insights about their unknown parameters – i.e. their structural identifiability – and unmeasured states – i.e. their observability. To this end we adopt a differential geometry approach that considers structural identifiability as a particular case of nonlinear observability, allowing to analyse both properties jointly. We define the relevant concepts and describe the methods used in Section 2. Then we provide an overview of the different types of compartmental models found in the literature in Section 3. We analyse their structural identifiability and observability and discuss the results in Section 4, where we also show different ways of remedying lack of observability using an illustrative model. Finally, we conclude our study with some key remarks in Section 5.

Methods

Notation, models, and properties

We consider models defined by systems of ordinary differential equations with the following notation: where and are analytical (generally nonlinear) functions of the states , known inputs , unknown constant parameters , and unknown inputs or time-varying parameters . The output represents the measurable functions of model variables. The expressions (1)a–(1)b are sufficiently general to represent a wide range of model structures, of which compartmental models are a particular case.

Structurally Locally Identifiable (DiStefano III, 2015)

A parameter of model is structurally locally identifiable (s.l.i.) if for almost any parameter vector there is a neighbourhood in which the following relationship holds: Otherwise, is structurally unidentifiable (s.u.). If all model parameters are s.l.i. the model is s.l.i. If there is at least one s.u. parameter, the model is s.u.

Likewise, a state is observable if it can be distinguished from any other states in a neighbourhood from observations of the model output and input in the interval , for a finite . Otherwise, is unobservable. A model is called observable if all its states are observable. We also say that is invertible if it is possible to infer its unknown inputs , and we say that is reconstructible in this case.

Structural identifiability can be seen as a particular case of observability (Sedoglavic, 2002, Tunali and Tarn, 1987, Villaverde, 2019), by augmenting the state vector with the unknown parameters , which are now considered as state variables with zero dynamics, . The reconstructibility of unknown inputs , which is also known as input observability, can also be cast in a similar way, although in this case their derivatives may be nonzero. To this end, let us augment the state vector further with as additional states, as well as their derivatives up to some non-negative integer : The augmented dynamics is: leading to the augmented system:

Unknown Inputs, Disturbances, or Time-Varying Parameters

In Section 4, when reporting the results of the structural identifiability and observability analyses, we will explicitly consider some parameters as time-varying. In the model structure defined in Eqs. (1)a–(1)b the unknown parameter vector is assumed to be constant. To consider an unknown parameter as time-varying we include it in the “unknown input” vector . Thus, changing the consideration of a parameter from constant to time-varying entails removing it from and including it in . The elements of can be interpreted as unmeasured disturbances or inputs of unknown magnitude or, equivalently, as time-varying parameters. Regardless of the interpretation, they are assumed to change smoothly, i.e. they are infinitely differentiable functions of time. For the analysis of some models it is necessary, or at least convenient, to introduce the mild assumption that the derivatives of vanish for a certain non-negative integer (possibly , i.e.  and for all . This assumption is equivalent to assuming that the disturbances are polynomial functions of time, with maximum degree equal to  (Villaverde, Tsiantis, & Banga, 2019).

Full Input-State-Parameter Observability, FISPO (Villaverde et al., 2019)

Let us consider a model given by (1)a–(1)b. We augment its state vector as (3), which leads to its augmented form (4). We say that has the FISPO property if, for every , every model unknown can be inferred from and in a finite time interval . Thus, is FISPO if, for every and for almost any vector , there is a neighbourhood such that, for all , the following property is fulfilled:

Structural identifiability and observability analysis

In this paper we analyse input, state, and parameter observability – that is, the FISPO property defined above – using a differential geometry framework. Such analyses are structural and local. By structural we refer to properties that are entirely determined by the model equations; thus we do not consider possible deficiencies due to insufficient or noise-corrupted data. By local we refer to the ability to distinguish between neighbouring states (similarly, parameters or unmeasured inputs), even though they may not be distinguishable from other distant states. This is usually sufficient, since in most (although not all, see e.g. Thomaseth and Saccomani (2018)) applications local observability entails global observability. This specific type of observability has sometimes been called local weak observability (Hermann & Krener, 1977).

This approach assesses structural identifiability and observability by calculating the rank of a matrix that is constructed with Lie derivatives. The corresponding definitions are as follows (in the remainder of this section we omit the dependency on time to simplify the notation):

Extended Lie Derivative (Karlsson, Anguelova, & Jirstrand, 2012)

Consider the system (1)a–(1)b with augmented state vector (3) and augmented dynamics (4). Assuming that the inputs are analytical functions, the extended Lie derivative of the output along is: The zero-order derivative is , and the order extended Lie derivatives can be recursively calculated as:

Observability–identifiability Matrix (Villaverde et al., 2019)

The Observability–identifiability matrix of the system (1)a–(1)b with augmented state vector (3), augmented dynamics (4), and analytical inputs is the following matrix,

The FISPO property of can be analysed by calculating the rank of the Observability–identifiability matrix:

Observability–identifiability Condition, OIC (Karlsson et al., 2012)

If the identifiability–observability matrix of a model satisfies , with being a (possibly generic) point in the augmented state space, then the system is structurally locally observable and structurally locally identifiable.

Analysis tools

In this paper we generally check the OIC criterion of 1 using STRIKE-GOLDD, an open source MATLAB toolbox (Villaverde, Barreiro, & Papachristodoulou, 2016). Alternatively, for some models we use the Maple code ObservabilityTest, which implements a procedure that avoids the symbolic calculation of the Lie derivatives and is hence computationally efficient (Sedoglavic, 2002). A number of other software tools are available, including GenSSI2 (Ligon, Fröhlich, Chiş, Banga, Balsa-Canto, & Hasenauer, 2018) in MATLAB, IdentifiabilityAnalysis in Mathematica (Karlsson et al., 2012), DAISY in REDUCE (Saccomani, Bellu, Audoly, & d’Angio, 2019), SIAN in Maple (Hong, Ovchinnikov, Pogudin, & Yap, 2019), and the web app COMBOS (Meshkat, Kuo, & DiStefano III, 2014). It should be taken into account that in the present work we are interested in assessing structural identifiability and observability both with constant and continuous time-varying model parameters (or equivalently, with unknown inputs), as explained in Remark 1. Ideally, the method of choice should provide a convenient way of analysing models with this type of parameters (inputs). It is always possible to perform this type of analysis by assuming that the time dependency of the parameters is of a particular form, e.g. a polynomial function of a certain maximum degree.

Models

In this article we review compartmental models, which are one of the most widely used families of models in epidemiology. They divide the population into homogeneous compartments, each of which corresponds to a state variable that quantifies the number of individuals that are at a certain disease stage. The dynamics of these compartments are governed by ordinary differential equations, usually with unknown parameters that describe the rates at which individuals move among different stages of disease. The total size of the population is known and commonly denoted as  (Roosa & Chowell, 2019). The basic compartmental model used for describing a transmission disease is the SIR model, in which the population is divided into three classes:

Susceptible: individuals who have no immunity and may become infected if exposed.

Infected and infectious: an exposed individual becomes infected after contracting the disease. Since an infected individual has the ability to transmit the disease, he/she is also infectious.

Recovered: individuals who are immune to the disease and do not affect its transmission.

Another class of models, called SEIR, include an additional compartment to account for the existence of a latent period after the transmission:

Exposed: individuals vulnerable to contracting the disease when they come into contact with it.

These idealized models differ from the reality. Contact tracing, screening, or changes in habits are some differences that are not considered in basic SIR or SEIR models, but are important for evaluating the effects of an intervention. Furthermore, it is not only important to enrich the information about the behaviour of the population; the characteristics of the disease must also be taken into account. These additional details can be incorporated to the model as new parameters, functions, or extra compartments. Compartments such as asymptomatic, quarantined, isolated, and hospitalized have been widely used in COVID-19 models. From 29 articles, most of which are very recent (Castro et al., 2020, Chatterjee et al., 2020, Dohare, 2020, Eikenberry et al., 2020, Fosu et al., 2020, Franco, 2020, Gaeta, 2020, Gevertz et al., 2020, Giordano et al., 2020, Hubbs, 2020, Jia et al., 2020, Liangrong et al., 2020, Lopez and Rodo, 2020, Lourenco et al., 2020, Maier and Brockmann, 2020, McGee, 2020, Pribylova and Hajnova, 2020, Rahman and Kuddus, 2020, Raimúndez et al., 2020, Roda et al., 2020, Sameni, 2020, Shi et al., 2020, Zha et al., 2020, Zheng, 2020), we have collected 36 models. Depending on whether they have an exposed compartment or not, they can be broadly classified as belonging to the SIR or SEIR families. However, most of these models include additional compartments.

SIR models

Susceptible individuals become infected with an incidence of: where is the transmission rate, is the contact rate and the probability that a contact with a susceptible individual results in a transmission (Martcheva, 2015). Individuals who recover leave the infectious class at rate , where is the average infectious period. The set of differential equations describing the basic SIR model is given by:

As mentioned above, compartmental models can be extended to consider further details. We have found models that incorporate the following features: asymptomatic individuals, births and deaths, delay-time, lock-down, quarantine, isolation, social distancing, and screening. Fig. 1 shows a classification of the SIR models reviewed in this article, and Table 1 lists them along with their equations. Multiple output choices have been considered in the study of the structural identifiability and observability of some models. In such cases the observations are listed in the Output column.

Fig. 1

Classification of SIR models. Each block represents a model structure. The basic, three-compartment SIR model structure is on top of the tree. Every additional block is labelled with the additional feature that it contains with respect to its parent block. The darkness of the shade indicates the number of additional features with respect to the basic SIR model.

Table 1

List of SIR models and their main features.

ID	Ref.	States	Parameters	Output	ICS	Input	Equations
6	Zheng (2020)	S, I, R	β, γ, K	(1)I(2)KI	1000001000		S˙(t)=−βSI∕NI˙(t)=βSI∕N−γIR˙(t)=γI
7	Zheng (2020)	S, I, R, Q	β, γ, δ	(1)Q(2)I,R,Q	10001000		S˙(t)=−βSI∕NI˙(t)=βSI∕N−γI−δIR˙(t)=γIQ˙(t)=δI
13	Maier and Brockmann (2020)	S, I, R, X	β, α, k, k_0, C(t_0)	NX	1−I(t_0)−X(t_0)I_0C(t_0)∕(X_0N)C(t_0)∕NR_0		S˙(t)=−αSI−k_0SI˙(t)=αSI−(β+k+k_0)IX˙(t)=(k+k_0)IR˙(t)=βI+k_0S
15	Roda et al. (2020)	S, I, R	β, ρ, μ, τ, d, I_0	ρI+τ	S_0I_0R_0		S˙(t)=−βSII˙(t)=βSI−(ρ+μ)IR˙(t)=ρI−dR
20	Giordano et al. (2020)	S, I, D, A, R, T, H, E	α, β, ρ, μ, τϵ, η, ξ, λ, σ, κ, θ, ν, γ, δ, ζ	D, R, T	S_0200∕60e820∕60e81∕60e82∕60e8000		S˙(t)=−S(αI+βD+γA+δR)I˙(t)=S(αI+βD+γA+δR)−(ϵ+ζ+λ)ID˙(t)=ϵI−(η+ρ)DA˙(t)=ζI−(θ+μ+κ)AR˙(t)=ηD+θA−(ν+ξ)RT˙(t)=μA+νR−(σ+τ)TH˙(t)=λI+ρD+κA+ξR+σTE˙(t)=τT
21	Castro et al. (2020)	S, I, D, C, R	p, q, r β, μ	(1)I(2)C(3)C,D			S˙(t)=−βSI∕N−pS+qCI˙(t)=βSI∕N−(r+μ)IR˙(t)=rIC˙(t)=pS−qCD˙(t)=μI
19	Gaeta (2020)	S, I, J, R, U	β, α, η, ξ	(1)R(2)I,R	S_0I_0(1−ξ)I_0∕ξR_0U_0		S˙(t)=−α(I+J)SI˙(t)=αξS(I+J)−βIJ˙(t)=α(1−ξ)S(I+J)−ηJR˙(t)=βIU˙(t)=ηJ
24	Kim and Milner (1995)	S, I, R	μ, γ, ϕc, K	KI		σ(t)	S˙(t)=Λ−μS−cϕSIS+II˙(t)=−μI+cϕSIS+I−γI−Iσ(S+I)S+IR˙(t)=−μR+γI+Iσ(S+I)S+I
25	Lourenco et al. (2020)	y, z, A	β, σ, ρ, θ	A	y_0z_0A_0	z_d	y˙(t)=βy(1−z)−σyz˙(t)=βy(1−z)A˙(t)=Nρθz_d
26	Gallina (2012)	S, I, R, A, Q, J	μ_1, μ_2, d_1, d_2, d_3, d_4, d_5, d_6, k_1, k_2, λ, γ_1, γ_2, ϵ_a, ϵ_q, ϵ_j	Q, J			S˙(t)=bN−S(Iλ+λQϵ_aϵ_q+λϵ_aA+λϵ_jJ+d_1)I˙(t)=k_1A−(γ_1+μ_2+d_2)IR˙(t)=γ_1I+γ_2J−d_3RA˙(t)=S(Iλ+λQϵ_aϵ_q+λϵ_aA+λϵ_jJ)−(k_1+μ_1+d_4)AQ˙(t)=μ_1A−(k_2+d_5)QJ˙(t)=k_2Q+μ_2I−(γ_2+d_6)J
22	Hethcote, Zhien, and Shengbing (2002)	S, I, Q, R	d, ϵ, βγ, δ, α_1, α_2	Q			S˙(t)=A−βSI−dSI˙(t)=βSI−I(γ+d+δ+α_1)Q˙(t)=δI−(ϵ+d+α_2)QR˙(t)=γI+ϵQ−R
27	Raimúndez et al. (2020)	S, A, I, R, D	γ, δ, k, ζ_0, β_0	(1)I,R(2)R,D(3)I,R,D	S_0A_0I_0R_0D_0	g(t)	S˙(t)=−β_0gSI∕N−ζ_0gSA∕NA˙(t)=β_0gSI∕N+ζ_0gSA∕N−kAI˙(t)=kA−(γ+δ)IR˙(t)=γID˙(t)=δI
28	Raimúndez et al. (2020)	S, A, I, R, RR, D	γ, δ, k, ν, ζ_0, β_0	(1)I,R(2)R,D(3)I,R,D	S_0A_0I_0R_0RR_0D_0	g(t)	S˙(t)=−β_0gSI∕N−ζ_0gSA∕NA˙(t)=β_0gSI∕N+ζ_0gSA∕N−kAI˙(t)=kA−(γ+δ)IR˙(t)=γIRR˙(t)=νAD˙(t)=δI
29	Franco (2020)	s, i	R_0	i	N−2∕e62∕e6		s˙(t)=−R_0sii˙(t)=(R_0s−1)i
30	Franco (2020)	s, i	R_0, κ	i	N−2∕e62∕e6		s˙(t)=−R_0si1+κii˙(t)=R_0s1+κi−1i
35	Fosu, Opong, and Appati (2020)	S, L, I, Q, R	γ, β, η, δ, θ_1, α_1, α_2	Q, L			S˙(t)=μN−βSI−(γ+η)S+δLL˙(t)=ηS−(γ+δ)LI˙(t)=βSI−(γ+θ_1+α_1)IQ˙(t)=θ_1I−(γ+α_2)QR˙(t)=α_1I+α_2Q−γR
37	Gevertz, Greene, Hixahuary Sanchez Tapia, and Sontag (2020)	Sd, Sn, Ad, An, I, R	ϵ_s, ϵ_i, f, h_1, h_2, β_i, δ, β_a, γ_ai, γ_ir	Sd, I	01−10^−50010^−50		Sd˙(t)=−ϵ_sβ_a(A_n+ϵ_aA_d)S_d−h_1S_d+h_2S_n−ϵ_sβ_iS_dISn˙(t)=−β_iS_nI−β_a(A_n+ϵ_aA_d)S_n+h_1S_d−h_2S_nAd˙(t)=ϵ_sβ_iS_dI+ϵ_sβ_a(A_n+ϵ_aA_d)S_d+h_2A_n−γ_aiA_d−h_1A_dAn˙(t)=β_iS_nI+β_a(A_n+ϵ_aA_d)S_n+h_1A_d−γ_aiA_n−h_2A_nI˙(t)=fγ_ai(A_d+A_n)−δI−γ_irIR˙(t)=(1−f)γ_ai(A_d+A_n)+γ_irI

SEIR models

Individuals in the SEIR model are divided in four compartments: Susceptible (S), Exposed (E), Infected (I) and Recovered (R). Compared to the SIR models, the additional compartment allows for a more accurate description of diseases in which the incubation period and the latent period do not coincide, i.e. the period between which an infected becomes infectious. This is why SEIR models are in principle best suited to epidemics with a long incubation period such as COVID-19 (Franco, 2020).

Susceptible individuals move to the exposed class at a rate , where is the transmission rate parameter. Exposed individuals become infected at rate , where is the average latent period. Infected individuals recover at rate , where is the average infectious period.

Thus, the set of differential equations describing the basic SEIR model is:

Existing extensions of SEIR models may incorporate some of the following features: asymptomatic individuals, births and deaths, hospitalization, quarantine, isolation, social distancing, screening and lock-down. Fig. 2 shows a classification of the models found in the literature; Table 2 lists them along with their equations.

Fig. 2

Classification of SEIR models. Each block represents a model structure. The basic, four-compartment SEIR model structure is on top of the tree. Every additional block is labelled with the additional feature that it contains with respect to its parent block. The darkness of the shade indicates the number of additional features with respect to the basic SEIR model.

Table 2

List of SEIR models and their main features.

ID	Ref.	States	Parameters	Output	ICS	Input	Equations
2	Roosa and Chowell (2019)	S, E, I, R	β, γ, k, K	(1)C(2)KC			S˙(t)=−βSI∕NE˙(t)=βSI∕N−kEI˙(t)=kE−γIR˙(t)=γIC˙(t)=kE
34	Chitnis (2017)	S, E, I, R	β, γ, μ, ϵ, r, K	KI			S˙(t)=Λ−rβSI∕N−μSE˙(t)=βSI∕N−ϵE−μEI˙(t)=ϵE−γI−μIR˙(t)=γI−μR
16	Roda et al. (2020)	S, E, I, R	β, ρ, μ, d, ϵ, τ, E_0, I_0	(1)ρI+τ(2)μI	S_0E_0I_0R_0		S˙(t)=−βSIE˙(t)=βSI−ϵEI˙(t)=ϵE−(ρ+μ)IR˙(t)=ρI−dR
51	McGee (2020)	S, E, I, De, Di, R, F	q, μ_i, μ_d, σ, σ_d, θ_e, θ_i, γ, γ_d, β, β_d, ϕ_i, ϕ_e, μ_0	(1)De,Di,F(2)Di,F			S˙(t)=−βSI∕N−qβ_dSDi∕N+νN−μ_0SE˙(t)=βSI∕N+qβ_dSDi∕N−σE−θ_eϕ_eE−μ_0EI˙(t)=σE−γI−μ_iI−θ_iϕ_iI−μ_0IDe˙(t)=θ_eϕ_eE−σ_dDe−μ_0DeDi˙(t)=θ_iϕ_iI+σ_dDe−γ_dDi−μ_dDi−μ_0DiR˙(t)=γI+γ_dDi−μ_0RF˙(t)=μ_iI+μ_dDi
14	Lopez and Rodo (2020)	S, E, I, Q, R, D, C	β, α, γ, δ, τ	C, Q, D		k(t), λ(t)	S˙(t)=μN−αS−βSIN−μSE˙(t)=βSIN−μE−γEI˙(t)=γE−δI−μIμSQ˙(t)=δI−λQ−kQ−μQR˙(t)=λQ−μSD˙(t)=kQC˙(t)=αS−μC−τC
61	Hubbs (2020)	S, E, I, R	p, α, β, γ, K	KI			S˙(t)=−pβSIE˙(t)=pβSI−αEI˙(t)=αE−γIR˙(t)=γI
5	Pribylova and Hajnova (2020)	S, E, I, R, A	μ_1, μ_2, γ, β, p	I			S˙(t)=−βS(I+A)E˙(t)=βS(I+A)−γEI˙(t)=γpE−μ_1IA˙(t)=γ(1−p)E−μ_2AR˙(t)=μ_1I+μ_2A
1	Zha et al. (2020)	S, E, I, R, Q	ϕ, β, γ, w	Q			S˙(t)=−βSIE˙(t)=βSI−wEI˙(t)=wE−ϕI−(1−ϕ)γIR˙(t)=γQ+(1−ϕ)γIQ˙(t)=−γQ+ϕI
3	Liangrong et al. (2020)	S, E, I, R Q, D, P	α, β, γ, δE_0, I_0	Q, R, D	S_0E_0I_0Q_0R_0D_0P_0	λ(t), κ(t)	S˙(t)=−βSI∕N−αSE˙(t)=βSI∕N−γEI˙(t)=γE−δIQ˙(t)=δI−λQ−κQR˙(t)=λQD˙(t)=κQP˙(t)=αS
4	Sameni (2020)	S, E, I, R, P	α_e, α_i, ρ, β, μ, κ, e_0	(1)S+vs(t)(1)I+vi(t)(1)R+vr(t)(1)P+vp(t)(2)P	N−e_0e_0000	vs(t)vi(t)vr(t)vp(t)	S˙(t)=−α_eSE−α_iSIE˙(t)=α_eSE+α_iSI−κE−ρEI˙(t)=κE−βI−μIR˙(t)=βI+ρEP˙(t)=μI
8	Eikenberry et al. (2020)	S, E, I, R A, H, D	δ, α, η, σ, γ_a, γ_i, γ_hΦ, E_0, R_0, A_0H_0, D_0	D, H, I	1e6E_050A_0H_0R_0D_0	β(t)	S˙(t)=−β(I+ηA)S∕NE˙(t)=β(I+ηA)S∕N−σEI˙(t)=ασE−ΦI−γ_iIA˙(t)=(1−α)σE−γ_aAH˙(t)=ΦI−δH−γ_hHR˙(t)=γ_iI+γ_aA+γ_hHD˙(t)=δH
38	Shi et al. (2020)	S, E, I, Sq, Eq, H, R	c, q, λ, βδ_i, δ_q, α, γ_iγ_h, θ	(1)I,R(2)H,viI,vrR(3)Sq,Eq		σ(t)	S˙(t)=−(cβ+cq(1−β))S(I+θE)+λSqE˙(t)=cβ(1−q)S(I+θE)−σEI˙(t)=σE−(δ_i+α+γ_i)ISq˙(t)=cq(1−β)S(I+θE)−λSqEq˙(t)=cqβS(I+θE)−δ_qEqH˙(t)=δ_iI+δ_qEq−(α+γ_h)HR˙(t)=γ_iI+γ_hH
41	Roosa and Chowell (2019)	S, E, A, I, J, R, C	β, k, γ_1γ_2, α, ρ, q	(1)C(2)J,I			S˙(t)=−βS(I+J+qA)∕NE˙(t)=βS(I+J+qA)∕N−kEA˙(t)=k(1−ρ)E−γ_1AI˙(t)=kρE−(α+γ_1)IJ˙(t)=αI−γ_2JR˙(t)=γ_1(A+I)+γ_2JC˙(t)=αI
39	McGee (2020)	S, E, I, De, Di, R, F	q, μ_i, μ_d, σ, σ_d, θ_e, θ_i, γ, γ_d, β, β_d, ϕ_i, ϕ_e	(1)F,Di(2)F,De,Di			S˙(t)=−βSI∕N−qβ_dSDi∕NE˙(t)=βSI∕N+qβ_dSDi∕N−σE−θ_eϕ_eEI˙(t)=σE−γI−μ_iI−θ_iϕ_iIDe˙(t)=θ_eϕ_eE−σ_dDeDi˙(t)=θ_iϕ_iI+σ_dDe−γ_dDi−μ_dDiR˙(t)=γI+γ_dDiF˙(t)=μ_iI+μ_dDi
17	Chatterjee et al. (2020)	S, E, I, R, Q, D	β, ϵ, γ, d, q, qt	(1)D(2)Q,D(3)D,I,Q	249E_0qQ_0Q_0235		S˙(t)=−βSIE˙(t)=βSI−ϵEI˙(t)=ϵE−γI−dI−qIQ˙(t)=qI−qtQ−dQR˙(t)=γI+qtQD˙(t)=dI+dQ
18	Jia et al. (2020)	S, E, I, A, R, Q, D	β, θ, λ, σ, ρ, ϵ_a, ϵ_i, γ_i, γ_a, γ_d, p, d_i, d_d	(1)D,I,Q(2)Q,D(3)D	9219849e34142251e232075955632273		S˙(t)=−βS(I+θA)−pS+λQQ˙(t)=pS−λQE˙(t)=βS(I+θA)−σEA˙(t)=σ(1−ρ)E−ϵ_aA−γ_aAI˙(t)=σρE−γ_iI−d_iI−ϵ_iID˙(t)=ϵ_aA+ϵ_iI−d_dD−γ_dDR˙(t)=γ_aA+γ_iI+γ_dD
31	Dohare (2020)	S, E, I, A, J, R	α, σ, h, r, q, f, β_1, β_2, ϕ, γ, I_0	(1)I,J(2)I	0.9N9(I_0+A_0)I_0I_0f00		S˙(t)=−αE+I+ANS−σSE˙(t)=αE+I+ANS−β_1EI˙(t)=β_1hE+β_2rA−ϕqI−γ(1−q)IA˙(t)=β_1(1−h)E−β_2rA−γ(1−r)AJ˙(t)=ϕqI−γJR˙(t)=γ(1−q)I+γ(1−r)A+γJ
32	Rahman and Kuddus (2020)	S, L, I, R	w, β, α, γ, μ	wL			S˙(t)=A−βSI1+αI−μSL˙(t)=βSI1+αI−(w+μ)LI˙(t)=wL−(γ+μ)IR˙(t)=γI−μR
33	Fosu et al. (2020)	S, L, E, I, Q, R	γ, β_1, η, δ, ξ, θ_2, ϵ, θ_1, α_1, α_2,	L, Q			S˙(t)=μN−β_1SI−(γ+η)S+δL+ξEL˙(t)=ηS−(γ+δ)LE˙(t)=β_1SI−(γ+θ_2+ϵ+ξ)EI˙(t)=ϵE−(γ+θ_1+α_2)IQ˙(t)=θ_1I+θ_2E−(γ+α_2)QR˙(t)=α_1I+α_2Q−γR

Results and discussion

We analysed the structural identifiability and observability of the 17 SIR model structures (a total of 98 model versions considering the different output configurations and time-varying parameter assumptions) and 19 SEIR models (with a total of 157 model versions) listed in Table 1, Table 2. The detailed results for each model are given in Appendix, which reports the structural identifiability of each parame-ter and the observability of each state, for every model version. In the remainder of this section we provide an overview of the main results.

General patterns

The general patterns regarding state observability are as follows. The recovered state (R) is almost never observable unless it is directly measured (D.M.) as output; the only exceptions are two SEIR models, 31 and 38, for which R is observable under the assumption of time-varying parameters. The susceptible state (S), in contrast, is observable in roughly two thirds of the models (SIR: 65/98, SEIR: 103/157); this is also true for the exposed state (E) in the SEIR models. The infected state (I) is included in most studies among the outputs, either directly (D.M.) or indirectly measured (as part of a parameterized measurement function). When it is not considered in this way, its observability is generally similar to that of S (in 18/157 model versions I is not an output and it is observable in 13/18).

The transmission and recovery rates (, ) are the two parameters common to all SIR models. The transmission rate is identifiable in 59/98 model versions, and in 51/98 and its derivatives in 12/98. SEIR models have a third parameter in common, the latent period (). It is identifiable in most of the models (145/157), as well as the recovery rate (111/157). The transmission rate is identifiable in 101/157 model versions, but it is not identifiable in any SEIR model version that accounts for social distancing (numbers 34 and 61); we found no clear pattern in the other models.

The effect of time-varying parameters

The transmission rate , the recovery rate , and in SEIR models the latent period , can vary during an epidemic as a result of changes in the population’s behaviour (Chen et al., 2020, Liangrong et al., 2020), the introduction of new drugs or new medical equipment (Liangrong et al., 2020), or the reduction of the period duration as a result of high temperatures (Li & Zhao, 2019). To account for such variations, the present study has considered both the constant and the time-varying cases, by including the corresponding variables either in the constant parameter vector or in the unknown input vector , respectively, as described in Remark 1. Changing a parameter from constant to time-varying naturally influences structural identifiability and observability. This effect is graphically summarized in Fig. 3, Fig. 4, Fig. 5, Fig. 6, Fig. 7, which represent classes of models in tree form and classify them according to their observability. Each model is shaded with a colour, according to the observability of the parameter studied. Some models include different rates for different population groups: for example, they may consider two different transmission rates for symptomatic and asymptomatic individuals. For those models, each rate may have different observability properties when considered time-varying parameters; in such cases the model is depicted between two colour blocks (see for example the SIR 20 model in Fig. 3).

Fig. 3

Observability of (transmission rate) in SIR models. Models in which is observable are shown in green, and non-observable in red. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 4

Observability of (recovery rate) in SIR models. Models in which is observable are shown in green. Models in which it is unobservable if constant and observable if time-varying are shown in a green–red gradient. Finally, models in which only its time-derivatives are observable are shown in orange. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 5

Observability of (transmission rate) in SEIR models. Models in which is observable are shown in green, and non-observable in red. Models in which it is unobservable if constant and observable if time-varying are shown in a green–red gradient. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 6

Observability of (latent period) in SEIR models. Models in which is observable are shown in green, and non-observable in red. Models in which it is unobservable if constant and observable if time-varying are shown in a green–red gradient. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 7

Observability of (recovery rate) in SEIR models. Models in which is observable are shown in green, and non-observable in red. Models in which it is unobservable if constant and observable if time-varying are shown in a green–red gradient. Finally, models in which only its time-derivatives are observable are shown in orange. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Changing from a constant to a time-varying parameter (or equivalently an unknown input) does not change its observability nor that of the other variables in SIR models. In contrast, this is not the case with the recovery rate , for which a somewhat counter-intuitive result may be obtained: by changing from a constant to a continuous function of time with at least a non-zero derivative, its model can become more observable and identifiable — despite the fact that it is an unknown function. An example of this is the SIR model 15: if is constant the model has only one identifiable parameter, , and no observable states; if is time-varying with at least one non-zero derivative, two parameters become identifiable (, ), two states become observable (I, S), and itself is observable. In the other models, when is not identifiable as a constant nor observable as an unknown input, its successive derivatives are observable.

For the SEIR models, the consideration of the parameter as an unknown input function follows a similar trend to that of the SIR models with the exception of model 38, which gains both observability and identifiability and becomes FISPO. Considering the recovery rate (Fig. 7) or the latent period (Fig. 6) individually as time-varying parameters generally leads to greater observability, except for model 31(1). As an example, in model 39(2) one of the unknown inputs becomes observable, three states become observable (S, E, I), and three parameters become identifiable (, , ); or the 16(2) model, in which both its input and three states (S, E, I) become observable and two parameters (, ) become identifiable.

Besides the transmission rate, latent period, and recovery rate, other rates (screening, disease-related deaths, and isolation) have also been considered as time-varying parameters in some studies. The observability of most models is not modified if these parameters are allowed to change in time; the exception being 8 models which gain observability. An example is the SEIR model 41(1), which has seven parameters, seven states, and one output. Assuming constant parameters, five of them are structurally identifiable (, , , , ) and two are unidentifiable (q, ), while there are three observable states (I, J, C) and four unobservable states (S, E, A, R) (Roosa & Chowell, 2019). However, when the parameter (which describes the proportion of exposed/latent individuals who become clinically infectious) is considered time-varying, all parameters become identifiable (including ) and six states become observable (all except R, which is never observable unless it can be directly measured, as we have already mentioned).

The fact that allowing an unknown quantity to change in time can improve its observability – and also the observability of other variables in a model – may seem paradoxical. An intuitive explanation can be obtained from the study of the symmetry in the model structure. The existence of Lie symmetries amounts to the possibility of transforming parameters and state variables while leaving the output unchanged, i.e. their existence amounts to lack of structural identifiability and/or observability (Yates, Evans, & Chappell, 2009). The STRIKE-GOLDD toolbox used in this paper includes procedures for finding Lie symmetries (Massonis & F. Villaverde, 2020). Let us use the SIR 15 model as an example. This model has five parameters (, , , , d), of which only is identifiable if assumed constant. The model contains the following symmetry: where is the parameter of the Lie group of transformations. Thus, there is a symmetry between and that makes them unidentifiable: changes in one parameter can be compensated by changes in the other one. However, if is time-varying and is constant, the latter cannot compensate the changes of the former, and the symmetry is broken. Indeed, if is considered time-varying the model gains identifiability (not only , but also and become identifiable) and observability (S, I and become observable).

Applying the results in practice: an illustrative example

Let us now illustrate how the results of this study may be applied in a realistic scenario. We use as an example the model SIR 26, which has 6 states (S, I, R, A, Q, J) and 16 parameters (, , , , , , , , , , , , , , , ); its equations are shown in Table 1. This model includes the following additional features with respect to the basic SIR model: birth/death, asymptomatic individuals (A), quarantine (Q), and isolation (J). In its original publication two states were measured (Q, J). With these two states as outputs the model has five identifiable parameters (, , , , ) and two observable states (A, I); thus, there are two unobservable states (S, R) and ten unidentifiable parameters.

If we are interested in estimating e.g. the number of susceptible individuals (S), this model would not be appropriate. How should we proceed in that scenario?

One way of improving observability could be by including more outputs (option 1). For example, since there is a separate class for asymptomatic individuals (A), the infected compartment (I) considers only individuals with symptoms, and we could assume that they can be detected. By including ‘I’ in the output set, the structural identifiability and observability of the model improves: six more parameters are identifiable (, , , , , ) and the state in which we are interested (S) becomes observable.

However, including more outputs is not always realistic. Another possibility would then be to reduce the complexity of the model by decreasing the number of additional features (option 2). For example, leaving out the asymptomatic compartment leads to the following model: The output of the model is the same, Q, J. In this case, the model has eight identifiable parameters (, , , , , , , ) and two observable states (S, I).

A third possibility is to simplify the parameterization of the model (option 3). This model considers a different death rate for every compartment (.). With some loss of generality, we could consider a specific death rate for infected individuals, , and a general death rate for all non-infected and asymptomatic individuals, . This reduction of the number of parameters leads to a better observability to the model: the only unidentifiable parameters are , and , and the only non-observable state is R. Thus, this option also allows to identify S.

Finally, a fourth possible course of action is to reparameterize the model using identifiable combinations. This solution entails reducing the number of parameters involved in the model, with new variables consisting of e.g. the product or quotient of some parameters and/or states. These new variables are identifiable (observable) combinations, although the individual variables are not independently identifiable nor observable. There are several methods for finding identifiable combinations, with theoretical bases including Taylor series (Evans & Chappell, 2000), Lie Algebra (Massonis and F. Villaverde, 2020, Merkt et al., 2015, Sedoglavic, 2002), Gröbner bases (Meshkat et al., 2014), and bootstrap (Hengl, Kreutz, Timmer, & Maiwald, 2007). These methods can be used to find functionally related parameters and/or identifiable combinations of them. This information provides hints for generating identifiable model reparameterizations, although this step requires some user intervention. As an example, consider the simplest SIR model 6, in which the parameter combination is identifiable. Reparameterizing the model in this way makes it identifiable, although it is still impossible to estimate either the transmission rate or the ratio of reported cases. In addition, the three states would also have to be scaled by one of the above-mentioned parameters, thus losing their original meaning.

Conclusions

Our analyses have shown that a fraction of the models found in the literature have unidentifiable parameters. Key parameters such as the transmission rate (), the recovery rate (), and the latent period () are structurally identifiable in most, but not all, models. The transmission and recovery rates are identifiable in roughly two thirds of the models, and the latent period in almost all () of them. Likewise, the states corresponding to the number of susceptible (S) and exposed (E) individuals are non-observable in roughly one third of the model versions analysed in this paper. The number of infected individuals (I) can usually be directly measured, but it is non-observable in one third of the model versions in which it is not measured. The situation is worse for the number of recovered individuals (R), which is almost never observable unless it is directly measured. Many models include other states in addition to S, E, I, and R, which are not always observable either.

The transmission rate and other parameters may vary during the course of an epidemic, as a result of a number of factors such as changes in public policy, population behaviour, or environmental conditions. To account for these variations, in the present study we have considered both the constant and the time-varying parameter case. Somewhat unexpectedly, we found that allowing for variability in an unknown parameter often improves the observability and/or identifiability of the model. This phenomenon might be explained by the contribution of this variability to the removal of symmetries in the model structure. Structural identifiability and observability depend on which states or functions are measured. The lack of these properties may in principle be surmounted by choosing the right set of outputs (Anguelova, Karlsson, & Jirstrand, 2012), but the required measurements are not always possible to perform in practice. Epidemiological models are a clear example of this; limitations such as lack of testing or the existence of asymptomatic individuals usually make it impossible to have measurements of all states. An alternative to measuring more states is to use a model with fewer compartments and/or a simpler parameterization, thus decreasing the number of states and/or parameters. Reducing the model dimension in this way may achieve observability and identifiability.

Even when it is not possible (or practical) to avoid non-observability or non-identifiability by any means, the model may still be useful, as long as it is only used to infer its observable states or identifiable parameters. For example, we may be interested in determining the transmission rate but not the number of recovered individuals R; in such case it is fine to use a model in which is identifiable even if R is not observable. Of course, this means that, to ensure that a model is properly used, it is necessary to characterize its identifiability and observability in detail, to know if the quantity of interest is observable/identifiable.

The contribution of this work has been to provide such a detailed analysis of the structural identifiability and observability of a large set of compartmental models of COVID-19 presented in the recent literature. The results of our analyses can be used to avoid the pitfalls caused by non-identifiability and non-observability. By classifying the existing models according to these properties, and arranging them in a structured way as a function of the compartments that they include, our study has answered the following question: given the sets of existing models and available measurements, which model is appropriate for inferring the value of some particular parameters, and/or to predict the time course of the states of interest?

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.