Literature DB >> 35513167

A multi-strain epidemic model for COVID-19 with infected and asymptomatic cases: Application to French data.

Mathilde Massard1, Raluca Eftimie2, Antoine Perasso3, Bruno Saussereau4.   

Abstract

Many SARS-CoV-2 variants have appeared over the last months, and many more will continue to appear. Understanding the competition between these different variants could help make future predictions on the evolution of epidemics. In this study we use a mathematical model to investigate the impact of three different SARS-CoV-2 variants on the spread of COVID-19 across France, between January-May 2021 (before vaccination was extended to the full population). To this end, we use the data from Geodes (produced by Public Health France) and a particle swarm optimisation algorithm, to estimate the model parameters and further calculate a value for the basic reproduction number R0. Sensitivity and uncertainty analysis is then used to better understand the impact of estimated parameter values on the number of infections leading to both symptomatic and asymptomatic individuals. The results confirmed that, as expected, the alpha, beta and gamma variants are more transmissible than the original viral strain. In addition, the sensitivity results showed that the beta/gamma variants could have lead to a larger number of infections in France (of both symptomatic and asymptomatic people).
Copyright © 2022 Elsevier Ltd. All rights reserved.

Entities:  

Keywords:  Basic reproduction number; COVID-19 Modeling; COVID-19 data for France; Variants of SARS-CoV-2

Mesh:

Year:  2022        PMID: 35513167      PMCID: PMC9059428          DOI: 10.1016/j.jtbi.2022.111117

Source DB:  PubMed          Journal:  J Theor Biol        ISSN: 0022-5193            Impact factor:   2.405


Introduction

Since the beginning of 2020, a pandemic of a novel coronavirus disease (COVID-19) has spread throughout the world and caused millions of deaths. COVID-19 is an infectious respiratory disease caused by the SARS-CoV-2 virus. The World Health Organization (WHO) learned of the existence of the SARS-CoV-2 on December 31, 2019 when an outbreak of “viral pneumonia” cases was notified in Wuhan, People’s Republic of China (Zhou et al., 2020, Tang et al., 2020). This disease has then spread to the whole world and very quickly led to saturated hospitals. The fast spread of SARS-CoV-2 also led to the emergence of different variants (i.e., viral genomes that may contain one or more mutations) for this virus, who have been circulating around the world since the beginning of this pandemic. While most of the genetic mutations observed in the circulating SARS-CoV-2 variants do not significantly change virus biology and its properties, some fitness-enhancing mutations (e.g., mutations leading to an increase in transmissibility, or in ability to evade the immune response) have been observed after the first months of virus spread (Harvey et al., 2021). The circulating variants are classified as: Variants Being Monitored (VBM) – those for which data indicate a potential impact on approved treatments but might not pose yet a significant and imminent risk to public health, Variants of Interest (VOI) – for which there is predicted increase in transmissibility of disease severity, and reduced efficacy of treatments, Variants of Concern (VOC) – for which there is evidence of an increase in transmissibility, increase in the severity of disease, and reduced effectiveness of treatments (Centers for Disease Control and Prevention, 2021, European Centre for Disease Prevention and Control, 2021). Variants can be re-classified based in their attributes and prevalence. As of October 2021, the European Centre for Disease Prevention and Control (ECDC) has listed three VOC (Beta, Gamma and Delta) and two VOI (Mu and Lambda) (European Centre for Disease Prevention and Control, 2021). To investigate the spread of COVID-19, including the role of the different SARS-CoV-2 variants on this spread, and to propose measures to slow-down this spread, researchers have focused their attention on various mathematical compartmental models, either stochastic (He et al., 2020, Faranda and Alberti, 2020) or deterministic (Calafiore and Fracastoro, 2021, He et al., 2020, Tang et al., 2020, Bentout et al., 2021, Yavuz et al., 2021, Arruda et al., 2021, Gonzalez-Parra et al., 2021, Khyar and Allali, 2020, Sridhar et al., 2021, Yagan et al., 2021); see also references therein. The great majority of these models focus on one single variant and investigate, for example, the impact of hospitalisation and quarantine (He et al., 2020, Tang et al., 2020), the impact of vaccination (Yavuz et al., 2021), or the impact of different age classes (Bentout et al., 2021, Calafiore and Fracastoro, 2021). Over the last few months, when it became clear the importance of different VOC on the fast increase in the number of COVID-19 cases in different countries, a series of mathematical models have been derived to investigate the role of multiple variants on the spread of SARS-CoV-2 (Arruda et al., 2021, Gonzalez-Parra et al., 2021, Khyar and Allali, 2020, Sridhar et al., 2021, Yagan et al., 2021). For example, Sridhar et al. (2021) and Yagan et al. (2021) used network models to investigate the effectiveness of mask-wearing in limiting the spread of COVID-19 in the context of viral mutations. Arruda et al. (2021) considered a SEIR model that incorporated several viral strains and also reinfection due to vaning immunity. They studied time-varying control strategies in the context of lockdown measures, and so they focused on the cost of infection control (i.e., lockdown) vs. the cost of elevated infection levels to the healthcare system, over a two-year time interval. Khyar and Allali (2020) focused on the global stability analysis of a two-strain SEIR epidemic model with two general incidence rates. They also calculated the basic reproduction number for their epidemic model. Finally, Gonzalez-Parra et al. (2021) extended a two-strain SEIR model to include also asymptomatic, hospitalized and dead individuals, to investigate the transmission of COVID-19 in Columbia. They consider variations in the contagiousness of the two strains, to see their impacts on the number of infections, hospitalisations and deaths. In this study we also focus on the role of different SARS-CoV-2 variants on the transmission of this virus across France in the absence of vaccination. However, unlike the previous studies, here we consider a simpler compartmental model (i.e., a generalisation of a SIR model) that includes also asymptomatic cases and dead individuals infected with the original variant, as well as cases infected with other variants. To calculate the basic reproductive number , we first focus on a general model with variants, and derive a formula for . Then, we apply this formula for the case with two variants (), corresponding to the alpha and beta/gamma variants (see discussion on French data in Section 2 ). This model is parametrised (with the help of a particle swarm optimisation algorithm) using French data selected over the longest possible time interval that can take into account the emergence of different SARS-CoV-2 variants without including also the vaccination. Sensitivity analysis is performed to understand the impact of uncertainty in model parameters on the overall outcome (i.e., symptomatic and asymptomatic infections). This sensitivity analysis allows us to investigate the severity of infections with different variants in France, in the absence of any vaccination.
Fig. 2

The evolution of and , as given by data on Geodes (see also Table 4, Table 5, Table 6). This is the rescaled data that includes the assumption of 13 days in the infected compartments.

The evolution of and , as given by data on Geodes (see also Table 4, Table 5, Table 6). This is the rescaled data that includes the assumption of 13 days in the infected compartments.
Table 4

Percentage of initial strain.

DateValueDateValueDateValue
2021-02-12-2021-02-1842,92021-02-13-2021-02-1940,72021-02-14-2021-02-2039,7
2021-02-15-2021-02-2139,52021-02-16-2021-02-2237,32021-02-17-2021-02-2335,3
2021-02-18-2021-02-2433,62021-02-19-2021-02-2532,02021-02-20-2021-02-2630,3
2021-02-21-2021-02-2729,52021-02-22-2021-02-2829,42021-02-23-2021-03-0127,2
2021-02-24-2021-03-0225,72021-02-25-2021-03-0324,32021-02-26-2021-03-0422,9
2021-02-27-2021-03-0521,62021-02-28-2021-03-0620,92021-03-01-2021-03-0720,7
2021-03-02-2021-03-0819,12021-03-03-2021-03-0918,12021-03-04-2021-03-1017,0
2021-03-05-2021-03-1115,72021-03-06-2021-03-1214,72021-03-07-2021-03-1314,2
2021-03-08-2021-03-1414,12021-03-09-2021-03-1513,02021-03-10-2021-03-1612,2
2021-03-11-2021-03-1711,52021-03-12-2021-03-1811,02021-03-13-2021-03-1910,4
2021-03-14-2021-03-2010,22021-03-15-2021-03-2110,12021-03-16-2021-03-229,4
2021-03-17-2021-03-238,92021-03-18-2021-03-248,42021-03-19-2021-03-258,0
2021-03-20-2021-03-267,62021-03-21-2021-03-277,42021-03-22-2021-03-287,3
2021-03-23-2021-03-296,92021-03-24-2021-03-306,72021-03-25-2021-03-316,5
2021-03-26-2021-04-016,32021-03-27-2021-04-026,12021-03-28-2021-04-036,0
2021-03-29-2021-04-046,02021-03-30-2021-04-055,92021-03-31-2021-04-065,6
2021-04-01-2021-04-075,42021-04-02-2021-04-085,42021-04-03-2021-04-095,3
2021-04-04-2021-04-105,22021-04-05-2021-04-115,22021-04-06-2021-04-125,1
2021-04-07-2021-04-134,92021-04-08-2021-04-144,82021-04-09-2021-04-154,6
2021-04-10-2021-04-164,32021-04-11-2021-04-174,32021-04-12-2021-04-184,2
2021-04-13-2021-04-194,02021-04-14-2021-04-203,92021-04-15-2021-04-213,8
2021-04-16-2021-04-223,82021-04-17-2021-04-233,72021-04-18-2021-04-243,7
2021-04-19-2021-04-253,72021-04-20-2021-04-263,62021-04-21-2021-04-273,5
2021-04-22-2021-04-283,42021-04-23-2021-04-293,42021-04-24-2021-04-303,4
2021-04-25-2021-05-013,42021-04-26-2021-05-023,42021-04-27-2021-05-033,3
2021-04-28-2021-05-043,42021-04-29-2021-05-053,32021-04-30-2021-05-063,4
2021-05-01-2021-05-073,42021-05-02-2021-05-083,42021-05-03-2021-05-093,4
2021-05-04-2021-05-103,32021-05-05-2021-05-113,22021-05-06-2021-05-123,3
2021-05-07-2021-05-133,2
Table 5

Percentage of alpha variant.

DateValueDateValueDateValue
2021-02-12-2021-02-1844,32021-02-13-2021-02-1946,02021-02-14-2021-02-2046,9
2021-02-15-2021-02-2147,12021-02-16-2021-02-2249,22021-02-17-2021-02-2351,0
2021-02-18-2021-02-2452,82021-02-19-2021-02-2554,32021-02-20-2021-02-2655,9
2021-02-21-2021-02-2756,62021-02-22-2021-02-2856,82021-02-23-2021-03-0159,0
2021-02-24-2021-03-0260,62021-02-25-2021-03-0362,12021-02-26-2021-03-0463,5
2021-02-27-2021-03-0564,82021-02-28-2021-03-0665,62021-03-01-2021-03-0765,8
2021-03-02-2021-03-0867,22021-03-03-2021-03-0968,12021-03-04-2021-03-1069,1
2021-03-05-2021-03-1170,12021-03-06-2021-03-1271,22021-03-07-2021-03-1371,6
2021-03-08-2021-03-1471,72021-03-09-2021-03-1573,02021-03-10-2021-03-1673,9
2021-03-11-2021-03-1774,62021-03-12-2021-03-1875,22021-03-13-2021-03-1975,6
2021-03-14-2021-03-2075,92021-03-15-2021-03-2175,92021-03-16-2021-03-2276,5
2021-03-17-2021-03-2376,92021-03-18-2021-03-2477,72021-03-19-2021-03-2578,1
2021-03-20-2021-03-2678,82021-03-21-2021-03-2779,22021-03-22-2021-03-2879,3
2021-03-23-2021-03-2980,12021-03-24-2021-03-3080,62021-03-25-2021-03-3180,8
2021-03-26-2021-04-0181,22021-03-27-2021-04-0281,42021-03-28-2021-04-0381,5
2021-03-29-2021-04-0481,52021-03-30-2021-04-0581,42021-03-31-2021-04-0681,8
2021-04-01-2021-04-0782,12021-04-02-2021-04-0882,22021-04-03-2021-04-0982,3
2021-04-04-2021-04-1082,62021-04-05-2021-04-1182,62021-04-06-2021-04-1282,7
2021-04-07-2021-04-1382,52021-04-08-2021-04-1482,32021-04-09-2021-04-1582,4
2021-04-10-2021-04-1682,42021-04-11-2021-04-1782,32021-04-12-2021-04-1882,3
2021-04-13-2021-04-1982,22021-04-14-2021-04-2082,52021-04-15-2021-04-2182,6
2021-04-16-2021-04-2282,52021-04-17-2021-04-2382,42021-04-18-2021-04-2482,3
2021-04-19-2021-04-2582,32021-04-20-2021-04-2682,22021-04-21-2021-04-2781,9
2021-04-22-2021-04-2881,62021-04-23-2021-04-2981,32021-04-24-2021-04-3080,8
2021-04-25-2021-05-0180,52021-04-26-2021-05-0280,52021-04-27-2021-05-0380,2
2021-04-28-2021-05-0479,92021-04-29-2021-05-0579,72021-04-30-2021-05-0679,4
2021-05-01-2021-05-0779,12021-05-02-2021-05-0879,12021-05-03-2021-05-0979,1
2021-05-04-2021-05-1078,22021-05-05-2021-05-1177,92021-05-06-2021-05-1277,6
2021-05-07-2021-05-1377,4
Table 6

Percentage of beta variant.

DateValueDateValueDateValue
2021-02-12-2021-02-184,52021-02-13-2021-02-194,62021-02-14-2021-02-204,6
2021-02-15-2021-02-214,62021-02-16-2021-02-224,62021-02-17-2021-02-234,8
2021-02-18-2021-02-244,92021-02-19-2021-02-255,02021-02-20-2021-02-265,2
2021-02-21-2021-02-275,42021-02-22-2021-02-285,42021-02-23-2021-03-015,5
2021-02-24-2021-03-025,42021-02-25-2021-03-035,32021-02-26-2021-03-045,2
2021-02-27-2021-03-055,02021-02-28-2021-03-064,92021-03-01-2021-03-074,8
2021-03-02-2021-03-084,72021-03-03-2021-03-094,72021-03-04-2021-03-104,9
2021-03-05-2021-03-115,02021-03-06-2021-03-125,02021-03-07-2021-03-135,0
2021-03-08-2021-03-145,02021-03-09-2021-03-155,02021-03-10-2021-03-165,0
2021-03-11-2021-03-174,92021-03-12-2021-03-184,82021-03-13-2021-03-194,7
2021-03-14-2021-03-204,62021-03-15-2021-03-214,62021-03-16-2021-03-224,6
2021-03-17-2021-03-234,52021-03-18-2021-03-244,42021-03-19-2021-03-254,4
2021-03-20-2021-03-264,42021-03-21-2021-03-274,32021-03-22-2021-03-284,3
2021-03-23-2021-03-294,32021-03-24-2021-03-304,22021-03-25-2021-03-314,2
2021-03-26-2021-04-014,12021-03-27-2021-04-024,12021-03-28-2021-04-034,0
2021-03-29-2021-04-044,12021-03-30-2021-04-054,12021-03-31-2021-04-064,0
2021-04-01-2021-04-073,92021-04-02-2021-04-083,92021-04-03-2021-04-093,9
2021-04-04-2021-04-103,82021-04-05-2021-04-113,82021-04-06-2021-04-123,8
2021-04-07-2021-04-133,92021-04-08-2021-04-144,02021-04-09-2021-04-154,0
2021-04-10-2021-04-164,12021-04-11-2021-04-174,22021-04-12-2021-04-184,2
2021-04-13-2021-04-194,42021-04-14-2021-04-204,62021-04-15-2021-04-214,7
2021-04-16-2021-04-224,92021-04-17-2021-04-235,02021-04-18-2021-04-245,1
2021-04-19-2021-04-255,12021-04-20-2021-04-265,12021-04-21-2021-04-275,1
2021-04-22-2021-04-285,22021-04-23-2021-04-295,32021-04-24-2021-04-305,4
2021-04-25-2021-05-015,52021-04-26-2021-05-025,52021-04-27-2021-05-035,6
2021-04-28-2021-05-045,72021-04-29-2021-05-055,72021-04-30-2021-05-065,7
2021-05-01-2021-05-075,72021-05-02-2021-05-085,72021-05-03-2021-05-095,7
2021-05-04-2021-05-105,92021-05-05-2021-05-116,02021-05-06-2021-05-125,9
2021-05-07-2021-05-136,0
The paper is structured as follows. In Section 2 we introduce a general compartment model that considers a generic number of N variants, that can lead to symptomatic as well as asymptomatic cases. We use this model to calculate a general formula for in terms of the parameters associated with different variants. In Section 3 we discuss SARS-CoV-2 data for France, and based on this data we focus on a simpler model with variants: alpha and beta. Using available data, we parametrise this simpler model with the help of a particle swarm optimisation algorithm. In Section 4 we estimate the basic reproductive number for the three SARS-CoV-2 variants considered here, and we perform a sensitivity analysis to investigate the impact of uncertainties in the previously-identified model parameters on the model outcome (i.e., the number of infectious individuals, as well as ).

A general model with N variants

We start by presenting a general compartmental model that takes into account a generic number N of variants, with . We choose N variants because it allows us to include all current and possible future VOC variants. Symptomatic and asymptomatic cases are taken into consideration with the assumption that asymptomatic people do not die from the disease. Natural births and deaths can be overlooked, due to the large size of the French population. Loss of immunity can also be ignored because the number of individuals infected twice is still low (Brouqui et al., 2021), and this loss of immunity is not present for all variants. For example, variants with the E484K mutation in the spike protein may be responsible for immune escape (Nonaka et al., 2021, Wise, 2021).

Model description

For the model considered in this study, we denote the initial strain by , the first variant by , …, the N-th variant by . The population is divided into different classes: susceptible individuals (S) who are healthy and can contract the disease; individuals infected with the original virus and having symptoms (); individuals infected with the original virus and having no symptoms (); individuals infected with the variants , …, and having symptoms (, …, ); individuals infected with the variants , …, and having no symptoms (, …, ); dead individuals (D); recovered individuals (R) who are immune to the disease. The dynamics of the population is described by the system below (see also Fig. 1 for a schematic description of the interactions between different model variables).
Fig. 1

Diagram describing the interactions incorporated into model (1).

Diagram describing the interactions incorporated into model (1). In the above system (1), the equation for describes the evolution of individuals infected with the initial strain and having symptoms (). Susceptible persons S come into contact with persons infected by the initial strain with symptoms () or without symptoms (). There is a certain probability of being infected by the initial virus which results in the terms and . A proportion of these individuals will have symptoms. Finally, infected individuals can die at a rate or can recover at a rate . Similar terms can be found in the equations describing the evolution of the other symptomatic infected compartments ,…,; see also Table 1 .
Table 1

Summary of parameters involved in model (1). Parameters and are respectively equal to and , with N the size of the population considered (hence the units for these parameters are ). Here, and are the average duration times between two contaminating contacts (with the strain k) by a symptomatic person and by an asymptomatic person.

SymbolBiological interpretationUnit
μV0,μV1,..,μVNMortality rate due to the initial virus and to other N variantday-1
1δV0,1δV1,,1δVNAverage time of infection for symptomatic individualsday
1bV0,1bV1,,1bVNAverage time of infection for asymptomatic individualsday
αiV0,αiV1,,αiVNTransmission rate of variants and initial virus from symptomatic peoplesize-1×day-1
αaV0,αaV1,,αaVNTransmission rates of variants and initial virus from asymptomatic peoplesize-1×day-1
pV0,pV1,,pVNProbability that an individual infected with the initial virus, variant 1, .. variant N, is symptomatic
Summary of parameters involved in model (1). Parameters and are respectively equal to and , with N the size of the population considered (hence the units for these parameters are ). Here, and are the average duration times between two contaminating contacts (with the strain k) by a symptomatic person and by an asymptomatic person. The equation for in (1) describes the evolution of the individuals infected by the initial strain and having no symptoms (). A proportion of susceptibles can become infected and have no symptoms, and as before there is a probability of infection following contact with other symptomatic individuals or asymptomatic individuals . In addition, individuals can leave this asymptomatic class following recovery at a rate . We remind the reader that the asymptomatic persons cannot die. Similar terms can be found in the equations describing the evolution of the other compartments ,…,; see also Table 1. The equation for in (1) describes the evolution of recovered individuals. The terms that appear in this equation describe the recovery of infected and asymptomatic individuals, as discussed above. The equation for in (1) describes the time-evolution of COVID-19-dead individuals. The terms in this equation are the death terms that we discussed above for the symptomatic infected compartments. The equation for in (1) describes the time-evolution of the susceptible people. These individuals leave the compartment S when becoming infected (after contact with symptomatic or asymptomatic people). All parameters involved in this general model (1) with N viral variants are summarized in the Table 1. In the following we discuss briefly the calculation of the basic reproduction number for this general model.

calculation

A fundamental tool in epidemiology is the basic reproduction number , which represents the number of secondary infections resulting from a single infectious individual introduced into a fully susceptible population (Diekmann et al., 1990, Van den Driessche and Watmough, 2002, Perasso, 2018). Note that in general a population is not fully susceptible, with some individuals already having immunity/cross-immunity to the virus. Therefore, even if throughout this study we refer to the basic reproductive number , we actually understand the effective reproduction number (i.e., the number of secondary infections resulting from a single infectious individual introduced into a population formed of susceptible and non-susceptible individuals); this will be more clear in Section 3 in the context of alpha and beta variants for COVID-19. One method that can be used to calculate the basic reproduction number for the finite-dimensional system (1) makes use of the next generation matrix (Van den Driessche and Watmough, 2002). This method is based on the definition of as the dominant eigenvalue of the “next generation matrix” (Diekmann et al., 1990), i.e., a matrix that relates the numbers of newly infected individuals in various categories in consecutive generations (Diekmann et al., 2010). Applying this method to model (1) we obtain the next generation matrix M below: Thus, we obtain the following formula for : Each eigenvalue consists only of the parameters corresponding to a single strain. It is like there are several epidemics at the same time, and they don’t interfere with each other. By obtaining a value of from the available data (see Section 3), we will be able to conclude which variant takes over the others.

Parameters estimation for a 2 variant-model (): alpha and beta/gamma variants

Throughout the rest of this paper we focus on a simpler model with only two variants, (i.e., ), and we use the available data to obtain an estimate for (or rather the effective reproduction number, as discussed above). We estimate the 18 parameters involved in model (1) and in Eq. (2) using a Particle Swarm Optimization algorithm.

Data

The data we use in this study is from Geodes (i.e., the cartographic observatory of epidemiological indicators produced by Public Health France1 ), and covers the period between 12th February and 7th May 2021, because before the 12 February the different strains of the virus were not recorded (only virus presence was recorded). Moreover, we consider data only until 7th May to avoid the effects of vaccines (since in France the vaccination really started off in May). We focus on the percentages of positive RT-PCR tests identifying the initial strain, the alpha variant (which appeared in France since mid-December 2020) and the beta & gamma variants (present in France since end of December 2020 & beginning of February 2021, respectively), calculated over a 7-day period. This period allows to remove the effect of reduced testing on Sundays. The data can be seen in Table 4, Table 5, Table 6, in Appendix A. Note that on Geodes, the data for the beta and gamma variants are presented together, and for this reason in this study we combine them into a single variant (called variant 2 in the next section). On Geodes, the data included also percentages of a so-called “indeterminate variant”. To remove this indeterminate variant (so we can have only the initial strain, the alpha variant and the beta/gamma variant), we considered the following approach. Denote the alpha strain by , the beta strain by and the initial strain by . Then we calculate:with Similar calculations were performed to obtain the new percentages for the beta/gamma variants, as well as the initial strain. The number of new confirmed cases per day in France is available on the official french public data site 2 . These data are summarised in Fig. 7, in Appendix. To remove the effects of Sunday testing, when fewer RT-PCR tests are carried out, we take an average over 7 days at each time point, to correspond to the values of the percentages of the variants. These data are shown in Fig. 8 in Appendix.
Table 7

Number of new confirmed cases in France per day.

DateValueValueValueValueValue
2021-02-12-2021-02-16207012123116546437619590
2021-02-17-2021-02-212501822501241162237122046
2021-02-22-2021-02-26464620064315192540325207
2021-02-27-2021-03-03239961995247032285726788
2021-03-04-2021-03-08252792350723306218255327
2021-03-09-2021-03-132330230303271662522929759
2021-03-14-2021-03-18263436471299753850134998
2021-03-19-2021-03-233508835327305811579214678
2021-03-24-2021-03-286537345641418694261937014
2021-03-29-2021-04-02909430702590385065946677
2021-04-03-2021-04-07139176679410793804512951
2021-04-08-2021-04-12849994124343284348958536
2021-04-13-2021-04-173911343505380453644235861
2021-04-18-2021-04-22293446696430983496834318
2021-04-23-2021-04-27323403263324465595230317
2021-04-28-2021-05-02315392653824299256709888
2021-05-03-2021-05-07376024371260002171219124
2021-05-08-2021-05-1220745912832921979121498
2021-05-1319461
Table 8

Number of new confirmed cases in France per day, averaged over 7 days.

DateValueDateValueDateValue
12/02-18/0218566,1428613/02-19/0219054,0000014/02-20/0219216,85714
15/02-21/0220002,5714316/02-22/0220041,1428617/02-23/0220108,85714
18/02-24/0221037,5714319/02-25/0221452,1428620/02-26/0221608,00000
21/02-27/0221840,1428622/02-28/0221541,0000023/02-01/0321549,14286
24/02-02/0321948,1428625/02-03/0321272,2857126/02-04/0321254,57143
27/02-05/0321011,7142928/02-06/0320913,1428601/03-07/0321180,71429
02/03-08/0321269,8571403/03-09/0321333,4285704/03-10/0321835,57143
05/03-11/0322105,1428606/03-12/0322351,1428607/03-13/0323273,00000
08/03-14/0323918,4285709/03-15/0324081,8571410/03-16/0325035,14286
11/03-17/0326206,2857112/03-18/0327325,1428613/03-19/0328733,57143
14/03-20/0329529,0000015/03-21/0330134,4285716/03-22/0331466,00000
17/03-23/0329280,7142918/03-24/0333119,5714319/03-25/0334640,00000
20/03-26/0335608,7142921/03-27/0336650,4285722/03-28/0337569,42857
23/03-29/0336612,5714324/03-30/0338901,7142925/03-31/0337996,71429
26/03-01/0438713,5714327/03-02/0439400,4285728/03-03/0435300,14286
29/03-04/0439554,4285730/03-05/0439797,1428631/03-06/0436560,42857
01/04-07/0429976,5714302/04-08/0434882,2857103/04-09/0434106,00000
04/04-10/0438301,2857105/04-11/0433744,2857106/04-12/0433421,85714
07/04-13/0437860,1428608/04-14/0442225,0000009/04-15/0435517,28571
10/04-16/0434831,4285711/04-17/0433771,0000012/04-18/0432978,00000
13/04-19/0432715,1428614/04-20/0433284,4285715/04-21/0432064,85714
16/04-22/0431532,4285717/04-23/0430946,4285718/04-24/0430485,28571
19/04-25/0429788,2857120/04-26/0429682,0000021/04-27/0427856,14286
22/04-28/0427366,2857123/04-29/0426254,8571424/04-30/0425106,14286
25/04-01/0524111,4285726/04-02/0522029,0000027/04-03/0521715,85714
28/04-04/0520866,4285729/04-05/0520075,1428630/04-06/0519385,71429
01/05-07/0518646,4285702/05-08/0517942,8571403/05-09/0517834,28571
04/05-10/0517767,4285705/05-11/0517113,1428606/05-12/0516470,00000
07/05-13/0516148,42857
From the percentages of each variant as well as the number of people infected at each time point t (i.e., day), we obtain three datasets corresponding to the number of new cases per day for each of the three strains. These three datasets are of size 85 corresponding to the period from February 12 to May 7. However, the different symptomatic infected compartments (I) in system (1) (with ) count the current numbers of infected individuals and not the newly infected per day. To transform the data so that we can use it to parametrise our model, we assume that individuals stay an average of 13 days in a symptomatic compartment before recovering (George et al., 2021). By including this recovery delay, we reduced the size of the three datasets from 85 to 73 days:with denoting the rescaled cumulative data that consider the assumption of 13 days infections (i.e., we sum up all daily infections that took place over the past 13 days), which is then used to parametrise the mathematical model. The evolution of the three infected compartments can be seen in Fig. 2. Note that the curve for the infected population with the alpha variant (panel (b)) starts decreasing around day (marked by the vertical dashed line). This corresponds to 2 April 2021, when the French government imposed a 4-weeks national lockdown (which lasted until 3 May). At the same time, the beta/gamma variant starts to increase slightly until , after which it decreases as well. Data for initial population sizes. For the numerical simulations performed throughout the rest of the paper, we consider the following initial population sizes: Using the latest census data from mainland France (Décembre, 2020), in this study we assume that the total population of France is . We consider this initial condition because on 12 February 2021 (when we start our investigation into the dynamics of these different variants) there were 3,267,386 people in France who recovered from the disease (i.e., ) and 81,448 people who died (i.e., ) (Santé publique france, 2021). For the infected sub-populations, we use the datasets in Table 4, Table 5, Table 6: (the first value of the first dataset), (the first value of the second dataset), (the first value of the third dataset). Regarding the infected people, we assume that are symptomatic and are asymptomatic (Xiao et al., 2020), and that the asymptomatic are not tested the same way as the symptomatic infected individuals and thus we take , . For the Susceptible population, we need to subtract from the total population the number of individuals infected with the different variants:

Particle Swarm Optimization

The Particle Swarm Optimization (PSO) algorithm was created in 1995 by Russel Eberhart, electrical engineer, and James Kennedy, social psychologist (Kennedy and Eberhart, 1995), and it is inspired by the collective behaviour of flocks of birds or schools of fish. For the PSO, the particle (i.e., potential solution of the model) moves by making allowances between getting closer to the optimal solution visited and getting closer to the solutions found in its neighborhood. This optimization algorithm performs well on parameter optimization for ordinary differential equation models (Akman et al., 2018), but convergence to the overall optimal solution is not always guaranteed. In the following we describe how we apply this PSO algorithm to identify the parameters of our model (1) with two SARS-CoV-2 variants, using the data described above.

Objective function and search bounds for PSO

For each viral strain we calculate the residual sum of squareswhere . describe the empirical observations at time corresponding to the number of individuals infected by strain . describe the numerical predictions at time for the individuals infected by strain (as given by system (1) with ). The objective function to be minimized is therefore Based on reasonable parameter bounds (see discussion below), we constrain our search space with lower and upper bounds for : This means that infected people take at most 25 days and at least 1 day to infect a person (Amiri Mehra et al., 2020). Indeed, the transmission rate has been estimated at for the whole world (Marinov et al., 2020) which corresponds to days to transmit the virus (because N = 7,870,000,000 is the world population). The symptomatic proportion is between 0.55 and 0.9 (Xiao et al., 2020), a symptomatically-infected person takes between 3 days (Pottier, 2020) and 13 days (George et al., 2021) to recover, and between 5 days (Fernández-Villaverde and Jones, 2020) and 13 days to die (also to match our infection hypothesis (George et al., 2021)).

Parameter estimation

In Fig. 3 we show the best fit between the data collected from Geodes (Geodes, 2021) on infected individuals (black dots), and the numerically-predicted number (red curve) of individuals infected with (a) initial strain, (b) alpha strain, (c) beta/gamma strain. The parameters for which the numerical solutions were obtained are summarised in Table 2 , while Table 3 gives the biological interpretation of these parameters. The parameters and are respectively equal to and , with N the size of the population considered, and the average duration between two contaminating contacts by a symptomatic person and by an asymptomatic person by the strain k. For the biological interpretation of the other parameters see Table 1.
Fig. 3

Fitting the numerically-predicted infection curves (red curves) for the three strains, to the data collected from Geodes (black dots). The fit was performed using the Particle Swarm Optimisation (PSO) algorithm.

Table 2

Parameter values estimated from Geodes data using the PSO algorithm.

SymbolValueSymbolValueSymbolValue
αiV02.198814e−09αiV16.748454e−09αiV21.970476e−09
αaV02.580832e−09αaV11.099686e−08αaV21.353279e−08
pV07.594355e−01pV18.531318e−01pV28.229163e−01
bV01.490951e−01bV12.949318e−01bV22.110977e−01
δV07.926397e−02δV13.122869e−01δV21.194538e−01
μV07.727916e−02μV11.184327e−01μV29.728384e−02
Table 3

Biological interpretation of parameters estimated here.

Viral strainParameter interpretationValue
V0Number of days for a contaminating contact with a symptomatic7.11
Number of days for a contaminating contact with a asymptomatic6.05
Percentage of symptomatic individuals0.76
Asymptomatics: number of days to recover6.71
Symptomatics: number of days to recover12.62
Number of days before dying12.94



V1 (alpha)Number of days for a contaminating contact with a symptomatic2.31
Number of days for a contaminating contact with a asymptomatic1.42
Percentage of symptomatic individuals0.85
Asymptomatics: number of days to recover3.39
Symptomatics: number of days to recover3.20
Number of days before dying8.44



V2 (beta/ gamma)Number of days for a contaminating contact with a symptomatic7.93
Number of days for a contaminating contact with a asymptomatic1.15
Percentage of symptomatic individuals0.82
Asymptomatics: number of days to recover4.73
Symptomatics: number of days to recover8.37
Number of days before dying10.28
Fitting the numerically-predicted infection curves (red curves) for the three strains, to the data collected from Geodes (black dots). The fit was performed using the Particle Swarm Optimisation (PSO) algorithm. Parameter values estimated from Geodes data using the PSO algorithm. Biological interpretation of parameters estimated here. The fit of the individuals infected with the initial and alpha strain is relatively good. In contrast, the fit of individuals infected with the beta strain is quite poor for time days and days. The evolution of asymptomatic infected people is presented in the Fig. 4 . We see that the number of asymptomatic individuals infected with the initial strain increases at the beginning and then decreases (following the trend of symptomatic infected individuals). The asymptomatic individuals infected with the other two variants (alpha and beta) follow the same general pattern as the symptomatic individuals but with fewer cases.
Fig. 4

Time-evolution of the three asymptomatic compartments of model (1).

Time-evolution of the three asymptomatic compartments of model (1). Looking at the numbers in Table 3, Table 2 we see that the people who die the fastest are those affected by the alpha variant. Indeed, . (Note that this result is also supported by the values of the case fatality rate (CFR) for each of the three variants: .) In addition, people who take the least time to transmit the virus are those affected by the alpha variant which is consistent with the fact that it is said to be more transmissible. There are a higher number of days for the transmission of the disease to symptomatic people than for the asymptomatic people, and this is true for all three cases.

Results

Estimation of the basic reproduction number

To calculate the value of , we have computed three non-zero eigenvalues for the next generation matrix: (corresponding to the initial strain), (corresponding to the alpha strain), (corresponding to the beta/gamma strain). It is clear that the two variants (with ) are more contaminating than the initial strain (with ). Both alpha and beta variants have very close values, so we can conclude at first that they have a similar intensity, at least for France, in the absence of vaccination. Note that this value is consistent with the value calculated by other French studies that investigated COVID-19 transmission during the same period of time (Gaymard et al., 2020).

Local sensitivity analysis

We now investigate the impact on when we vary by one day the biological parameter estimates given in Table 3. For example, if we estimated at the number of days required for the transmission of the disease by a symptomatic person, we now consider the interval days. Each time we vary one parameter, while fixing all other parameters. The results are presented in Fig. 5 .
Fig. 5

Local sensitivity of when we vary: (a) parameters associated with the original strain ; (b) parameters associated with the alpha strain ; (c) parameters associated with the beta/gamma strain . The right-hand sub-panel in (c) shows, on a much larger vertical axis, as we vary .

Local sensitivity of when we vary: (a) parameters associated with the original strain ; (b) parameters associated with the alpha strain ; (c) parameters associated with the beta/gamma strain . The right-hand sub-panel in (c) shows, on a much larger vertical axis, as we vary . First, we see in Fig. 5(a) that the parameters of the initial strain do not impact because the corresponding value is never the maximum among the three eigenvalues of the next generation matrix. For the alpha variant (Fig. 5(b)), we see that is especially sensitive to and , parameters describing the transmission of infection following contacts with symptomatic and asymptomatic infected individuals. Thus, reducing the contacts (e.g., via lockdown) could lead to a reduction in . Also notice that none of the parameters related to the alpha variant increase above 2. For the beta/gamma variants (Fig. 5(c)), we see that is very sensitive to . Changes in this parameter can increase up to (unlike the case for the alpha variant, where was always below 2). This shows us the importance of controlling the parameter to be able to control the outbreak of the beta/gamma variant. Another parameter that impacts is , the recovery rate of asymptomatic individuals. This shows the key role of the asymptomatic people in the rapid spread of the beta/gamma strain.

Global sensitivity analysis for the infected compartments

Since knowledge about model parameters is incomplete (especially knowledge about the role of asymptomatic people on disease transmission), we are now performing a global sensitivity and uncertainty analysis of the different infected compartments, where we vary all parameters at the same time. To this end, we consider a classical approach that combines the Latin Hypercube Sampling (LHS) with the Partial Rank Correlation Coefficient (PRCC) (Marino et al., 2008). The parameters are varied in the same way as before (i.e., by one day for each of the rates), and the sample size for the LHS is 100. In Fig. 6 we show the variation in the number of individuals infected with the original strain, as we vary all model parameters. The black curves show the median time evolution of and , while the gray regions show the range between quantiles and . We can see that both and are very sensitive for days. For example, a 1-day change in the various transition rates incorporated into the model, leads on day (i.e. February 22) to a variation between 50,000 to 80,000 in the symptomatic infected people (), and a variation between 10,000 to 25,000 asymptomatic infected people approximately. In Fig. 7 we show the variation in the number of individuals infected with the alpha strain, as we vary all model parameters. Note here the change in the maximum peak of both and : not only the amplitude but also the shift in time: from day for the median black curve, to day for the quantiles. If we compare these results with the results in Fig. 8 for the beta/gamma strain, we observe that while the maximum for is reached around days, for the maximum is reached for days. Therefore, the beta/gamma strains are more aggressive. Moreover, by comparing Fig. 7, Fig. 8, we see that and could reach much higher amplitudes (i.e., 10-fold higher) compared to and . We will return to this aspect in the Discussion Section 5.
Fig. 6

Uncertainty in the and , as we vary by a maximum of day all model parameters (according to the LHS scheme).

Fig. 7

Uncertainty in the and as we vary by a maximum of day all model parameters (according to the LHS scheme).

Fig. 8

Uncertainty in the and as we vary by a maximum of day all model parameters (according to the LHS scheme).

Uncertainty in the and , as we vary by a maximum of day all model parameters (according to the LHS scheme). Uncertainty in the and as we vary by a maximum of day all model parameters (according to the LHS scheme). Uncertainty in the and as we vary by a maximum of day all model parameters (according to the LHS scheme). Finally, for completeness, in Fig. 9 we also show the time-variation in the recovered and dead people. Again, we see a large variability in the outcome between days and which, for lasts also for days.
Fig. 9

Uncertainty in the and as all model parameters are varied by one day.

Uncertainty in the and as all model parameters are varied by one day. To quantify the sensitivity of model outcomes (i.e., number of symptomatic and asymptomatic infected individuals) to changes in model parameters, and identify the critical parameters, in Fig. 10, Fig. 11, Fig. 12 we show the Partial Rank Correlation Coefficient (PRCC) – a sampling-based measure for nonlinear but monotonic relationships between inputs and outputs (Marino et al., 2008). We see here that the parameters with the greatest impact on the outcome (i.e., parameters with PRCC indexes above and below ) are:
Fig. 10

PRCC of and .

Fig. 11

PRCC of and .

Fig. 12

PRCC of and .

For : parameters and ; while for : parameters , and ; For : parameters and ; while for : parameters and ; For : parameter and ; while for : parameters and . PRCC of and . PRCC of and . PRCC of and . First we observe that due to the interactions between the model variables, parameters associated with the second variant () impact the infections with the original () strain and the infections with the first variant (). For example, impacts not only and , but also and , which suggest that the people infected by beta/gamma strain could have a strong impact on the overall evolution of epidemics. This is despite the fact that for , the impact of beta/gamma variant was decoupled from the impact of the original variant and alpha variant (see Eqn. (2)).

Discussion & conclusion

In this paper we investigated the evolution of COVID-19 epidemic in France between February-May 2021 in the presence of multiple viral strains, while estimating the parameters that characterise this epidemic. The time interval for the data was chosen to reduce the impact of vaccination on the results (although other control measures such as masks and social distancing were still in place in France during this period). We focused on three viral variants identified in the data: the original (i.e., March 2020) SARS-CoV-2 variant, the alpha variant (first detected in the UK) and the beta/gamma variant (first detected in South Africa and in Brazil). The last two were considered together here, because in France the tests did not discriminate between them. First, we showed that the value for was given by the maximum of three basic reproduction values corresponding to the three different variants. This suggested a sort of decoupling between the evolution of different variants, the most transmissible one having the greatest influence on the evolution of the outbreak. Using French data on Geodes (2021) and a PSO algorithm, we estimated , consistent with the value estimated by other French studies (Gaymard et al., 2020). Moreover, this value was similar for the alpha and beta/gamma variants, but larger than the value corresponding to the original strain (thus suggesting that between February-May the alpha and beta/gamma variants were more transmissible than the original strain). A local sensitivity analysis for showed that the parameters with the largest impact were the transmission rates of symptomatic and asymptomatic individuals; see Fig. 5. In regard to parameter estimation (see Table 3) we also found a higher number of days necessary to create contaminating contacts for the symptomatic people than for the asymptomatic people (for all three variants). This unexpected result could be explained by the fact that asymptomatic people come into contact with more peers ignoring distancing rules. The results in Table 3 show also that the number of days to recover when symptomatic is slightly lower than for asymptomatic people for the alpha variant, while it is almost twice more for the other strains. It would be interesting to investigate whether there is a specificity of the alpha variant in contrast to the other variants. At this point we could not find any epidemiological data to test the validity of this theoretical result. Second, a global sensitivity and uncertainty analysis identified the variations in the amplitude of symptomatic and asymptomatic infections for all three strains, as well as the day when these infections peak (see Fig. 6, Fig. 7, Fig. 8). Moreover, the PRCC analysis identified some critical parameters for the evolution of the epidemics (see Fig. 10, Fig. 11, Fig. 12). The interesting result was that parameters associated with the variant () seemed to impact also the evolution of infections with the and variants, thus suggesting that the infections with the different viral strains are not really decoupled (as suggested by the formula (2)). The uncertainty analysis also showed that the evolution of those infected with the beta/gamma strain i.e. has more uncertainty compared to the other two strains (when we vary the interaction rates by one day). Also in regard to the uncertainty of the results in this study, we need to mention the fact that since we could not find mortality data for each of the different strains discussed here, the exact identification of parameters actually depends on all other model parameters listed in Table 2. Throughout this study we chose the average infection period to be 13 days, since this was similar to other published studies (George et al., 2021). Even reducing this period by 1–2 days did not seem to lead to significant changes. However, in the future it would be interesting to investigate model dynamics when we change (increase/decrease) this infection period in a more significant way. This study focused on the dynamics of COVID-19 epidemics in the presence of alpha and beta/gamma strains, with distancing measures (and masks worn indoor and outdoor) but no vaccinations (as the vaccination program in France really took off in May 2021). Our results about the potential large amplitude in the infections with beta and gamma strains, can be understood in this context: these strains would have caused a disastrous outbreak situation if the situation had not changed with the arrival of the vaccine. The subsequent emergence of the delta variant also changed the dynamics of the epidemics, and if we include it into our model (1) the simulation results and predictions will inevitably change. The study can be further extended to the new SARS-CoV-2 variants that are appearing, which might avoid the anti-viral immune responses generated by the vaccination. In regard to this, a very recent review (Malik et al., 2022) discussed vaccine effectiveness in the context of the multiple SARS-CoV-2 variants that emerged over the last months, and concluded that since a large number of variants have mutations mainly associated with the spike protein, which is also a key component of most of the vaccines on the market, vaccine efficacy needs to be assessed for each variant. While some vaccine efficacy studies have been published for earlier variants (Bian et al., 2021), many more such studies are ongoing.

CRediT authorship contribution statement

Mathilde Massard: Conceptualization, Writing – original draft, Writing – review & editing. Raluca Eftimie: Supervision. Antoine Perasso: Supervision. Bruno Saussereau: Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
  23 in total

1.  On the definition and the computation of the basic reproduction ratio R0 in models for infectious diseases in heterogeneous populations.

Authors:  O Diekmann; J A Heesterbeek; J A Metz
Journal:  J Math Biol       Date:  1990       Impact factor: 2.259

2.  Covid-19: The E484K mutation and the risks it poses.

Authors:  Jacqui Wise
Journal:  BMJ       Date:  2021-02-05

3.  A discrete stochastic model of the COVID-19 outbreak: Forecast and control.

Authors:  Sha He; San Yi Tang; Libin Rong
Journal:  Math Biosci Eng       Date:  2020-03-16       Impact factor: 2.080

4.  A pneumonia outbreak associated with a new coronavirus of probable bat origin.

Authors:  Peng Zhou; Xing-Lou Yang; Xian-Guang Wang; Ben Hu; Lei Zhang; Wei Zhang; Hao-Rui Si; Yan Zhu; Bei Li; Chao-Lin Huang; Hui-Dong Chen; Jing Chen; Yun Luo; Hua Guo; Ren-Di Jiang; Mei-Qin Liu; Ying Chen; Xu-Rui Shen; Xi Wang; Xiao-Shuang Zheng; Kai Zhao; Quan-Jiao Chen; Fei Deng; Lin-Lin Liu; Bing Yan; Fa-Xian Zhan; Yan-Yi Wang; Geng-Fu Xiao; Zheng-Li Shi
Journal:  Nature       Date:  2020-02-03       Impact factor: 69.504

5.  Parameter Estimation and Prediction of COVID-19 Epidemic Turning Point and Ending Time of a Case Study on SIR/SQAIR Epidemic Models.

Authors:  Amir Hossein Amiri Mehra; Mohsen Shafieirad; Zohreh Abbasi; Iman Zamani
Journal:  Comput Math Methods Med       Date:  2020-12-27       Impact factor: 2.238

6.  Genomic Evidence of SARS-CoV-2 Reinfection Involving E484K Spike Mutation, Brazil.

Authors:  Carolina K V Nonaka; Marília Miranda Franco; Tiago Gräf; Camila Araújo de Lorenzo Barcia; Renata Naves de Ávila Mendonça; Karoline Almeida Felix de Sousa; Leila M C Neiva; Vagner Fosenca; Ana V A Mendes; Renato Santana de Aguiar; Marta Giovanetti; Bruno Solano de Freitas Souza
Journal:  Emerg Infect Dis       Date:  2021-02-19       Impact factor: 6.883

7.  COVID-19 pandemic and its average recovery time in Indian states.

Authors:  Noel George; Naresh K Tyagi; Jang Bahadur Prasad
Journal:  Clin Epidemiol Glob Health       Date:  2021-04-15

Review 8.  SARS-CoV-2 variants, spike mutations and immune escape.

Authors:  William T Harvey; Alessandro M Carabelli; Ben Jackson; Ravindra K Gupta; Emma C Thomson; Ewan M Harrison; Catherine Ludden; Richard Reeve; Andrew Rambaut; Sharon J Peacock; David L Robertson
Journal:  Nat Rev Microbiol       Date:  2021-06-01       Impact factor: 78.297

Review 9.  The SARS-CoV-2 mutations versus vaccine effectiveness: New opportunities to new challenges.

Authors:  Jonaid Ahmad Malik; Sakeel Ahmed; Aroosa Mir; Mrunal Shinde; Onur Bender; Farhan Alshammari; Mukhtar Ansari; Sirajudheen Anwar
Journal:  J Infect Public Health       Date:  2022-01-05       Impact factor: 3.718

10.  Ratio of asymptomatic COVID-19 cases among ascertained SARS-CoV-2 infections in different regions and population groups in 2020: a systematic review and meta-analysis including 130 123 infections from 241 studies.

Authors:  Xiao Chen; Ziyue Huang; Jingxuan Wang; Shi Zhao; Martin Chi-Sang Wong; Ka Chun Chong; Daihai He; Jinhui Li
Journal:  BMJ Open       Date:  2021-12-07       Impact factor: 3.006

View more
  4 in total

1.  Dynamic of a two-strain COVID-19 model with vaccination.

Authors:  S Y Tchoumi; H Rwezaura; J M Tchuenche
Journal:  Results Phys       Date:  2022-06-30       Impact factor: 4.565

2.  Analysis of multi-strain infection of vaccinated and recovered population through epidemic model: Application to COVID-19.

Authors:  Olusegun Michael Otunuga
Journal:  PLoS One       Date:  2022-07-29       Impact factor: 3.752

3.  A new approach to modeling pre-symptomatic incidence and transmission time of imported COVID-19 cases evolving with SARS-CoV-2 variants.

Authors:  Sam Li-Sheng Chen; Grace Hsiao-Hsuan Jen; Chen-Yang Hsu; Amy Ming-Fang Yen; Chao-Chih Lai; Yen-Po Yeh; Tony Hsiu-Hsi Chen
Journal:  Stoch Environ Res Risk Assess       Date:  2022-09-11       Impact factor: 3.821

4.  The impact of lockdown in Wuhan on residents confidence in controlling COVID-19 outbreak at the destination cities.

Authors:  Xiaoxin Guo; Shihu Zhong; Yidong Wu; Yalin Zhang; Zhen Wang
Journal:  Front Public Health       Date:  2022-08-15
  4 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.