Assessing the Outbreak Risk of Epidemics Using Fuzzy Evidential Reasoning.

Epidemic diseases (EDs) present a significant but challenging risk endangering public health, evidenced by the outbreak of COVID-19. Compared to other risks affecting public health such as flooding, EDs attract little attention in terms of risk assessment in the current literature. It does not well respond to the high practical demand for advanced techniques capable of tackling ED risks. To bridge this gap, an adapted fuzzy evidence reasoning method is proposed to realize the quantitative analysis of ED outbreak risk assessment (EDRA) with high uncertainty in risk data. The novelty of this article lies in (1) taking the lead to establish the outbreak risk evaluation system of epidemics covering the whole epidemic developing process, (2) combining quantitative and qualitative analysis in the fields of epidemic risk evaluation, (3) collecting substantial first-hand data by reviewing transaction data and interviewing the frontier experts and policymakers from Chinese Centers for Disease Control and Chinese National Medical Products Administration. This work provides useful insights for the regulatory bodies to (1) understand the risk levels of different EDs in a quantitative manner and (2) the sensitivity of different EDs to the identified risk factors for their effective control. For instance, in the case study, we use real data to disclose that influenza has the highest breakout risk level in Beijing. The proposed method also provides a potential tool for evaluating the outbreak risk of COVID-19.

INTRODUCTION

Epidemic diseases (EDs) can spread faster and wider than even after their outbreaks in today's community, where more and more megacities of over 10 million population appear and people communicate between cities more frequently thanks to advanced transport techniques. It is often the case that after the outbreak of an ED, it is extremely difficult and costly, if not impossible, to control and eliminate it. It is evident that a few disasters witness spread of EDs at a global scale, including avian influenza in 2006, H1N1 influenza in 2009, dengue fever and H7N9 in 2013, Ebola hemorrhagic fever in 2014, and Middle East respiratory syndrome (MERS) 2015. More specific illustrative examples include the large‐scale epidemic of SARS in 2003 and African swine fever in 2018–2019 in China and global pandemic of COVID‐19 that caused and are causing huge economic loss and social system disorder, leading to a high demand for effective ED risk assessment mechanism.

Since the painful experience caused by SARS in 2003, China has made a series of moves to strengthen the capacity of ED controlling. However, the outbreak of COVID‐19 causes over 1.4 million confirmed cases and over 82,000 death toll (by April 08, 2020), and the numbers are still increasing fast in 2020. The nation‐wide outbreak of epidemic in December 2017 also questions the effectiveness of the current running system, with a shortage of 310,000 packets of the special drug “Duffy” merely in its capital, Beijing. Consequently, it is direly urgent to develop a new proactive risk assessment system to assess ED's outbreak risk and take corresponding prevention actions. One of the innovations of this study is to use primary data obtained through our collaboration with Chinese National Medical Products Administration (CNMPA) to develop a quantitative risk assessment method for evaluating the outbreaks of EDs.

Infectious disease outbreaks can spread rapidly, causing enormous losses to individual health, national economies, and social wellbeing. Through the early detection of an infectious disease outbreak, a small outbreak can potentially be contained at the local level, thereby reducing adverse impacts. Early detection has been and remains the current narrative of infectious disease surveillance. However, it is challenging to assess the outbreak risk of EDs, given that it is affected by a variety of factors, both qualitative and quantitative, such as nature of the pathogen, people's communication style, medical treatment level, weather and natural environment. Besides, limitations to the accessibility to data on certain occasions, some factors cannot be scaled precisely but described in a fuzzy way. For example, people's hygiene awareness is often described by "very good", "good", "fair", "bad" and "very bad". Moreover, domain experts that include all the senior managers of the Ministry of Health, the State Drug Administration, Chinese Center for Disease Control and Prevention (CCDC) and the designated (by the healthcare authorities) experts in the field of public medicine healthcare argue that they have low/medium confidence to make a reliable evaluation without the aid of advanced risk tools that can address the inherent high uncertainty in data. Obviously, the assessment of ED's outbreak risk involves the integration of qualitative and quantitative factors and requires to deal with fuzzy and incomplete information, where a fuzzy evidential reasoning (ER) approach fits in well. The aim of this article is (1) to propose a fuzzy ER assessment approach to the ED's outbreak risk, (2) to apply the proposed approach to assess the outbreak risk of four kinds of major EDs in Beijing.

The remainder of this article is organized as follows. Section 2 reviews the relevant literature on emergent risk assessment methods. Section 3 introduces the evidence reasoning approach and proposes a new index system for the assessment and elaborates the ER based EDRA. Section 4 conducts an empirical study related to ED risk of Beijing using the primary data from the experts of the highest healthcare authorities in China. Section 5 concludes the article with implications and limitations.

LITERATURE REVIEW

There are few studies on risk assessment of ED's outbreak in megacities in the current literature. This section describes the relevant literature of risk assessment in the areas of “humanitarian logistics,” “disaster operations management,” and “outbreak of infectious diseases.”

Humanitarian Logistics

Some studies address the network designing and inventory problems of humanitarian logistics based on cost analysis such as monetary cost, time cost, social cost, etc. For example, by considering relief time, Ahmadi, Seifi, and Tootooni (2015) propose a multidepot location‐routing model to determine the locations of local depots and routing for last mile distribution after an earthquake. By minimizing facility placement, logistics, and deprivation costs, Loree & Aros‐Vera (2018) propose a model to determine the location of points of distribution and inventory allocation in Post‐Disaster Humanitarian Logistics.

However, some others argued that humanitarian should highlight “equity” (e.g., Gutjahr & Fischer, 2018). Once there is a disaster, relief commodities are to be supplied to the victims of a natural disaster, aid the organizations that not only have to take the total degree of demand satisfaction into account, but also require relief goods to be distributed as equally as possible among the affected population. Ideally, no region or population group should be disadvantaged (Farris, 2010; Yitzhaki & Schechtman 2013).

Substantial studies on humanitarian logistics have been published, which mainly focused on transportation and allocation of disaster relief resources, taking into account the goal of equity (e.g., Huang, Jiang, Yuan, & Zhao, 2015; Lin, Batta, Rogerson, Blatt, & Flanigan, 2011; Ransikarbum & Mason, 2016; Tomasini & Van Wassenhove, 2009). For example, a fuzzy multicriteria, multiperiod linear programming model for relief distribution was presented in Tzeng, Cheng, & Huang (2007). They measured the equity of distribution of relief materials to demand nodes by cumulating the least satisfaction score among demand points for each item and for each time period. Moreover, Sun, DePuy, & Evans (2014) dealt with a patient–hospital allocation model for the case of an influenza pandemic. Equity was taken into account by considering the maximal travel distance to the assigned hospital as a second objective function in a biobjective optimization model. The risk assessment of the outbreak of infectious diseases in this article also involved the principle of “equity,” focusing on the infectious diseases affecting public health.

Disaster Operations Management

Research about disaster operations management mainly focuses on “natural disaster,” “war or terrorism,” but rarely on the management of ED. Disaster operations management can be divided into resource/facilities management, response management, and risk management (e.g., Al‐Dahash, Kulatunga, & Amaratunga, 2014; Tsadikovich, Kamble, & Elalouf , 2019).

At first, several studies aimed to develop and increase the ability and facilities to cope with disasters and to mitigate some of the effects in order to minimize the consequences of the disaster (e.g., Al‐Dahash, Kulatunga, & Amaratunga, 2014; Harding, 2007). Furthermore, the case of the Katrina hurricane was considered by Baker & Refsgaard (2007) to identify successful strategies that enabled institutions to respond effectively at an appropriate scale. Finally, the article by Carreño, Cardona, & Barbat (2007) proposed a risk management index. A group of indicators were brought together to measure risk management performance and effectiveness. Organizational and developmental capacity and institutional actions were taken to reduce vulnerability and losses in a given area. Such factors were reflected by these indicators to prepare for a crisis and to recover efficiently from disasters. This index was designed to assess risk management performance. Four public policies, include the identification of risk, risk reduction, disaster management, and governance and financial protection were constructed by the proposed risk management index. Although there are few studies on the risk management of ED, research on disaster operations management provides theoretical support for the construction of the ED risk assessment model in this article.

Outbreak of Infectious Diseases

Regarding the outbreak of infectious diseases, there has been more studies in the medical field, focusing on the factors affecting the transmission of pathogens, propagation mode, and the control of infectious diseases (Lal et al., 2018). Some found that global environmental changes, especially climate change and human exploitation of productive ecosystems (e.g., Tilman, Balzer, Hill, & Befort, 2011) had important implications for infectious disease risk (Cutler, Fooks, & Van, 2010; Murray & Daszak, 2013). As for the propagation mode, the parasites are primarily spread through contaminated drinking or recreational water; however, infection in humans may arise through contaminated food, contact with animals, especially livestock or infected individuals (Cacciò, Thompson, McLauchlin, & Smith, 2005; Savioli, Smith, & Thompson, 2006). With regard to diseases control, a few studies identified that the environmental and sociodemographic exposures can help develop disease control priorities in high‐risk areas (Lal, Cornish, Fearnley, Glass, & Kirk, 2015; Yoder & Beach, 2010).

In addition to these theoretical studies, practitioners have also made many outstanding contributions to the investigation of the outbreak of infectious diseases. Innovative governance structures have been established to promote early detection. Disease surveillance networks have been formed, such as the World Health Organization (WHO), Global Outbreak Alert and Response Network (GOARN), combining human and technical resources around the world to rapidly identify, confirm, and respond to outbreaks. Cross‐border regional disease surveillance networks have been established across the globe, connecting epidemiologists, scientists, ministry officials, health workers, border officers, and community members to engage in activities, such as training, capacity‐building, and multidisciplinary research. Agreements have been instituted, setting legal mandates around surveillance activities, such as the IHR (2005), which call for all the WHO member states to build, improve, and strengthen their capacity to prevent, detect, and respond to infectious diseases outbreaks that can have global spread (Steele, Orefuwa, & Dickmann, 2016).

To the best of our knowledge, the risk assessment of ED's outbreak is still an open task, owing to the high complexity and requirements of integrating qualitative and quantitative factors and dealing with uncertainty in data. Fuzzy ER presents an effective solution to dealing with the above requirements. Therefore, this article aims to use a fuzzy ER approach to develop a new risk assessment model for evaluating the ED's outbreak in megacities.

RISK ASSESSMENT OF EDS USING FUZZY EVIDENTIAL REASONING

As far as the risk assessment of ED's outbreak is concerned, some factors cannot be precisely measured but often can be described by subjective judgements using linguistic terms. Such linguistic terms are then modeled by with fuzzy membership functions to address their discretization. Linear (e.g., triangular and trapezoidal) fuzzy membership functions are widely used to describe the linguistics variables because of their simplification (Wang, 1997). A triangular fuzzy number is a fuzzy set with three parameters (a, b, c), where a is the membership function's left intercept (lower boundary) with a grade of 0 and c is the membership function's right intercept (upper boundary) with a grade equal to 0, while b is the value of having the highest membership with a grade equal to 1 (Pam, Li, Wall, Yang, & Wang, 2013). Fuzzy logic provides experts with a wide scope instead of a precise point to define the risk grade at which a key risk indicator (KRI) may contribute to a specific epidemic disease.

Fig. 1 visually depicts the proposed methodology composed of three major steps. The first step is to establish a fuzzy link‐based KRI hierarchy, which can offer a generic platform for carrying out the risk assessment of a particular place under the outbreak of any specific ED. The developed generic framework can be appropriately modified with reference to the disease's specifications, involving less or more KRIs. The second step is to analyze the risk level of a targeted area facing the impact of the specific disease.

Fig 1

The methodology of EDRA using fuzzy evidential reasoning.

The evaluation of KRIs is a dynamic process depending on different EDs. All the estimations of the identified KRIs can then be aggregated using an ER approach to obtain an assessment result. It will then be measured by a risk threshold indicating whether the criticality of the outbreak under an ED is acceptable. The risk index value is obtained through aggregating the marginal risk estimation of each KRI. Otherwise, all the acceptable risk assessment results obtained from different impact factors and from different conditions will be aggregated to assess the overall risk level of an ED. The overall risk level can then be used as a benchmark to improve the level of prevention and monitoring of infectious diseases in CCDC, as well as a theoretical reference for the government to regulate and control drug reserve (Comfort, 2002). Details about ER can be found in the book (Lee & Yang, 2018) providing a comprehensive review.

Establishment of a Fuzzy Link‐based Hierarchy

This section extracts the KRIs for the risk assessment from the four stages of the ED's outbreak process, including prevention stage, propagation stage, resistance stage, and post‐control stage. We obtained the final indicator system in Table I by interviewing 10 experts. Out of these experts, six are from public healthcare authorities, three from the Chinese Food and Drug Administration, three from the Chinese Center for Disease Control and Prevention (CCDC). The other four are the domain frontline experts from Chinese PLA General Hospital and People's Hospital of Peking University, two top‐level medical institutions in China. We first drafted a set of indicators referring to the existing studies and some national regulations like National Regulations on Emergency Response to Public Health Emergencies. Then, we presented the draft to the experts and asked them to make adjustment to the draft including adding new ones, alter the existing or drop the unsuited. The feedbacks from the experts are aggregated and sent back to the experts again for second round improvement. After two rounds, we finalized the indicator system by dropping those rejected by more than six experts. Appendix 2 presents the mapping from indicators in Table I to corresponding references.

Table I

The Hierarchy of KRIs (R – Inhibiting level; P – Stage level; I – Indicator level)

Code	Key indicators (KRIs)	KRIs linguistics grades
R	Capacity of inhibiting diseases	Very high	High	Low	Lowest
R‐P1	Prevention stage	Good	Average	Fair	Poor
R‐P1‐I1	Have corresponding effective vaccines	Full	Partial	Poor	Barely	No
R‐P1‐I2	Personal hygiene awareness	Very good	Good	Average	Bad	Very bad
R‐P1‐I3	The pathogenicity of the pathogen itself	Very high	High	Average	Low	Very low
R‐P2	Propagation stage	Good	Average	Fair	Poor
R‐P2‐I1	Speed of pathogen transmission	Very fast	Fast	Average	Slowly	Very slowly
R‐P2‐I2	People's anti‐disease capacity	Very high	High	Average	Low	Lowest
R‐P2‐I3	Mortality rate	Very high	High	Average	Low	Lowest
R‐P3	Resistance stage	Good	Average	Fair	Poor
R‐P3‐I1	Relevant stocks of drugs/vaccines	Very sufficient	Sufficient	Average	Inadequate	Very inadequate
R‐P3‐I2	The possibility of death and serious sequelae after infection in healthy people	Very likely	likely	Average	Unlikely	Very unlikely
R‐P3‐I3	Have good diagnosis and control ability	Strongly agree	Agree	Not sure	Disagree	Strongly disagree
R‐P3‐I4	The effectiveness of the treatment	Very good	Good	Average	Bad	Very bad
R‐P4	Post‐control stage	Good	Average	Fair	Poor
R‐P4‐I1	Have comprehensive prevention and control plan	Full	Partial	Poor	Barely	No
R‐P4‐I2	The level of drug stocks available	very high	High	Average	Low	Lowest
R‐P4‐I3	The ability of the government to regulate the supply of drugs	Very high	High	Average	Low	Lowest
R‐P4‐I4	Targeted science popularization work	Very good	Good	Average	Bad	Very bad

It is notable that the KRIs in Table I, the value of R denotes the capacity of inhibiting diseases, thus a lower value of R implies higher outbreak risk.

The real meaning of risk in practice is varied and specific. Therefore, it is necessary to measure the risk of using KRI level descriptions of different units in common space. Triangular fuzzy membership functions developed on a [0−1] utility domain have been widely used to define different risk grades/descriptors (Yang, Bonsall, & Wang, 2009), which include four to five linguistic terms (Table I). The typical membership functions for four to five linguistic variables are defined and characterized (Table II).

Table II

Fuzzy Membership Functions of KRI Grades

Number of grades	4	5
Triangular fuzzy membership functions	(0, 0, 0.3), (0.2, 0.4, 0.6), (0.4, 0.6, 0.8), (0.7, 1, 1)	(0,0, 0.3), (0.1, 0.3,0.5), (0.3, 0.5, 0.7), (0.5, 0.7, 0.9), (0.7, 1, 1)

There is some flexibility in the definition of membership functions to accommodate different risk situations. However, it is worth noting that any change in the defined member functions requires very cautious reasons from domain experts (Yang, Ng, & Wang, 2014). New definitions of memberships need to be verified through empirical tests before being applied in practice. Once the fuzzy memberships are defined, the grades of the lowest level KRIs will need to be transformed and presented by the ones of the top level KRI so as to aggregate the risk estimations of all the KRIs on the same plate. Fuzzy similarity calculation can be used to model the transformation process, as follows (Yang et al., 2014): where x covers the domain [0,1] of the fuzzy memberships and of the lower level KRI grades A and the upper level KRI grades A, ; presents the similarity degree to which A belongs to A. To keep the completeness of risk estimation is normalized into b, as follows:

Therefore, all KRI scores at the lowest level can be converted and presented through the top KRI scores (Good, Average, Fair, Poor). An example of transforming KRI “R‐P1‐I2” to “R” can be found in Fig. 2. The belief degrees assigned against each link in the figure is calculated by Equations (1) and (2).

Fig 2

The transformation of KRI grades.

Risk Assessment of Major Infectious Diseases

In the face of different infectious diseases, the risk of a particular region will vary. Therefore, disease is identified in a region. When identifying specific diseases in a region, the risk level can be assessed by considering two types of estimates. One of them is to estimate the risk of all KRIs at the lowest level (level I), and the other is to assign weights to all relevant KRIs. For a specific disease, not all KRIs within the hierarchy have the same importance/weight. Some of them may be less (or even less) influential than others. The weights of all KRIs are dynamic and affected by identified diseases, so a set of KRI weights is needed for each region against a specific disease. Here Analytic Hierarchy Process (AHP) (Saaty, 1998) can be repeatedly used to assign the weights w, of KRIs (Yang, Ng, & Wang, 2013).

After analyzing the weight allocation, the KRI risk assessment for each region–disease pair becomes simple and clear. To present assessors’ uncertainty when estimating KRI I, (4 or 5), a set of probabilities p is introduced correspondingly. For instance, of the KRI "R‐P1‐I1" can be estimated as { Full, Partial, Poor, Barely, No}. Such estimation can be transformed through in Equation (2) and the pathways in Fig. 2 and expressed based on the four linguistics grades of R using Equation (3): where indicates the probabilities assigned to the grades in the lower‐level indicator; means the normalized similarities between the grades of lower and upper levels (see Equation (2)); represents the transformed probabilities assigned to the grades in the upper‐level indicator. Assume that and indicate the relative weights associated with the lower and upper‐level indicators, respectively. Then, where represents the importance of the ith indicator at the lowest level in the process of synthesizing all the relevant lower‐level indicators to their common (the upper level jth) upper‐level indicator. Equations. (3) and (4) can be repeatedly used to transform the risk estimation and importance of each indicator at the lowest level to the top‐level R. We note that the sum of all the associated with the “R” is equal to one.

As the risk estimations and weights of all the KRIs at the lowest level are transformed to their counterparts and at the “R” with respect to one region–disease pair (k = (1, 2, …, L), where L means the total number of all the KRIs at the lowest level. To capture the nonlinear relationship between all the KRIs at the lowest level, the ER approach (Yang & Xu, 2002a) is used to combine all (transformed from each KRI at the lowest level) and generate a risk estimation of the region–disease pair. First, it is required to transform into basic probability masses using Equations (5)–(8) (Yang & Xu, 2002a): where each is a degree to which each KDI supports the final synthesized estimation D (inhibition level in Table I); each represents the relevant importance of the kth KRI and thus ; and for all k = (1, 2, …, L). The probability mass of the kth KRI () unassigned to the final synthesized estimation D, which is unassigned to any individual grades D (j = 1, 2, 3, 4), is split into two parts, one caused by the relative importance of the kth KRI (), and the other due to the incompleteness of the belief degree assessment .

In view of the above, it is possible to aggregate all the risk estimation from each indicator to generate the combined degree of belief () in each possible D of D. Suppose is the combined belief degree in D by aggregating all and is the remaining belief degree unassigned to any D. Let and . Then, the overall combined belief degree in D is generated as follows: where indicates the normalized belief degree assigned to D in the final synthesized estimation D and represents the normalized remaining belief degree unassigned to any D. To facilitate the calculations involved in Equations (5)–(15), IDS Software has been developed through a user‐friendly interface by Yang and Xu (2002b). The risk estimation result for the region–disease pair can then be expressed as { Highest, High, Low, Lowest, Unknown}. To measure the acceptance of such risk result, D needs to be given utility values for a crisp risk result R and require to be assigned back to and for the possible best risk R and the possible worst risk R. Consequently, where each U (j = 1, 2, 3, 4) represents the utility values of D, which can be calculated using a centroid defuzzification method as the set of {0.1, 0.4, 0.6, 0.9} from Table II. Such defuzzied utility values can be used as the criteria to define the three risk levels (Yang, 2001), as follows:

When R < 0.4, the risk level of the region–disease pair is unacceptable at a normal situation. Additional risk control measures should be developed and adopted to the KRIs with the highest risk contributions. Although this strategy seems to be debatable on the perspective of cost–benefit analysis, it is actually reasonable given that cost is not the first concern in a pandemic.

When , the risk level is acceptable. When the risk levels of all the region–disease pairs related to a terminal become acceptable, the risk level of the terminal can be obtained through synthesizing all the risk levels of the involved region–disease pairs for risk improvement purposes.

and obtained from Equations (14) and (15) are based on a specific region–disease pair by an expert. In practice, the risk level of a certain area with respect to all the possible diseases and disease response capacity in specific regions are likely to be assessed by multiple experts. Therefore, the risk estimations from multiple experts and all the region–disease pairs are synthesized using the ER approach. If there are different risk results from multiple experts with respect to a specific pair, then in Equation (5) means the estimation from the kth expert. The weight indicates the role/importance that the expert plays compared to the other experts.

If all the region–disease pairs associated with a particular area under different disease are combined to calculate the risk level of the facility, then in Equation (5) represents the estimation from the kth region–disease pair. The weight is determined by taking into account both likelihood and consequence through a simple risk matrix approach of each disease. Similarly, if the risk estimations of all the facilities are synthesized to calculate the risk level of a certain area, then in Equation (5) represents the estimation from the kth type of disease under investigation. The corresponding R can be used a benchmark to monitor the risk control of an area under the outbreak of diseases in a longitude study, as well as to priories all the region–disease pairs to identify the risk vulnerability for optimal risk improvement.

Development of a Decision‐making Tool to Select the Infectious Disease for Prevention

In order to further analyze and verify the key factors affecting the outbreak of EDs, such as “which factor should attract the most attention by healthcare authorities,” we will conduct a sensitivity analysis. Decision makers are often interested in examining how much changes in attribute weights and alternative performance on an attribute can affect the overall ranking of alternatives. On top of score ranges, there are three other types of sensitivity analyses in ER and its associated computing software IDS: Change Weight, Change Input Data, and Trade Off Analysis. In this article, in order to obtain the key means to suppress the outbreak of infectious diseases, the method of change weight is selected to conduct sensitivity analysis.

At present, the scope of drug and medical device reserve in China is mainly general accidents and disasters. However, in recent years, with the change of natural environment and ecosystem, as well as the increase of international exchanges of personnel and materials, the incidence and scale of new infectious diseases and infectious diseases in overseas epidemic areas have been continuously increasing. The incidence of unconventional epidemics and new infectious diseases is increasing, and epidemics and diseases beyond the control of the use of existing drugs and medical devices occur. At the same time, due to the shortage of supply, there is no domestic production capacity or insufficient production capacity, in the emergency response process, the event handlers often require the use of imported products on the premise of having domestic products. The inaccurate forecasting of epidemic trend and scale leads to the inability of reserve varieties and quantities to reflect emergency needs. Through sensitivity analysis, the key factors affecting the suppression of infectious disease outbreaks in the region can be obtained, which will provide useful insights for CCDC and other relevant departments in disease prevention, response and control decision‐making.

EMPIRIEAL STUDY OF EDS IN BEIJING

Description of Problem

As the capital of China, Beijing is the political, economic, and cultural center of China, which has a dense population and the number of permanent residents ranks 1st in the country. Taking Beijing's public health emergency drug reserve as the object of empirical research has important theoretical value and practical significance for emergency measures of public health emergencies in megacities.

The aforementioned expert panel recommend four diseases worth careful investigation in light of their background knowledge, including influenza, human infection with highly pathogenic avian influenza (Avian Influenza), hand, foot and mouth disease (HFMD), and tuberculosis. Subsequently, we take Beijing as an example to explore the risk of the outbreak of the four infectious diseases. Firstly, based on the risk assessment system in Section 3, a questionnaire survey was conducted with the panel of experts. On the one hand, the scientific nature of the assessment system will be tested by the survey data, on the other hand, the data will be substituted for evidence reasoning method to make the risk assessment. Finally, we will conduct a sensitivity analysis on the key indicators affecting the risk of infectious disease outbreak.

Question Design

The indicators used in this study are evaluated by a subset of perceptual items in the outbreak of a disaster. First, we adopted indicators that had been validated by previous studies. Second, when indicators had not been well documented in the literature, we develop new indicators based on our observations and face‐to‐face interviews with the senior managers in the Ministry of Health, CNMPA, and CCDC.

The survey instruments included questions on the basic information of respondents and evaluation of four main infectious diseases in four stages. In the prevention stage, we evaluate the risk in terms of effective vaccines (Lopes, Okino, Fernando, Pavani, & Montassier, 2018; Lucero et al., 2019; Wiederman, 2017), personal hygiene habits (Pehlivan et al., 2011; Zivich, Gancz, & Aiello, 2018), and pathogens (Xue et al., 2019; Yanagihara, Matsumoto, Aoki, & Sato, 2019); in the propagation stage, in terms of transmission speed (Ding & Wang, 2019), human beings’ resistance and mortality rate (Emam, Livernois, Paibomesai, Atalla, & Mallard, 2019; Rao & Ayres, 2017); in the resistance stage, in terms of diagnosis and control of diseases, storage of drugs/vaccines, effectiveness of the treatment, and possibility of death (Arji et al., 2019; Odedra, Lalloo, Kennedy, Llewellyn, & McCarthy, 2019; Vaughn, Salas‐Wright, Cohen, & Holzer, 2019); and in the post‐control stage, in terms of the level of drug stocks available, the ability to regulate the supply of drugs and targeted science popularization work (Spencer et al., 2019; Tong et al., 2016). On the four stages affecting the outbreak of infectious diseases, all indicators were computed on a five‐point Likert scale and are listed in Appendix 1, with “1” indicating “strongly disagree” and “5” indicating “strongly agree.”

Sample Selection

Through reliable research collaboration, we managed to survey CCDC, Chinese Drug Administration and the Ministry of Health using anonymous questionnaire in the period from 1st Oct to 31st Oct, 2018. Four weeks were given to each respondent, and every two weeks was considered as one period. We then sent reminding emails to the respondents when the first period passed. By the end of the fourth week, 76 respondents had responded to the survey. On the basis of a comprehensive check of the returned answers, 55 questionnaires were valid, translating to a response rate of 72.3%. The samples cover all the senior managers of the Ministry of health, the State Drug Administration, the CCDC, and the designated (by the healthcare authorities) experts in the field of public healthcare. The good response rate and comprehensive coverages of the stakeholders ensure the representativeness of the sample data. Profiles of the respondents and the organizations are summarized in Table III.

Table III

Profile of Respondents

Characteristics	Percentage (%)
Respondent's position
Senior managers	13
Middle managers	37
General staff	35
Medical experts	15
Age of respondents
<30	15
31−40	37
41−50	33
>50	15
Organisations
CCDC	13
Food and Drug Administration	30
National Ministry of Health	12
Local Health Bureau	45

Analysis Results

According to Armstrong & Overton (1977), nonresponse bias should be tested based on the differences of measured items between early and late responses. In this study, 19 answers were collected at the end of the first period and 36 questionnaires were returned during the second period. By comparing the late responses obtained after a reminder email with early responses on the measured items and demographic variables, the results of t‐tests indicate no significant differences between the responses in the two periods, suggesting that nonresponse bias was not a concern in this study. At the same time, we test the reliability of the data by using the Cronbach's alpha, and the result of this test is 0.786, showing a good reliability of the dataset. In addition, to ensure the suitability of collected data for the factor analysis, a Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett's test were conducted (Eltayeb, Zailani, & Ramayah, 2011). The overall result of the KMO and Bartlett's test for the dataset is 0.873 and the minimum value of a single variable reached 0.753, which is for sphericity, showing a satisfactory result when p < 0.000. The results indicate that the correlation between variables were strong and the data set structure was robust.

Principal components analysis (PCA) is employed by using SPSS on 14 measured variables with VARIMAX rotation to extract the crucial factors. An eigenvalue greater than 1 was used to determine the number of factors (Churchill, 1979). For simplicity, only loadings of 0.50 or higher were extracted (Hair, Anderson, Tatham, & Black, 1998). Therefore, these 14 variables were loaded on four factors (shown as Table IV). The cumulative variance explained by the four factors was 0.857 and the alpha value of each factor group was higher than the suggested threshold of 0.70 (Skipper & Hanna, 2009), which indicates a good construct validity. Moreover, it shows that the factor model produced by factor analysis was largely consistent with the theoretical one.

Table IV

Results of PCA

Item	Factor 1	Factor 2	Factor 3	Factor 4
R‐P1‐I1	0.693
R‐P1‐I2	0.768
R‐P1‐I3	0.643
R‐P2‐I1		0.794
R‐P2‐I2		0.836
R‐P2‐I3		0.778
R‐P3‐I1			0.762
R‐P3‐I2			0.775
R‐P3‐I3			0.837
R‐P3‐I4			0.830
R‐P4‐I1				0.891
R‐P4‐I2				0.878
R‐P4‐I3				0.646
R‐P4‐I4				0.642
Mean	3.214	1.578	3.567	3.923
S.D.	0.887	1.166	0.862	0.773
Eigenvalue	5.002	4.888	4.549	4.028
Cronbach's α	0.846	0.944	0.929	0.924

Next, the fuzzy link base hierarchy in Table I is directly employed in this section (Step 1). The risk assessment of the four diseases starts with the analysis of the KRI weights with respect to each region–disease pair and the risk estimation of each KRI at the lowest level (Step 2). AHP is first used to assign the four parameter (P1–P4) weights in the influenza analysis in Table V. In a similar way, the relative weights of all other KRIs at the lowest level are assigned (Table V).

Table V

Parameter Pairwise Comparison Matrix in Terms of Influenza

P	#1	#2	#3	#4	Weight	Consistency ratio (CR)
#1	1.00	4.00	1.50	1.00	0.34	0.0821
#2	0.25	1.00	0.30	0.20	0.08
#3	0.67	3.33	1.00	0.90	0.26
#4	1.00	5.00	1.11	1.00	0.33

Next, the probabilistic risk estimation of each KRI using the defined linguistics grades is evaluated in the context of influenza by the panel of experts mentioned in Section 3—Step 1 (Table VI). We adopt a three‐round Delphi algorithm to narrow the range of experts’ idea, and then average them into a unified one. Using Equations (1)–(3), such probabilistic risk estimations can be transformed and expressed by the four linguistic grades of the R at the top level (Table VI). Based on Equations (4)–(15), all the transformed risk estimations of the KRIs can be synthesized using the ER approach.

Table VI

KRIs’ Weights and Risk Estimations in The Context of Influenza. Italic Values Show the Weights of R‐Ps (Which Are at The Same Level)

Code	Weights	Probabilistic risk estimation of each KRI						Transformed risk estimation at the R level
R‐P1	0.34							Good	Average	Fair	Poor
R‐P1‐I1	0.40	0.3 Full	0.3 Partial	0.2 Poor		0.1 Barely	0.1 No	0.64	0.25	0.11	0
R‐P1‐I2	0.30	0.4 Very good	0.3 Good	0.3 Average				0.75	0.25
R‐P1‐I3	0.30	0.4 Very High	0.3 High	0.2 Average		0.1 Low		0.77	0.23	0
R‐P2	0.08
R‐P2‐I1	0.30	0.55 Very fast	0.25 Fast	0.15 Average			0.05 Very slowly	0.82	0.10	0.08
R‐P2‐I2	0.30	0.3 Highest	0.2 High	0.2 Average		0.2 Low	0.1 Lowest	0.60	0.40
R‐P2‐I3	0.40	0.3 Highest	0.3 High	0.2 Average		0.2 Low		0.60	0.40
R‐P3	0.26
R‐P3‐I1	0.30	0.5 Very sufficient	0.3 Sufficient	0.2 Average				0.60	0.40
R‐P3‐I2	0.25	0.35 Very likely	0.25 Likely	0.25 Average		0.15 Unlikely		0.85	0.15
R‐P3‐I3	0.25	0.35 Strongly agree	0.25 Agree	0.25 Not sure		0.1 Disagree		0.81	0.10	0.09
R‐P3‐I4	0.20	0.5 Very good	0.3 Good	0.2 Average				1
R‐P4	0.33
R‐P4‐I1	0.30		1 Partial					0.75	0.25
R‐P4‐I2	0.30		1 High					0.75	0.25
R‐P4‐I3	0.30	0.5 Highest	0.5 High					0.75	0.25
R‐P4‐I4	0.10	0.5 Very good	0.5 Good					0.75	0.25

IDS software package is used to facilitate the synthesis process with the final combined risk estimation result of the influenza obtained as

R(influenza) = {0.3533 Highest; 0.3432 High; 0.2653 Low; 0.0382 Lowest}.

The above result can be interpreted as that, given the risk input of each KRI in Table VI, the risk level of the influenza pair is estimated as the highest with 35.33% degree of belief, high with 34.32% degree of belief, low with 26.53% belief degree, the lowest with 3.82% degree of belief. This result is visually presented in Fig. 3.

Fig 3

The risk level of influenza.

The above results obviously indicate that, to a rather large extent, the KRIs have been assessed as the highest or the second highest grade. For example, R‐P1‐I1 has been assessed as "Full" with a belief of 30%; R‐P2‐I1 has been evaluated to a significantly larger extent as "Good" with 30%. Since the risk level of a region–disease pair is determined by the risk performance of each basic indicator, the top risk level should be evaluated as "Highest" to a large extent. This is harmonious with the results obtained above as the risk level of influenza has been assessed as "Highest" to the extent of 35.33%. The acceptance of the risk level could be measured by Rc using Equation (16) and the result is as follows:

R = 0.20402 < 0.4 implies that the risk level of influenza is worth special vigilance. Similarly, the risk levels and their crisp utility values of the other three diseases can be obtained (Table VII). The results indicate that the risk levels of the other three infectious diseases are needed heightened vigilance, while the smaller the inhibitory effect on influenza, the greater the risk of outbreak (can see in Fig. 4).

Table VII

The Risk Levels of The Four Typical Infectious Diseases

Infectious diseases	Risk estimation	Crisp utility value
Influenza	0.3628 Highest; 0.5331 High; 0.0857 Low; 0.0164 Lowest and 0 Unknown	0.20402
Avian Influenza	0.7051 Highest, 0.1566 High, 0.1193 Low, 0.0190 Lowest and 0 Unknown	0.55887
HFMD	0.5955 Highest, 0.2819 High, 0.1298 Low, 0.0234 Lowest and 0 Unknown	0.52174
Tuberculosis	0.5845Highest, 0.2758High, 0.1143 Low, 0.0199Lowest and 0.0056Unknown	0.51035

Fig 4

The risk level of infectious diseases‐inhibition.

Fig. 5 shows the scoring of four infectious diseases in four stages, which indicates that in the prevention stage and resistance stage, the differences of four infectious diseases are small, while in the propagation and post‐control stages, the scoring differences are large. In order to further analyze the key factors to inhibit the risk of infectious diseases outbreak, the sensitivity analysis of the indicators is made in the ensuing section.

Fig 5

Risk level comparison of four diseases in various stages.

Sensitivity Analysis

We invited the aforementioned 10 experts to provide the ranges of the weights of KRIs, considering each of them has at least 8 years of serving the governmental departments or medical institutes on the infectious disease. We formulate and adopt the overlapping range in the sensitivity analysis: the range of the weight of R‐P2 (indicators in the propagation stage) is [0.07, 0.61], while R‐P4 (indicators in the post‐control stage) [0.2, 0.8]. The results of sensitivity analysis are shown in Figs. 6 and 7, from which the following implications can be drawn:

Fig 6

Sensitivity analysis on the weight in propagation stage.

Fig 7

Sensitivity Analysis on the Weight in post‐control stage.

First, as shown in Fig. 6, the weights of indicators in the transmission stage impose a significant impact on the results. Along with the weight increasing from 0.07 to 0.61, tuberculosis and avian influenza transpose their rankings in terms of inhibition level, which implies the ranking of outbreak risk change reversely; besides, influenza and HFMD drop a lot though their rankings remain unchanged. This indicates that the weight in the transmission stage should be determined with much cautiousness. To achieve this goal, integration of expert opinions, like in this study, could be a good option.

Second, as depicted in Fig. 7, the weight of R‐P4 (indicators in the post‐control stage) has no significant impact on the risk estimation of influenza, HFMD and tuberculosis, but relatively bigger influence on avian influenza. With the weight increasing from 0.2 to 0.8, of the first three diseases, the rankings keep unchanged and the inhibition levels gain only slight increase. In terms of avian influenza, the ranking keep unchanged, though the inhibition level gains bigger increase. This implies that the empirical result is robust on the weight of R‐P4.

DISCUSSION AND CONCLUSION

The article reviews the current status of risk control process to identify the need of developing a quantitative risk assessment method. Following the development of a novel risk assessment method, a case study and sensitivity analysis are undertaken to validate the results. The unique contributions of this study comprise (1) constructing a new hierarchical system of indicators for outbreak risk evaluation, based on previous studies and the first‐hand practical experience of the Chinese authoritative infectious disease control institutions like Ministry of Health, CNMPA and CCDC; (2) proposing a general methodology of outbreak risk evaluation based on fuzzy evidential reasoning, which could be easily extended to other areas or diseases. The proposed system of indicators thoroughly covers key elements in successive stages of fighting a disease including prevention stage, transmission stage and resistance stage, involving human, disease and preventive measures. By adopting fuzzy evidential reasoning, the proposed EDRA methodology is enabled to integrate the valuable expertise and quantitative analysis for more successful disease outbreak risk evaluation.

The development of a quantitative EDRA methodology capable of realizing the integrity of risk identification, analysis, control and security improvement will enjoy substantial benefits, including a consistent regime which addresses all the aspects of infectious disease in a certain area through an integrated and quantitative manner; a rational basis that prevention control and regulatory requirements are in proportion to the severity of the risks; a proactive approach, enabling threats that have not yet given rise to tragedies to be properly considered. To further appreciate the benefits, the applications of the developed methodology in more real cases will be carried out. The method will be tailored to deal with specific requirements of CCDC or other healthcare authorities in a wide context (i.e., Other public health events in a certain area) in order to reinforce its validity and flexibility. To conclude, this study offers substantial value to scholars, planners and policymakers in the design, planning and evaluation of effective risk control measures and strategies, thus reducing the outbreak risk of infectious disease in megcities.

Moreover, the developed model is designed to deal with EDs in a generic manner. It therefore shows potential significance for tackling the risk analysis of COVID‐19. Within this context, the model can be used at different scales, referring to a community, a city, and a regional/country. The uniqueness of this model lies in the fact that it covers all the stages of the pandemic development process, which enables it to evaluate the risk of the countries suffering from COIVD‐19, as well as, the outbreak risk of COIVD‐19 in the other countries where the large‐scale outbreak of COVID‐19 has not occurred yet. Furthermore, while this model can aid risk analysts and policymakers to make rational decision based on the final overall risk evaluation score, the score against each specific KPI can also provide useful insights into the response system against COIVD‐19 and facilitates the exploration of the deficiency of the system. It offers an actionable method to enhance the effects of fighting the pandemic.

Furthermore, although the world has put much effort to fight COIVD‐19, there is a scarcity of interdisciplinary studies on how COIVD‐19 reacts with other social systems. For example, how the logistics system can adapt to large‐scale epidemic outbreak. Hence, by focusing on COVID‐19, the proposed model can be tailored to develop effective logistics strategies and planning in tackling the outbreak risk caused by sudden, large‐scale epidemic outbreak and to enhance the resilience of emergency logistics chains (e.g., medical and food). It can strive to achieve two interactive analysis results to provide insightful solutions to COVID‐19: outbreak risk assessment of COVID‐19 and logistics adaptive responses. It can aid to investigate and define the key factors influencing the outbreak risk (e.g., both likelihood and consequence) of COVID‐19 in a country/society and to develop a novel dynamic multiobjective optimization model for logistics planning to support societies with respect to their dynamic COVID‐19 risk levels.

Although this study contributes to both academics and practice, there are still some limitations. For example, by interviews and questionnaires, the indicators in this study are selected by surveying infectious disease institutions located in Beijing, subsequently more indicators should be considered when it comes to other areas.

ACKONWLEDGMENTS

This study is supported by the National Natural Science Foundation of China (Grant number 71831001), the international cooperation project with Liverpool John Moores University “Emergency logistics response to outbreak risk of COVID‐19: Phase 1 risk analysis” and Beijing Logistics Informatics Research Base.

Table A1 References for The Indicators in Table I

R‐P1	Prevention stage References
R‐P1‐I1	Have corresponding effective vaccines	The State Council of The People's Republic of China (2003)
R‐P1‐I2	Personal hygiene awareness	Aiello, Coulborn, & Perez (2008)
R‐P1‐I3	The pathogenicity of the pathogen itself	National Health of Commission of The People's Republic of China (2006)
R‐P2	Propagation stage References
R‐P2‐I1	Speed of pathogen transmission	Vescovi, Rebetez, & Rong (2005)
R‐P2‐I2	The level of immunity/resistance of the population to the disease	Fisman, Leung, & Lipsitch (2014)
R‐P2‐I3	Mortality rate	National Health of Commission of The People's Republic of China (2006)
R‐P3	Resistance stage References
R‐P3‐I1	Relevant stocks of drugs/vaccines	National Health of Commission of The People's Republic of China (2006)
R‐P3‐I2	The possibility of death and serious sequelae after infection in healthy people	National Health of Commission of The People's Republic of China (2004)
R‐P3‐I3	Have good diagnosis and control ability	National Health of Commission of The People's Republic of China (2004)
R‐P3‐I4	The effectiveness of the treatment	National Health of Commission of The People's Republic of China (2004)
R‐P4	Post‐control stage References
R‐P4‐I1	Have comprehensive prevention and control plan	The State Council of The People's Republic of China (2003 National Health of Commission of The People's Republic of China (2004) National Health of Commission of The People's Republic of China (2006)
R‐P4‐I2	The level of drug stocks available	National Health of Commission of The People's Republic of China (2006)
R‐P4‐I3	The ability of the government to regulate the supply of drugs	National Health of Commission of The People's Republic of China (2004) National Health of Commission of The People's Republic of China (2006)
R‐P4‐I4	Targeted science popularization work	Aiello, Coulborn, & Perez (2008)