Multi-Criteria Decision Analysis to prioritize hospital admission of patients affected by COVID-19 in low-resource settings with hospital-bed shortage.

OBJECTIVE: To use Multi-Criteria Decision Analysis (MCDA) to determine weights for eleven criteria in order to prioritize COVID-19 non-critical patients for admission to hospital in healthcare settings with limited resources.
METHODS: The MCDA was applied in two main steps: specification of criteria for prioritizing COVID-19 patients (and levels within each criterion); and determination of weights for the criteria based on experts' knowledge and experience in managing COVID-19 patients, via an online survey. Criteria were selected based on available COVID-19 evidence with a focus on low- and middle-income countries (LMICs).
RESULTS: The most important criteria (mean weights, summing to 100%) are: PaO2 (16.3%); peripheral O2 saturation (15.9%); chest X-ray (14.1%); Modified Early Warning Score-MEWS (11.4%); respiratory rate (9.5%); comorbidities (6.5%); living with vulnerable people (6.4%); body mass index (5.6%); duration of symptoms before hospital evaluation (5.4%); CRP (5.1%); and age (3.8%).
CONCLUSIONS: At the beginning of a new pandemic, when evidence for disease predictors is limited or unavailable and effective national contingency plans are difficult to establish, the MCDA prioritization model could play a pivotal role in improving the response of health systems.

Background

As of 13th May 2020, there have been 4,170,424 confirmed cases and 287,399 confirmed deaths from SARS CoV-2 worldwide (World Health Organization. COVID-19 disease pandemic, 2020). Since the first case of the coronavirus disease 2019 (COVID-19) was recorded in Italy on 21st February, 2,735,628 nasopharyngeal swabs have been processed. The number of cases and deaths has reached 222,104 and 31,106 respectively, with Italy having one of the highest national rates of local transmission. The Italian government imposed aggressive measures to contain the spread of the disease. Nevertheless, the daily incidence of new COVID-19 cases and deaths reached alarming rates (Ministero della Salute. COVID-19 Situazione in Italia, 2020). SARS CoV-2 appeared in Italy in the middle of flu season, contributing to the over-crowding of primary care, outpatient clinics and emergency departments. Due to the COVID-19 pandemic emergency, the Italian National Health System (NHS), which is regionally based and offers universal access to healthcare, has been close to collapse (Armocida et al., 2020). The shortage of available hospital beds and the lack of beds in intensive care units (ICUs) for critically ill patients have been among the major challenges faced.

Because even countries with robust health care systems and strong economies can be rapidly overwhelmed by this emergency, attention starts to be focused on less advantaged areas of the world (Hopman et al., 2020). In low- and middle-income countries (LMICs), where over-crowding renders social distancing almost impossible, shortages of hand sanitizers and clean water are the norm and prevention measures are difficult to establish, the spread of the pandemic could have catastrophic consequences. Healthcare facilities, already congested and lacking personnel and supplies, are likely to be rapidly overwhelmed and not able to provide potentially life-saving services – such as caesarean sections or basic surgery – anymore (Bong et al., 2020). An African task force for coronavirus preparedness and response (AFTCOR) has been established, focusing on: laboratory diagnosis and subtyping, surveillance, infection prevention and control in health care facilities, clinical treatment of people with severe COVID-19, risk communication, and supply chain management (Nkengasong et Mankoula, 2020). Nonetheless, prioritizing access to care in settings at extremely high risk of collapse appears to be unavoidable.

Unlike triage for prioritizing admissions to ICUs – which has been debated worldwide (Emanuel et al., 2020, White and Lo, 2020, The Hastings Center, 2020) – no explicit recommendations have been developed to identify which COVID-19 patients are prioritized for hospital admission in settings with an unsolvable shortage of beds and in LMICs. The quality of such prioritization decision-making when multiple criteria need to be considered together can be improved by using structured and explicit methods. Multi-Criteria Decision Analysis (MCDA) is useful in such a context. Fundamental to MCDA is specifying the criteria that are relevant for the decision at hand and determining their relative importance (usually represented in terms of weights). Widely used in many sectors, MCDA is increasingly employed in healthcare applications to increase the consistency, transparency, and legitimacy of decisions (Thokala et al., 2016, Marsh et al., 2014).

The objective of this study was to use MCDA to identify non-critical COVID-19 patients who should be admitted to hospital because of their risk of rapid clinical deterioration.

Methods

MCDA and the PAPRIKA method

The MCDA was applied in two main steps: 1. specification of criteria for prioritizing COVID-19 patients for hospitalization and the levels within each criterion, and 2. determination of weights for the criteria (and their levels), representing their relative importance, based on experts’ knowledge and preferences. At the first step, evidence from the scientific literature on predictors of outcomes in patients affected by COVID-19 was reviewed up to March 15. At the second step, a large group of Italian experts were invited to complete an online survey to determine the weights for the criteria. The experts were selected according to their experience in dealing with COVID-19 patients, and included physicians based in emergency, infectious diseases, pneumology, and internal medicine departments and working in a variety of institutions (i.e. university hospitals, institutes for research and treatment, and community hospitals). Attention was paid to the prevalence of COVID-19 cases in the experts’ region: more experts based in northern Italian regions were invited than experts in southern regions where the disease is less prevalent.

The survey was run using 1000minds MCDA software (www.1000minds.com) which implements the PAPRIKA (Potentially All Pairwise RanKings of all possible Alternatives) method (Hansen and Ombler, 2008). Previous applications of the software and method include prioritizing patients for elective surgery and creating the World Health Organization’s priority list of antibiotic-resistant bacteria to support research and development of new drugs (Hansen et al., 2012, Tacconelli et al., 2018). The PAPRIKA method involved each participant being shown a series of pairs of combinations of levels on two criteria at a time (in effect, representing a pair of imaginary patients) and asked for each pair: “Which one of these combinations of criteria is more relevant for the hospitalization of a COVID-19 patient during a health emergency, considering a shortage of hospital beds?”. Each pair of combinations involved a trade-off between the two criteria, such that when participants answered the question – by choosing one of the two combinations or indicating they are equal – they revealed their opinion about the relative importance of the two criteria. Such questions (always involving a trade-off between the criteria, two at a time) were repeated with different combinations of the criteria until enough information was collected to determine each participant’s set of weights for the criteria (using mathematical methods based on linear programming) (Hansen and Ombler, 2008). The criteria were not disclosed to the experts before the survey in order not to influence their answers. Two questions were repeated at the end of the survey as an internal consistency check. The software recorded the number of questions answered and the time taken to answer each question. At the end of the survey, the experts were also asked for their opinion about the usefulness of lung ultrasound (US) compared to chest X-ray for diagnosing COVID-19 pneumonia.

Participants’ weights were averaged to produce mean weights (and standard deviations, SD) for the group of experts as a whole. Significant differences in the mean weights for the criteria (p < 0,05) were assessed through a one-way analysis of variance for normally distributed variables, and the Kruskal-Wallis rank test when the normality assumption was not met.

Role of the funding source

The funders of the study had no role in study design, data collection, data analysis, data interpretation, writing of the article, or the decision to submit for publication. All authors were responsible for the decision to submit the article for publication.

Results

A board of five Infectious Diseases (ID) physicians with experience in treating COVID-19 patients selected 11 criteria to prioritize hospital admission, based on the current evidence and the availability and feasibility of criteria in LMICs (Zhang et al., 2020, Chen et al., 2020, Wang et al., 2020, Guan et al., 2020, Mo et al., 2020). The criteria (levels in parentheses) were: 1. age (18–50, 50–70, and >70); 2. body mass index (BMI: <30, 30–40, and >40); 3. Co morbidities (diabetes, pre-existing respiratory/cardiovascular diseases, and onco-hematological diseases); 4. respiratory rate (<20 breaths/min and >20 breaths/min); 5. PaO2 (>80 mmHg, 70–80 mmHg, and <70 mmHg); 6. peripheral oxygen (O2) saturation (>96%, 92–96%, and <92%); 7. findings at chest X-ray (normal, consolidation, and bilateral interstitial lung abnormalities); 8. Modified Early Warning Score-MEWS (Subbe et al., 2001), a clinical scoring system including pulse rate, respiratory rate, systolic blood pressure, body temperature, and neurological symptoms (score: 0–2 and 3–4); 9. duration of symptoms before hospital evaluation (<3 days, 4–7 days, and >7 days); 10. C-reactive protein (CRP: normal / high by local cut off); and 11. living with vulnerable people (i.e. people with comorbidities, pregnant women, or immunosuppressed patients). CRP was selected considering its potential availability as a point-of-care (POC) test worldwide (Drain et al., 2014).

Launched on 23rd March 2020, the online survey to determine the criteria weights ran for 15 days and was completed by 103 experts. Of them, 96 (93%) answered the two repeated questions consistently and were therefore included in the final analysis. These 96 experts were from 11 Italian regions, with the majority (70%) from Lombardy, Piedmont and Veneto, the three regions in northern Italy with the highest burden of cases. Fifty-three percent of the experts were working at institutions dealing with more than 500 COVID-19 patients since the beginning of the pandemic; 32% were based at university hospitals and 20% at institutes for research and treatment; 77% were ID physicians; and 53% were female. The mean number of questions answered by each participant was 36 (IQR 12), taking most participants 10–15 min in total.

From the experts’ answers to the survey, the most important criterion [mean weights, summing to 100%] was revealed to be PaO2 [16.3%], followed by peripheral O2 saturation [15.9%], chest X-ray [14.1%], MEWS [11.4%], respiratory rate [9.5%], comorbidities [6.5%], living with vulnerable people [6.4%], BMI [5.6%], duration of symptoms before hospital evaluation [5.4%], CRP [5.1%], and age [3.8%]. The criteria and their levels and mean weights are reported in Figure 1 . The values for each criterion’s highest level (bolded in the figure) sum across the criteria to one (100%), and thus each of these values is easily interpretable as the attribute’s relative weight overall. The value assigned to any middle levels of a criterion represents the combined effect of the level’s relative position on the particular criterion as well as the criterion’s weight; and each criterion’s lowest level has a value of zero. For any pair of criteria, the ratio of their overall weights measures their relative importance; for example, MEWS (Subbe et al., 2001) was 1.2 times (e.g. 11.4%/9.5%) more important than tachypnoea alone (respiratory rate >20 breaths/min) and approximately twice as important as duration of symptoms, C-reactive protein and BMI respectively (Figure 2 ). The majority of experts (70%) indicated that they think lung-US is more valuable than chest X-ray as an imaging tool for evaluating COVID-19 patients. The table presents an example of applying the mean weights to 10 randomly selected COVID-19 patients attending the emergency department from 1st March 2020 at the Verona University Hospital. More in detail, patients with a total score <33% were not admitted to inpatient COVID-19 unit. At follow-up they had no adverse outcome in terms of need of hospitalization, and/or need of oxygen therapy and/or death. Patients ranked ≥47% were all admitted (data not shown in the table). These patients needed high-flow oxygen therapy or non-invasive ventilation during inpatient stay (Table 1 ).

Figure 1

Mean weights for the criteria

The bolded values represent the relative weights of the criteria overall (i.e. the bolded values sum to 100%).

Abbreviations: MEWSmodified early warning score; “comorbidities” criterion includes: diabetespre-existing respiratory/cardiovascular diseases, and onco-hematological diseases; “living with vulnerable people” criterion includes: people with comorbiditiesand/or pregnant women, and/or immunosuppressed patients.

Figure 2

Relative importance of the criteria

Based on the mean weights, each number in the figure is a ratio corresponding to the importance of the criterion on the left relative to the criterion at the top (weights reported too, faded). The ratios are obtained by dividing the left weights by the top weights (i.e.: MEWS score is 2 times more important than duration of symptoms; duration of symptoms is 1.5 times more important than age, etc.). Abbreviations: MEWS, modified early warning score; BMI, body mass index; CRP, C-reactive protein; “comorbidities” criterion includes: diabetes, pre-existing respiratory/cardiovascular diseases, and onco-hematological diseases; “living with vulnerable people” criterion includes: people with comorbidities, and/or pregnant women, and/or immunosuppressed patients.

Table 1

Application of the weights to 10 COVID-19 randomly selected patients attending the emergency room from the 1st March 2020 at Verona University Hospital, Italy. Total scores are calculated by summing the weights for each patient according to the patient’s rating on the levels for the criteria.

Rank	Age range	Comorbidities	BMIa	Duration of symptoms (days)	Respiratory rate (breath/min)	SpO2b (%)	CRPc	Chest X-ray	Living with vulnerable people	MEWSd	PaO2 (mmHg)	Total score (%)
1	>70	Yes	<30	>7	>20	<92	Ne	interstitial lung abnormalities	No	0–2	65–70	69
2	18–50	Yes	>40	<3	>20	92–96	N	N	No	3–4	65–70	54
3	18–50	No	31–40	>7	>20	92–96	Hf	pulmonary consolidation	No	3–4	71–80	54
4	51–70	No	<30	4–7	>20	92-96	H	N	No	3–4	71–80	50
5	51–70	No	<30	4–7	<20	92–96	H	N	Yes	3–4	71–80	47
6	51–70	No	<30	4–7	<20	>96	N	interstitial lung abnormalities	No	0–2	71–80	32
7	18–50	No	<30	<3	>20	>96	N	N	Yes	3–4	>80	25
8	18–50	Yes	>40	>7	<20	>96	N	pulmonary consolidation	No	0–2	>80	23
9	>70	No	<30	4–7	<20	>96	N	N	No	3–4	>80	22
10	>70	Yes	<30	>7	<20	>96	N	N	No	0–2	>80	15

Body Mass Index;

peripheral oxygen saturation;

C-reactive protein;

Modified Early Warning score;

Normal;

High.

Discussion

The criteria included in the MCDA prioritization model developed here were deliberately selected in order to be able to be applied ‘anywhere and by anyone’, including by unskilled health personnel and in low-resource settings. This approach was intended to meet the needs of LMICs where, due to very limited resources, effective national contingency plans are difficult to establish.

At early stage, mild hypoxemia due to an impaired gas exchange can be easily identified using an arterial blood gas test (ABGT). Accordingly, PaO2 was the most important criterion identified by the experts. The second most important criterion, with a similar weight, was peripheral O2 saturation – suggesting that in settings where ABGT is unavailable, such as LMICs or even during domestic self-isolation, pulse oximetry may be a useful alternative to more invasive procedures. Chest X-ray was ranked lower than PaO2, probably because of the lower accuracy especially at an early phase of the disease. Indeed, a ‘normal’ chest radiograph should not exclude the possibility that an interstitial disorder is present in the appropriate clinical context (Ryu et al., 2007). The use of lung-US for evaluating COVID-19 patients has several advantages – such as lower risk of exposure to healthcare workers, repeatability during follow-up and lower costs and easier application, especially in LMICs (Soldati et al., 2020a). Moreover, lung-US can be applied in outpatient settings, as a triage for symptomatic patients at home as well as in the prehospital phase (Soldati et al., 2020b).

MEWS is a score that uses readily available and inexpensive clinical parameters to identify patients at increased risk of ICU admission or death (Subbe et al., 2001). With respect to the criteria’ ranking, MEWS score was considerably less important than PaO2 and O2 saturation. Moreover, MEWS was only 1.2 times more important than tachypnoea alone, corroborating the importance of parameters related to the respiratory system (O2 saturation and respiratory rate) outlined by this analysis. MEWS can be obtained quickly by physical examination and also by unskilled healthcare workers, and it has the advantage of combining both respiratory and non-respiratory parameters to assess a possible rapid worsening of clinical conditions – making it the fourth most important criterion.

Although it is well known that age negatively affects the outcome in COVID-19 patients (Li et al., 2020), age was found to be the least-important criterion. Remarkably, both BMI and CRP were 1.5 and 1.3 times more important than age, respectively. As recently published by Zhang et al. (2020), CRP testing could be used at the point of care in order to direct patients further along the treatment path. Finally, living with vulnerable people was also deemed to be a relevant criterion to consider when deciding whether to admit a COVID-19 patient, even though it is not a clinical parameter.

According to experts’ evaluation of COVID-19 patients, all ages are potentially at risk of rapid clinical deterioration. Although PaO2 – or alternatively O2 saturation – are essential parameters, both MEWS and BMI should be considered to predict negative clinical outcome and not deferrable need of hospitalization. Finally, in case of a large volume of patients entering healthcare facilities, POC CRP testing can be adopted as a useful criterion in the proposed prioritization model.

To the best of the authors’ knowledge, this is the first time that MCDA has been used in a pandemic event for ranking non-critical patients for hospitalization. Since most of the criteria can be collected also by patients themselves, a “simplified domestic model” for patients self-isolated would be easily adapted by excluding some criteria (e.g. chest X-ray and ABGT) and including others like peripheral O2 saturation. This approach represents an innovative way of coordinating efforts during a pandemic caused by a novel virus. Determining criteria and weights for prioritizing patients is even more relevant in conditions of critical imbalance between need and available resources. Furthermore, this model (criteria and weights) can be adapted to different settings and stages of the pandemic in response to emerging evidence. In the demonstrative case series shown in the table, for example, a threshold above 33% may be proposed for the identification of patients to be hospitalized, as all the patients ranked below this cut-off did not need hospitalization and had an overall positive outcome. The most adequate method to validate a threshold definition would be that of applying MCDA results to a cohort study. At the beginning of a new pandemic, it may be feasible to prospectively gather patients’ information based on the MCDA prioritization model (possibly with a multicentric approach). In this way, a threshold to support clinical decisions could be quickly available. Future research could include the validation of the patients’ scores also through machine learning. The results of this study suggest that, when evidence is limited, using MCDA to codify experts’ knowledge is a rapid and effective approach for creating tools to support difficult decision-making.

Contributors

PDN and EG conceived the study and wrote the first draft of the manuscript. FM and EC reviewed the literature. PH worked on the statistical analysis and revised the manuscript. HG and ET critically revised the manuscript. Each member of the COVID-19 MCDA Group was involved in the survey and contributed significantly to the work. All authors have seen and approved the final manuscript and contributed significantly to the work.

Funding

The Value-Dx project was supported by the 2 Joint Undertaking under grant agreement No 820755. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation program and EFPIA and bioMérieux SA, Janssen Pharmaceutica NV, Accelerate Diagnostics S.L., Abbott, Bio-Rad Laboratories, BD Switzerland Sàrl, and The Wellcome Trust Limited.

Ethical approval

Approval was not required.

Declaration of interests

The authors declare that they have no competing interests.