Rapid Development of a Tool for Prioritizing Patients with Coronavirus Disease 2019 for Intensive Care.

OBJECTIVES: To explain and demonstrate a new approach for rapidly developing a decision-support tool for prioritizing patients with coronovirus 2019 disease for admission to ICUs.
DESIGN: An expert group used multi-criteria decision analysis methods to specify criteria and weights, representing their relative importance, for prioritizing patients with coronovirus 2019 disease with respect to likely clinical benefit. Specialized multi-criteria decision analysis software, implementing the "Potentially All Pairwise RanKings of all possible Alternatives" method to determine the weights, was used. Social equity considerations for prioritizing patients were also identified as important.
SETTING: The prioritization tool was developed in New Zealand.
SUBJECTS: An expert group comprising specialists from intensive care medicine and nursing, Māori (New Zealand's indigenous population) health, infectious diseases, and neonatology was formed. The group's work was supported by health economists and decision analysts and overseen by an ethicist and a senior representative from the New Zealand Ministry of Health.
INTERVENTIONS: Multi-criteria decision analysis to create a prioritization tool.
MEASUREMENTS AND MAIN RESULTS: The prioritization tool comprised eight criteria with respect to likely clinical benefit. In decreasing order of importance (weights in parentheses): Sequential Organ Failure Assessment score (15.7%), preexisting cardiovascular conditions (15.7%), functional capacity (15.7%), age (12.4%), preexisting respiratory conditions (11.1%), immunocompromised (11.1%), body mass index (9.2%), and other relevant medical conditions (9.2%). Two social equity considerations were also included in the overarching decision framework to be used alongside the clinical criteria: prioritizing Māori and Pacific people (and, potentially, other at-risk groups), and healthcare and other frontline workers.
CONCLUSIONS: The criteria and weights in the prioritization tool can be easily revised as new evidence emerges. The approach for developing the tool could be used in other countries whose ICUs are at risk of being overwhelmed by the coronavirus disease 2019 pandemic to rapidly develop their own prioritization tools. In the event that future crises threaten to overload ICUs, other prioritization tools could also be rapidly developed.

The coronavirus disease 2019 (COVID-19) pandemic has generated unprecedented demand for intensive care beds and ventilators in health systems around the world. The disease has a 2–7% mortality rate, with up to 20% of patients requiring hospitalization, of which 4–14% are admitted to intensive care with 60–88% receiving invasive ventilatory support (1–6). ICUs in China, Italy, the United Kingdom, and the United States have been overwhelmed, with ICUs in other countries similarly threatened (7). This pressure on ICUs has resulted in patients needing to be prioritized for admission. Valid, transparent, and pragmatic decision-support tools to help clinicians with these ethically difficult, often “life or death” decisions, are required (8, 9).

Prioritizing patients for ICU involves clinicians weighing up each patient’s likely benefit from treatment and survival, by considering the severity of illness and comorbid conditions, as well as other factors (10–13). Because COVID-19 patients can require long periods of mechanical ventilatory support (median 13 vs 9 days for noncoronavirus disease [non-COVID] ICU patients) (6), a patient’s clinical assessment should also be compared with other patients’ opportunities to benefit. Mortality rates of invasively ventilated COVID-19 patients are high—almost double the rate of patients with non-COVID viral pneumonia (40% vs 21%) (6). Initiation of futile treatment likely to further distress patients in their final days is inappropriate. Judgments about withdrawing critical support from one patient in favor of another (with the objective of maximizing the number of patients successfully treated) are even more fraught.

Prioritizing patients with the best prognoses is underpinned by utilitarian ethical principles associated with maximizing lives or life years saved (9, 14, 15). The “Fair Innings” principle is also commonly applied: younger patients are prioritized over older patients, so that everyone has the opportunity for a “fair innings” in life (16). These principles can conflict. Is it better to ventilate a multimorbid 30-year-old with a poor prognosis and long length of stay, or an otherwise well 60-year-old with a substantially lower risk of mortality and length of stay? How do clinicians (and ethicists) resolve such difficult decisions, and what tools can support them?

Pragmatic considerations of staff workload and other essential therapy components are also important when ICUs are under pressure (17, 18). Medically complex and vulnerable patients—such as those needing renal replacement therapies and patients at both extremes of body mass index (BMI)—require more expert nursing. The capacity to benefit from ventilation may be diminished by inadequate workforce expertise, essential equipment, or consumables.

Should healthcare workers who become infected with COVID-19 be prioritized because of their important and sometimes irreplaceable role in the pandemic response? “Essential workers”—however designated—may be prioritized based on the ethical principle of reciprocity in recognition of the extra risks they take to help others (9, 14). In many countries, ICU workforces were already severely constrained before the pandemic; losing staff would curtail capacity further.

In New Zealand, where the current study is set, Māori (the indigenous population) and Pacific people are significantly disadvantaged in terms of health inequities and are likely to be disproportionately harmed by a COVID-19 outbreak. The World Health Organization promotes actions in pursuit of equity, regarding health as a human right (19), and in New Zealand, the Treaty of Waitangi (a constitutional document) requires government to achieve equity for Māori (20). Analogous considerations are relevant for other countries.

As the COVID-19 pandemic emerged globally in early 2020, a simple prioritization framework developed by White and Lo (21), adapted from an earlier framework (22), was adopted by many ICUs in the United States (23). Patients are prioritized on their likelihood to benefit, assessed by their Sequential Organ Failure Assessment (SOFA) score (24) and prognosis for near-term survival. Both criteria have four levels that are equally weighted, representing identical relative importance; thus, patients are prioritized according to their score in the range of 1–8 points. Pandemic healthcare workers receive additional priority, and if patients are prioritized equally, younger patients are selected (21).

Tyrrell et al (25) systematically reviewed another eight “guidelines” for prioritizing COVID-19 patients for ICU. Most are based primarily on patients’ likelihood of survival, with the objective of maximizing lives saved, although several are not explicit about their ethical principles. Most of these guidelines ignore the other prioritization considerations discussed earlier (e.g., staff workload and prioritization of essential workers). Tyrrell et al (25) concluded there is “a lack of high-quality evidence and guidelines on resource allocation during the pandemic” (p. 1).

Other COVID-19 ICU prioritization tools (also variously known as “protocols,” “algorithms,” and “schemes”) have also been developed, each with their own strengths and weaknesses. The triage protocol by Maves et al (26) includes a mortality prediction tool but does not recommend a particular one for use. The algorithm by Sprung et al (27), although relatively easy to implement due to its simplicity, has the disadvantage of sorting patients into just four prioritization categories, requiring tie-breakers based first on patients’ predicted life years saved and then “first-come, first-served”—an ethical principle rejected by most ethicists (8, 9). The prioritization scheme by LeClerc et al (28) has separate pathways for how prioritization is implemented depending on the pressure ICUs are under, and although patients may be excluded based on clinical frailty scores, preexisting conditions, and acute conditions, prioritization is still based on just SOFA score or occupational exposure. These shortcomings are consistent with the recommendation by Tyrrell et al (25), based on their abovementioned review, that “[f]uture guidelines need to be evidence based and developed using robust methodologies” (p. 9). The aim of the present article is to contribute to this objective.

As the pandemic began to threaten New Zealand in February 2020, it was immediately apparent that because the country had only 4.6 ICU beds per 100,000 population—for example, compared with 12.5 in Italy and 29.4 in the United States—a COVID-19 prevalence of just 0.4% would exceed New Zealand’s ICU capacity. The Australian and New Zealand Intensive Care Society emphasized that the allocation of intensive care resources must be “consistent, transparent, objective and ethical” (p. 98) (12). The development of a prioritization tool based on valid and reliable criteria and weights if ICUs were to become overwhelmed was identified as a vital preparation.

This article reports on a new approach implemented in New Zealand for rapidly developing a prioritization tool based on methods from multi-criteria decision analysis (MCDA)—a decision-making methodology when multiple criteria need to be considered together (29). The tool was developed over a period of just 10 days as the country was preparing to go into national lockdown, with the tool ready for use on 26 March—27 days after New Zealand’s first reported COVID-19 case and 1 day after lockdown (30). Fortunately for New Zealand, the tool has not been used because to date public health measures have successfully eliminated the disease from New Zealand. Despite it not being possible to report on the tool’s performance “in the field” (ICU), the article’s primary objective is to explain the approach, supported by a demonstration of its feasibility for rapid implementation.

METHODS

Participants and Ethics Approval

On 17 March 2020, an expert group for developing the prioritization tool was convened. The group comprised representatives from intensive care medicine (including the sixth and seventh authors), Māori health (the fifth author), intensive care nursing, infectious diseases, neonatology, and ethics. The group’s work was supported by the first four authors—representing public health medicine, health economics, and decision analysis—and overseen by the chairperson of the New Zealand National Ethics Advisory Committee and a senior representative from the New Zealand Ministry of Health. Ethics approval was provided by the University of Otago Human Ethics Committee (reference D19/071).

Inclusion and Exclusion Criteria

The inclusion and exclusion criteria from the Canadian pandemic triage protocol (31) were adapted to screen patients who could potentially benefit from ICU treatment.

MCDA

MCDA is increasingly used for prioritization in the health sector (32), such as for determining access to elective services (33) and developing national guidelines for allocating pandemic influenza vaccines (34). The methodology is intended to reduce biases from decision-makers relying on their “clinical instinct,” resulting in more transparent and consistent decisions (35).

Two elements are fundamental to MCDA in the present context. First is the specification of criteria for prioritizing COVID-19 patients for ICU and levels of severity within each criterion. Second is the determination of weights on the criteria and levels, representing their relative importance. The resulting prioritization tool (criteria and weights) is implemented by rating each patient on the criteria and then summing the weights to produce a score. Patients are ranked (prioritized) by their scores.

Specialized MCDA software, 1000minds (www.1000minds.com), which implements the “Potentially All Pairwise RanKings of all possible Alternatives” (PAPRIKA) method to determine the weights (36), was used to support the group’s work. This software and method have been used since 2004 in a wide range of health applications, including for prioritizing patients for elective services (33), antibiotic-resistant diseases for research into new drugs (37), and noncritical COVID-19 patients for hospital admission (38) (at about the same time as the present project).

Prioritization Tool

The process for creating the prioritization tool involves eliciting and codifying the experts’ clinical knowledge and preferences. As well as their training, the experts have tacit knowledge based on their clinical practice and networks and are up-to-date with the emerging COVID-19 literature and in touch with international colleagues on the COVID-19 frontline. The process has four stages, as outlined below.

Patient Vignette Rankings.

To enable identification of relevant criteria for prioritizing COVID-19 patients for ICU, the experts were asked to use their clinical judgment to prioritize 10 patient case “vignettes” representative of people likely to be considered for admission (Supplementary Table 1, http://links.lww.com/CCX/A529). Each vignette is a concise description of patient information, including demographic characteristics, salient medical history, and clinical presentation. The experts ranked the vignettes individually and then by consensus as a group, which promoted discussion about relevant prioritization criteria and validation of clinical judgments within the group.

Criteria and Levels.

The vignette ranking exercise and ensuing discussion resulted in the identification of clinically and ethically relevant criteria for prioritizing COVID-19 patients. The levels of severity within each criterion were also defined.

Weights.

The PAPRIKA method (36) (for determining the weights on the criteria and their levels) involved the experts voting on a series of simple questions based on choosing which of two hypothetical patients should be prioritized for ICU. To minimize the experts’ cognitive burden, the hypothetical patients in each question were defined on just two criteria at a time, with the other criteria assumed to be identical for both patients (Fig. ).

Fundamental to the questions is that the two criteria included in each question are specified so that the experts had to confront a trade-off when prioritizing the two patients, thereby revealing the relative importance of the criteria. If the experts’ votes for a trade-off question were not unanimous, there was a discussion until a consensus or clear majority (> 80%) was reached. From the answers to the questions, the method calculated the weights on the criteria and levels.

Reliability and Validity.

The test-retest reliability of the experts’ answers was evaluated by repeating some of the trade-off questions and voting on them again. To evaluate face validity, the experts considered the intuitive plausibility of the relative importance of the criteria and levels implied by the weights.

As a further validity test, the experts’ consensus ranking of the 10 vignettes from stage 1 was used as a “pseudo-gold standard” to compare the ranking produced by the tool. Finally, a new sample of COVID-19 patient cases was used to check that they were appropriately differentiated with respect to the distribution of their scores from the tool.

RESULTS

The inclusion and exclusion criteria are reported in Supplementary Table 2 (http://links.lww.com/CCX/A530). These criteria apply to adult COVID-19 patients (≥ 18 yr) who could consent to and potentially benefit from intensive care and for whom such care is appropriate.

Twelve experts participated in the patient vignette ranking exercise and discussion, which resulted in the specification of eight clinical criteria and two additional “social” considerations for prioritizing patients. Classifications of common preexisting conditions into appropriate levels are reported in Table ; existing disease classification systems were included for ease of use and to facilitate shared understanding among the wider healthcare community.

The weights for the clinical criteria and their levels were determined in three meetings: a pilot-test of the voting exercise, which resulted in the criteria and their levels being refined in a second meeting, and then a final voting and consensus exercise. In the final meeting, which lasted approximately 2 hours, the experts voted on 49 trade-off questions, mostly resulting in a unanimous vote or a clear majority, with discussions for the remaining questions until consensus was reached.

The experts’ answers exhibited test-retest reliability, and the ranking of the vignettes produced by the tool matched the experts’ consensus ranking (the “pseudo-gold standard”). Overall, the experts were satisfied with the validity of the criteria and weights.

The criteria and weights are reported in Table . To implement the prioritization tool, each patient is rated on the eight criteria, and the corresponding weights are summed to produce a score (in the range 0–100%). As an illustration, the tool’s ranking of the 10 vignettes is reported in Supplementary Table 3 (http://links.lww.com/CCX/A531).

Social Equity Considerations

The COVID-19 pandemic poses a disproportionate risk to marginalized populations, including Māori and Pacific people of New Zealand who experience poorer health outcomes significantly contributed to by the effects of colonization (39, 40). Therefore, the expert group, endorsed by the representatives from ethics and the Ministry of Health, agreed that Māori and Pacific people should have some degree of higher priority. It was also deemed that healthcare workers and other critical frontline personnel who become infected with COVID-19 while working should have higher priority. The healthcare workers in the expert group recused themselves from this specific discussion, although they contributed to the wider dialogue.

These two social equity considerations were initially intended to be incorporated with the eight clinical criteria within the tool. However, when the voting exercise was being pilot-tested, it soon became apparent that the experts found thinking about possible trade-offs between social equity considerations and clinical criteria cognitively difficult. Furthermore, doctors’ respect for the principles of the Hippocratic Oath may prevent them from differentiating between patients based on ethnicity or occupation, which, in practice, may result in the social equity considerations receiving no weight at all. A pragmatic solution was to separate the social equity considerations and evaluate them alongside the clinical prioritization tool.

Thus, the prioritization tool involves rating patients on the clinical criteria—generating a score in the range 0–100%—and collecting supporting information about patients with respect to the abovementioned social equity considerations. The score, representing clinical likelihood of benefit, as well as social equity considerations and other supporting information, can then be assessed on a case-by-case basis by a panel of three ICU experts within the overarching decision framework.

Further work is being undertaken by experts in Māori health and ethics to effectively integrate social equity concerns within the prioritization tool. One approach suggested is to implement a reserve system: a designated proportion of ICU beds could be reserved for Māori and Pacific people in proportion to their percentage of the population and augmented for their disparity in health outcomes due to greater prevalence of comorbid conditions.

DISCUSSION

Intensive care medicine involves complex, rapid decision-making that usually depends on individual clinical judgment (11). In “normal” times, when resources are relatively unconstrained, clinicians usually commence a “trial of ICU” when a patient’s potential to benefit is uncertain. In contrast, during a major pandemic, when resources are severely constrained and not all patients can be treated, rationing ICU resources is inevitable. Prioritization tools support clinicians in making these difficult decisions and reduce the associated burden and distress.

Evidence about prognostic clinical factors in COVID-19 patients is accumulating, but as successful treatment strategies are identified, survival likelihood is continually changing. This variability limits the reliability of mortality calculators developed from historical datasets. Arguably, clinical decision-making is still best guided by expert judgment informed by available evidence—as summarized in a tool like the one developed here. Alternative approaches include the use of machine learning or other data-driven algorithms, but this is problematic due to issues with biases in case reporting (41), which increases the likelihood of errors compounding the effects of socioeconomic disparities (42).

Although based on existing pandemic triage tools (31), the proposed prioritization tool for New Zealand was developed specifically for prioritizing COVID-19 patients for admission to ICUs. It represents the consensus of a group of intensive care experts about valid and reliable criteria and weights for assessing patients’ likely clinical benefit. Social equity considerations can also be incorporated within the overarching decision framework.

One limitation is that this proposed tool has not been used in practice yet—because, as mentioned earlier, public health measures have successfully controlled the pandemic in New Zealand so far—and so it has not been possible to evaluate the tool’s performance in practice. Nonetheless, the main purpose of reporting on the New Zealand application here was to demonstrate the feasibility of the MCDA methods explained in the article for rapidly developing a prioritization tool, which could be used in other countries too.

A new tool can be created from scratch in a matter of just a few days, or a tool such as the one presented here could be adapted or updated in just a few hours. Because all the data from the process are stored electronically by the 1000minds MCDA software, the criteria and weights can be easily revised without repeating the whole process. As new evidence emerges, further criteria can be added or obsolete criteria deleted without involving criteria unaffected by such changes. If clinical judgments change about the relative importance of the criteria, clinicians are required to answer only the affected trade-off questions at stage 3 of the process outlined earlier, resulting in revised weights on the criteria. For example, if COVID-19 were to threaten to overwhelm New Zealand’s ICUs again, the method could be rapidly reapplied to update the tool to incorporate the latest evidence about the disease.

Obvious possible refinements to the current tool include replacing the “functional capacity” criterion with clinical frailty scores, reviewing the levels defined for the SOFA criterion, and including other factors that are prognostic of a poor outcome, such as the presence of lymphopenia. Within the social equity considerations, other at-risk groups could potentially be included. Further work is needed to determine how this tool would be used within an overarching decision framework to address these issues of social equity.

Additional work would also be required to extend the tool to triage COVID-19 patients alongside other ICU patients. Many of the criteria are helpful for assessing any critically ill patient, but that is not the purpose of the tool. Clinicians already assess and triage the general ICU patient population on a regular basis. The tool is designed to assist within a wider decision-making setting to manage an unfamiliar disease threatening to overwhelm ICUs.

Although other COVID-19 ICU prioritization tools are available (e.g., the framework by White and Lo [21]), they focus mainly on isolated clinical factors such SOFA score and prognosis for near-term survival and do not adequately consider all relevant clinical factors (also, the validity of equal weights is moot). Factors such as BMI, functional status, and the presence and severity of comorbid conditions clearly affect outcomes differentially, and so they should be included and accurately weighted to assess a patient’s likely clinical benefit (6). These additional considerations were able to be included in the proposed tool in large part because of the PAPRIKA method used for determining the weights on the criteria and levels.

Fundamental to PAPRIKA’s usefulness was the cognitive ease with which the participating experts were able to answer PAPRIKA’s trade-off questions (involving repeatedly choosing between two hypothetical patients defined on two criteria at a time with respect to who should be admitted first). Also, with just three possible answers (Fig. 1), PAPRIKA’s questions are suited to rapid group decision-making via voting and consensus.

Figure 1.

Example of a trade-off question. BMI = body mass index, COVID-19 = coronavirus disease 2019.

The resulting weights—codifying the experts’ clinical knowledge and preferences—are likely to be more valid and reliable than weights produced by other MCDA methods, which usually involve more cognitively difficult elicitation techniques based on ratio-scale measurements of preferences (also making them less amenable to group decision-making). “The advantage of choice-based methods is that choosing, unlike scaling, is a natural human task at which we all have considerable experience, and furthermore it is observable and verifiable” (43). PAPRIKA also enables the criteria and weights in the prioritization tool to be easily revised as new evidence emerges.

The approach explained and demonstrated in this article could be used in other countries whose ICUs are at risk of being overwhelmed by the COVID-19 pandemic to rapidly develop their own prioritization tools—in effect, adapting the proposed tool for New Zealand to other settings. In the event that future crises threaten to overload ICUs, other prioritization tools could also be rapidly developed.

ACKNOWLEDGMENTS

We thank the Coronavirus Disease 2019 ICU Multicriteria Decision Analysis Working Group: Craig Carr, Andrew Stapleton, Gillian Bishop, Seton Henderson, Liza Edmonds, Brendon Arnold, Pam Adams, Elaine Fernandes, Mathew Kiore, Gilbert Taurua. We also thank Neil Pickering and Chris McEwan.

TABLE 1.

Examples of Diseases Corresponding to Levels Within Each Criterion

Criterion	Level	Examples
Preexisting cardiovascular conditions	Significant	Heart failure class II, severe hypertension > 180/120, severe valvular disease, ischemic heart disease; angina class II–III
	Moderate	Class 1 heart failure, atrial fibrillation, poorly controlled hypertension (> 160/110). Prior coronary percutaneous intervention
	Mild	Well-controlled hypertension
Preexisting respiratory conditions	Significant	FEV1 30–40% predicted, severe COPD
	Moderate	FEV1 40–80% predicted, moderate asthma/COPD, heavy smoker (> 20/d)
	Mild	FEV1 > 80% predicted, mild asthma
Other relevant medical conditions: renal, endocrine, neuromuscular, malignancy	Moderate/significant	Neuromuscular disease with respiratory impairment, metastatic malignancy treated with palliative intent, stage 4–5 chronic kidney disease, diabetes with end-organ damage
	Mild	Diabetes without end-organ damage, stage 2–3 chronic kidney disease, malignancy managed with long-term stability or curative intent
Immunocompromised	Moderate/significant	Chemotherapeutic or posttransplant medications with significant immunocompromize, long-term high dose prednisone
	Mild	Inhaled steroids, low-dose steroids

COPD = chronic obstructive pulmonary disease, FEV1 = forced expiratory volume during first second.

TABLE 2.

Criteria and Weights for ICU Prioritization

Criterion/Levels	Weight, %
Sequential Organ Failure Assessment score
> 11	0
≤ 7 or single organ failure	15.0
8–11	15.7
Preexisting cardiovascular conditions
Significant	0
Moderate	9.2
None/mild	15.7
Functional capacity
Poor: MEWS < 5	0
Normal: MEWS > 5	15.7
Age (independent of other criteria), yr
> 75	0
60–75	4.6
40–59	6.5
19–39	11.1
≤ 18	12.4
Preexisting respiratory conditions
Significant	0
Moderate	4.6
Mild	9.2
None	11.1
Immunocompromised
Moderate/significant	0
None/mild	11.1
Other relevant medical conditions
Moderate/significant	0
None/mild	9.2
Body mass index (independent of comorbidity and function)
> 50	0
40–50	4.6
< 18	6.5
19–40	9.2

MEWS = modified early warning score.

The bolded values represent the relative weights of the criteria overall (i.e., bolded values sum to 100%). The expert group considered that patients with Sequential Organ Failure Assessment score < 7 were less likely to require admission to ICU but may do so in the event of isolated respiratory failure. The ranking of the levels and their weights may change in future revisions of the tool.