Allocation plans for crisis triage: How well would they actually work?

COVID-19 has forced US state governments to create plans for rationing critical care resources that ensure the greatest population benefit. But a study by Jezmir and colleagues in this issue of Cell Reports Medicine raises doubts about whether these plans can distinguish those who would most benefit.

Main text

A year and a half into the COVID-19 pandemic, new infections are rapidly decreasing across most of the country. Yet it is hard to forget images from the early days of the pandemic and the fear they evoked. Scenes of desperately crowded emergency rooms and intensive care units stretched to the limit; the reality of limited life-saving resources like ventilators. In the United States, where access to critical care has never been explicitly rationed, doomsday scenarios in which doctors choose which patients are worthy of the last ventilators became a real possibility.

In response to the crisis, US states and hospital systems rushed to create allocation plans for limited critical care resources. The goal was to ensure that limited resources would be dispensed in a manner that facilitated population health—or in ethical terms, “provided the greatest good for the greatest number.” Yet while the ethical underpinnings of these plans received significant debate,, until recently, little attention has been paid to the practical operation of these plans. Are they workable? Are they based on sound science? And more fundamentally, would they do what they said they would, namely help promote the greater good?

While state crisis allocation plans each take slightly different forms, almost all rely on two components to assign patients allocation scores: a predictor of short-term mortality from acute illness (most commonly the Sequential Organ Failure Assessment or SOFA) and a consideration of preexisting medical comorbidities. The idea—adapted from the battlefield—is that limited resources should not be misspent on those whose illness is so advanced that chances of survival are low. But because crisis triage planning on this scale has never been done before, the validity of this approach is unknown. Do these prediction tools discern those likely to survive from those who are not? And are these tools objective, or could they unwittingly introduce bias and inequality into seemingly impartial decisions?

The study by Jezmir et al., published in this issue of Cell Reports Medicine, analyzes the performance of two representative state crisis allocation algorithms (from New York and Colorado), and these plans’ ability to discern 28-day in-hospital mortality. Additionally, the authors simulated clinical scenarios to explore how well these algorithms function as triage tools in real life, and finally, they examined the impact of race on the algorithm’s performance. Their findings add to a growing literature questioning how well these decision algorithms work and raise questions about unrecognized bias.,

Using the STOP-COVID multicenter cohort, the authors examined over 2,700 COVID-19 patients with acute respiratory failure requiring intubation and mechanical ventilation. Taking patient data from the cohort, they calculated patients’ priority scores according to two state rationing plans. Since all these patients would presumably have died without mechanical ventilation allocation, scores were literally a matter of life and death. The authors then examined how well these allocation scores discerned those who died from those who survived.

The authors found that SOFA scores provided only modest discrimination for 28-day mortality. Whether using raw scores (as New York does) or ranges (as Colorado does), the area under the receiver operator characteristic (AUROC) for discriminating 28-day hospital mortality was only 0.61 (0.59–0.63). Adding consideration of comorbidities improved performance, but only modestly, increasing the AUROC to 0.67 (0.65–0.69). These findings are not totally surprising. The SOFA was developed primarily as a tool to stratify patients in clinical studies—not for crisis triage. And while it is useful for predicting mortality among patients with multisystem organ dysfunction, many COVID-19 patients needing mechanical ventilation present with only isolated respiratory failure.

The study by Jezmir et al. also undermines another major assumption of current crisis allocation plans, specifically, that these plans would largely obviate the need for using non-scientific methods of triage, such as age or random lottery. Using bootstrap analyses to create real-life clinical scenarios, the authors calculated how often the New York or Colorado crisis allocation would yield a “winner” when algorithms were applied to multiple patients with similar degrees of illness who all needed critical care resources. When deciding between two patients, the New York algorithm chose a winner 51% of the time (95% CI, 47–55) but when choosing among five patients, it only could choose a winner 6% of the time (95% CI, 4–7). This suggests that in real life, the New York crisis allocation plan would function almost as a pure lottery!

Over the last year, the impact of racial inequalities in healthcare received considerable attention, and crisis allocation standards did not escape scrutiny, specifically on the issue of whether to consider medical comorbidities. On the one hand, accounting for serious medical comorbidities might improve mortality prediction. But on the other hand, if these comorbidities were at least partially determined by social factors—such as race or socioeconomic status—considering comorbidities in crisis algorithms could amplify inequalities, and further institutionalize bias. On top of that concern, Jezmir et al.’s analyses suggest that for unexplained reasons both the New York and Colorado algorithms might predict death less accurately in Blacks than they do in whites.

So far, there are no reports of crisis allocation frameworks for explicit rationing having been deployed in the United States. Although sporadic cases of implicit rationing almost certainly have occurred, at least in the short-term, it is likely (even with lower than desired vaccination rates) that we will not be forced to institute crisis triage plans. However, the study by Jezmir and colleagues raises serious questions about the viability of current allocations plans if we ever had to.