How Common SOFA and Ventilator Time Trial Criteria Would Have Performed During the COVID-19 Pandemic: An Observational Simulated Cohort Study.

OBJECTIVES: To evaluate how key aspects of New York State Ventilator Allocation Guidelines (NYSVAG)-Sequential Organ Failure Assessment score criteria and ventilator time trials -might perform with respect to the frequency of ventilator reallocation and survival to hospital discharge in a simulated cohort of coronavirus disease (COVID-19) patients.
METHODS: Single center retrospective observational and simulation cohort study of 884 critically ill COVID-19 patients undergoing ventilator allocation per NYSVAG.
RESULTS: In total, 742 patients (83.9%) would have had their ventilator reallocated during the 11-day observation period, 280 (37.7%) of whom would have otherwise survived to hospital discharge if provided with a ventilator. Only 65 (18.1%) of the observed surviving patients would have survived by NYSVAG. Extending ventilator time trials from 2 to 5 days resulted in a 49.2% increase in simulated survival to discharge.
CONCLUSIONS: In the setting of a protracted respiratory pandemic, implementation of NYSVAG or similar protocols could lead to a high degree of ventilator reallocation, including withdrawal from patients who might otherwise survive. Longer ventilator time trials might lead to improved survival for COVID-19 patients given their protracted respiratory failure. Further studies are needed to understand the survival of patients receiving reallocated ventilators to determine whether implementation of NYSVAG would improve overall survival.

By the time the World Health Organization (WHO) declared coronavirus disease (COVID-19) a pandemic on March 11, 2020, 113 countries had confirmed cases, with 80 955 cases reported in China and 696 in the United States. By the end of that month, 5% of all worldwide cases and 1 in 3 of US cases and deaths were in New York City. This created a real concern that hospitals might suffer a shortfall of resources and need to ration ventilators. Developed in 2007 and updated in 2015 to address such a scenario, the New York State Department of Health along with New York State Task Force on Life and the Law released the New York State Ventilator Allocation Guidelines (NYSVAG). This document draws upon a community’s duty to care, resource stewardship, transparency, and distributive justice to save the most lives. Briefly, this unvalidated strategy heavily utilizes the Sequential Organ Failure Assessment (SOFA), a summative 0-24 score of organ dysfunction across neurologic, pulmonary, cardiovascular, hematologic, hepatobiliary, and renal domains, to prioritize (colored red for highest or yellow for intermediate) and potentially exclude patients from a ventilator (colored blue). Patients triaged and provided a ventilator receive 1 for a pre-specified duration of time. After the ventilator time trial, their SOFA scores are recalculated, and patients are re-triaged to maintain their ventilators or have them reallocated to other patients. NYSVAG are very similar to other available crisis standard of care guidelines, 62.1% of which utilize a SOFA score. Moreover, and similar to NYSVAG, 55.6% exclude patients from a ventilator for a SOFA score > 11, and 65.5% use the same ventilator time trial intervals (2 days, 5 days, and every subsequent 2 days). With only 1 documented case of ventilator rationing by NYSVAG exclusionary critiera in New York City during the original surge, and with Alaska and Idaho having both activated regional crisis standards of care that explicitly permit ventilator rationing (Alaska uses verbatim SOFA cutoffs as NYSVAG and Idaho use a similar SOFA and time trial criteria as NYSVAG), it remains crucial to understand how these guidelines might perform.

Objectives

It remains uncertain how such ventilator allocation guidelines would perform if implemented. This study simulates how specific SOFA cutoffs for resource prioritization and the ventilator time trials, used in NYSVAG and other crisis standards of care documents, might affect ventilator allocation and patient survival in a COVID-19 cohort during the pandemic. To address this research question, the relevant portions NYSVAG were simulated on every patient independently to one another and independent to the number of available ventilators, thereby aligning temporally discordant patient hospitalizations. This design can reveal important guideline inefficiencies that might potentiate mortality and remain obscure when incorporating a fixed ventilator supply with variable or low ventilator strain. Moreover, this analysis is helpful to evaluate how explicit triage criteria (SOFA cutoffs and ventilator time trials) would perform in an isolated manner that is independent to the resources available at a facility. Such an evaluation can determine whether a resource allocation strategy may be effective given the trajectory of patient survival or whether it might reallocate resources away from patients who would otherwise survive. Additionally, a retrospective analysis where each patient received a ventilator provides a unique control group to simulate resource allocation protocols because their survival with a ventilator is known. The results of this analysis remain germane to developing efficient crisis standard of care procedures and could provide reassurance of existing ventilator allocation guidelines or suggest the need for further revisions.

Methods

Adult patient charts were retrospectively reviewed from a single academic hospital system encompassing 1 tertiary care referral center and 2 community hospitals from 3/1/2020–7/1/2020. Patients were included if they were SARS-CoV-2 positive and were invasively mechanically ventilated. Components of the SOFA score were obtained for each patient daily as close to 10:00 am as possible. Missing components of the SOFA score were presumed normal. If there was no reported SOFA score on the day of intubation, the SOFA score from the next day was used instead for the day of intubation (and for relevant comparisons). Every patient received a ventilator as medically indicated, and ventilator allocation or reallocation per NYSVAG only occurred in a virtual simulation.

NYSVAG were interpreted for this study in the following manner: Step 1, the application of exclusionary criteria, was deferred because the number of such patients was negligible (Supplement Figure 1). Steps 2 and 3, the allocation and potential reallocation of ventilators respectively, were simulated on each patient as described in NYSVAG (Table 1; Supplement Figures 2, 3a-b). Patients categorized as Red received the highest priority for a ventilator, followed by those patients classified as Yellow. Patients classified as Blue were ineligible for a ventilator.

Table 1.

New York State ventilator allocation guidelines steps 2 and 3

	Step 2	Step 3
Ventilator color triage	Intubationassessment	Day 2 reassessment	Day 5 reassessment	Every subsequent2 day reassessment
Blue Ineligible for a ventilator	SOFA > 11	SOFA > 11SOFA 8-11 and no change in SOFA compared to prior assessmentIncrease SOFA compared to prior assessment	SOFA > 7 a SOFA ≤ 7 and no change in SOFA compared to prior assessmentIncrease SOFA compared to prior assessment	SOFA > 7 a Increase or no change in SOFA compared to prior assessment
Red Ventilator provided or maintained with highest priority	SOFA ≤ 7 a	SOFA ≤ 11 and decrease in SOFA compared to prior assessment	SOFA ≤ 7 and progressive decrease in SOFA compared to prior assessment (≥ 3)	SOFA ≤ 7 a and decrease SOFA compared to prior assessment
Yellow Ventilator provided or maintained with intermediate priority	SOFA 8-11	SOFA ≤ 7 and no change in SOFA compared to prior assessment	SOFA ≤ 7 and minimal decrease in SOFA compared to prior assessment (< 3)

Three gaps in NYSVAG that were addressed as follows: (1) Patients with a SOFA score of 7 on the day of intubation were categorized as Red; (2) individuals with a SOFA score greater than 7 on any assessment after 5 days of mechanical ventilation were ineligible for a ventilator; and (3) patients ventilated for 7 or more days who continued to show a SOFA score ≤ 7 and decreasing were placed in the Yellow category.

NYSVAG were applied to the cohort in the following simulation (see Supplemental Methods for additional details): Every patient’s SOFA score was calculated on the day of intubation, Day 2 of mechanical ventilation, Day 5, and every subsequent 2 days of mechanical ventilation. Any patient who was characterized as Red or Yellow would receive (or maintain) a ventilator; however, any patient characterized as Blue would have her or his ventilator withheld (or reallocated) on the day that patient was characterized as ineligible for a ventilator. This design was chosen because this analysis can evaluate a central component of the guidelines (SOFA score cutoffs and ventilator time trials) independent to the number of available ventilators, patient throughput, ventilator strain, or impact from randomization. Patients who had their ventilator withheld or reallocated were simulated to expire as modeled in other simulation studies withholding or withdrawing life-sustaining treatment. Patients were also removed from the simulation if they expired, were extubated, or placed on extracorporeal membrane oxygenation (ECMO) between assessment days.

Outcome measurements include both observed (what occurred in the real world) and simulated (what occurred hypothetically) endpoints. Because every patient in our cohort received a ventilator in the real world, their performance with respect to survival to hospital discharge with continued ventilation is known (referenced as an observed outcome). The primary observed endpoints include death or survival to discharge. The primary simulated endpoints include whether the patient’s ventilator was simulated to be withheld or reallocated on the day of intubation, 2, 5, 7, 9, or 11 days post-intubation (Figure 1). Combined simulated-observed endpoints include whether the patient had her or his ventilator reallocated (simulated) but was observed to survive to hospital discharge (observed, how the patient would have performed if provided with a ventilator). Comorbidities were queried from the ICD-10 problem list.

Figure 1.

Study Methodology. Patients were observed from time of intubation until observed endpoints of death or survival to discharge. Patient’s had SOFA reassessments on day 2, 5, 7, 9, and 11 to determine if their ventilator would have been reallocated in the simulation. Patients received a ventilator in the simulation if their SOFA score on day of intubation was less than 12. On the Day 2 Reassessment, patients would have had their ventilator reallocated if their interval SOFA score increased, did not change from an initial SOFA of 8-11, or was greater than 11. On the Day 5 Reassessment, patients would have had their ventilator reallocated if their SOFA score increased, was greater than 7, or was less than 7 and without a change in their previous SOFA score. On the Day 7 Reassessment and every 2 days thereafter, patients would have had their ventilator reallocated if their SOFA score did not improve or was greater than 7. A patient who did not have their ventilator reallocated in the simulation (No *) would only progress to the subsequent simulated ventilator reallocation assessment if they were not extubated, expired, or placed on ECMO prior to that assessment.

Missing data were amended in 2 subsequent simulations. First, all Glasgow Coma Scale (GCS) SOFA subscores were normalized (GCS-nl). Second, missing pulmonary SOFA subscores were supplemented by the modified SOFA pulmonary category (mSOFA-FiO2). As oxygen saturation was not gathered and FiO2 was, pulmonary mSOFA scores were computed with a presumptive oxygen saturation of 85% or better. A fourth simulation delayed the Day 2 reassessment to Day 5, the Day 5 reassessment to Day 7, and the subsequent 2-day reassessments to start on Day 9 (MVD 2to5).

Categorical data were analyzed with the chi-square test using the 95% confidence interval (CI). Continuous variables were compared with a 2-tailed t-test using a 5% significance level.

This study was approved by the NYU Grossman School of Medicine Institutional Review Board (study #i20-01653). No treatment, ventilator allocation, or reallocation decisions were made for any patient. Patients were exempt from informed consent.

Results

Observed Results

COVID-19 patients, 884 in total, were intubated between 3/1/2020 and 7/1/2020 and included in the simulation. The average age of the cohort was 63 years, predominately male (70.0%), with varied racial ethnicities (Table 2). Comorbidity information was only available for 50% the cohort (Supplement Table 1). The median (IQR) duration of mechanical ventilation for the cohort was 10.2 (3.74, 23.73) days, with 41.0% survival to discharge.

Table 2.

Patient cohort demographics

Characteristic	Variable
No. of patients	884
Age, mean (SD)	63.3 (14.6)
Male sex, N (%)	618 (70.0)
Length of mechanical ventilation(median days, IQR)	10.2 (3.74, 23.73)
Survival to discharge % (N)	41.0 (362)
Race ethnicity	Total cohort (%)
White	43.6
Hispanic	27.5
Black	12.3
Asian	11.4
Other	7.7
Unknown	4.6
Native American	0.5

Simulation Results

NYSVAG were applied to the observed patient SOFA scores in a simulation shown in Figure 2 and Table 3. Table 4 depicts how simulated patients, on each day of reassessment, were triaged as ineligible to receive or maintain a ventilator (Blue) and how many were observed to survive to discharge.

Figure 2.

Sankey diagram of a limited NYSVAG simulation on a cohort of COVID-19 patients. Each color represents the number of patients triaged to that color on the day of triage (Day 0 or potential intubation, Day 2, 5, 7, 9, and 11). Patients triaged as Red or Yellow received or maintained their ventilator until the next Reassessment Day while those triaged as Blue had their ventilator withheld or reallocated and were simulated to expire. Flow between each day of assessment describes the number of patients who moved between triage categories. The paths from left to the far right bars describe the course of patients who satisfied the combined simulated-observed endpoint of patients who had their ventilator reallocated (simulated) but were observed to survive to hospital discharge (observed, how the patient would have performed if provided a ventilator in the simulation). Patients exited the simulation because they were extubated, expired, or placed onto ECMO.

Table 3.

Numerical representation of a simulated application of NYSVAG to a cohort of critically ill COVID-19 patients

	Intubation(%){95% CI}	Interim (%)	Day 2(%){95% CI}	Interim (%)	Day 5(%){95% CI}	Interim (%)	Day 7(%){95% CI}	Interim (%)	Day 9 (%)	Interim (%)	Day 11 (%)
Total N	884		678		226		56		6		2
Red	478(54.1){50.8-57.4}		223(32.9){29.5-36.5}		25(11.1){7.0-15.2}		7(12.5){6.2-23.6}		2(33.3)		0
Red who survived to discharge a	221(46.2){41.8-50.7}		126(56.5){48.6-61.5}		22(88.0){75.3-100}
Yellow	285(32.2){29.2-35.4}		50(7.4){5.6-9.6}		39(17.3){12.3-22.2}
Yellow who survived to discharge a	110(38.6){33.1-44.4}		24(48.0){34.2-61.9}		22(56.4){40.9-72.0}		5(71.4)		1(50.0)		0
Blue b	121(13.7){11.6-16.1}		404(59.7){55.9-63.3}		162(71.7){65.5-77.2}		49(87.5){76.4-93.8}		4 (66.7)		2(100)
Blue who survived to discharge c	31(25.6){18.7-34.1}		136(33.7){28.6-37.8}		75(46.3){38.8-54.0}		34(69.4){55.5-80.5}		3 (75.0)		1(50.0)
ECMO		10(11)		7(15)		0		0		0
Extubated Survived		35(41)		24(51)		5(63)		1(100)		0
Deceased		41(38)		16(34)		3(37)		0		0

Numerical representation of Figure 2 depicting the simulated application of NYSVAG on a cohort of critically ill COVID-19 patients.

Survival to hospital discharge is an observed outcome.

Characterized as ineligible for a ventilator (Blue) and simulated for ventilator removal/withholding is a simulated outcome.

Combined simulated-observed outcome; those who were classified as ineligible for a ventilator (Blue) in the simulation but were also observed to survive to hospital discharge when they were provided a ventilator in the real world.

Table 4.

Etiology of patients triaged as Blue and ineligible for a ventilator during simulation

BlueReason for ventilator ineligibility	Intubation	Day 2	Day 5	Day 7	Day 9	Day 11
	Number survived to discharge/Total (%) {95% CI}
SOFA > 11	31/121(25.6){18.7-34.1}	21/110(19.1){12.7-27.5}	1/13(7.7){0-35.4}	0/0	0/0	0/0
SOFA 8-11 a	N/A	72/204(35.3){29.1-42.1} P = 0.003 b	25/61(41.0){29.5-53.5} P = 0.02 b	1/7(14.3)	0/1(0)	0/0
SOFA ≤ 7 ↑ a	N/A	43/90(47.8){37.8-58.0} P < 0.001 b P = 0.04 c	49/88(55.7){45.3-65.6} P = 0.001 b P = 0.07 c	33/42(78.6.){63.9-88.5} P < 0.001 c	3/3(100)	1/2(50)
Total	31/121(25.6){18.7-34.1	136/404(33.7){28.6-37.8}	75/162(46.3){38.8-54.0}	34/49(69.4){55.5-80.5}	3/4(75)	1/2(50)

Etiology of patients classified as Blue and triaged as ineligible for a ventilator on the specified reassessment day of mechanical ventilation.

See Figure 1 and Table 1 regarding specific criteria triaging patients as Blue or ineligible to receive or maintain a ventilator for the specific day reassessed.

P value comparing SOFA > 11 of the same day.

P value comparing SOFA 8-11 of the same day.

Step 2: Assessment at intubation

In total, 884 patients underwent Step 2 of NYSVAG—prioritization and allocation of a ventilator. Average SOFA on the day of triage for mechanical ventilation was 7.4 ± 3.3; 478 (54.1%) individuals qualified as Red, the highest priority for a ventilator, 285 (32.2%) as Yellow or intermediate priority for a ventilator, and 121 (13.7%) as Blue or ineligible for a ventilator. Of the 121 individuals triaged as Blue and not provided with a ventilator in the simulation, 31 (25.6%) were observed to survive to discharge. Of the 763 patients who were initially prioritized to receive a ventilator (Red or Yellow), 249 (32.6%) would later be triaged in the simulation to have their ventilator reallocated (Blue) but would have otherwise been observed to survive to discharge (observed outcome).

Step 3: Day two reassessment

From the day of intubation to 2 days post-intubation, 41 individuals had expired, 35 had been extubated, and 10 had been placed onto ECMO. As such, 678 patients were reassessed on post-intubation Day 2. Of this group, 223 (32.9%) were characterized as Red and 50 (7.4%) as Yellow. Per NYSVAG, for those being reassessed after 2 days of mechanical ventilation, 404 individuals (59.7%) would have had their ventilator reallocated, 136 (33.7%) of whom were observed to survive to discharge. Among these 404 individuals, both the 90 individuals (22.3% of all reassessed as Blue on this reassessment, 47.8% observed survival) who had a worsening SOFA score less than 7 and the 204 individuals (50.5%, 35.3% survival) who had a worsening SOFA score of 8-11 were more statistically likely to be discharged alive if provided a ventilator than those 110 individuals (27.2%, 19.1% survival) who had a SOFA score greater than 11 (P < 0.001 and P = 0.003, respectively). Individuals characterized as Blue with a worsening SOFA score less than 7 were also more likely to survive to hospital discharge with a ventilator than those with a SOFA score of 8-11 (P = 0.04).

Step 3: Day five reassessment

Of the 678 patients from Day 2, 24 individuals were extubated, 16 expired, 7 were placed onto ECMO, and 405 were extubated due to the ventilator reallocation that occurred because of the Day 2 reassessment. As such, 226 patients were present to be reassessed after 5 days of mechanical ventilation. At this time, 25 individuals (11.1%) were triaged as Red and 39 (17.3%) as Yellow; 162 patients (71.7%) were triaged as Blue and were simulated to have their ventilator reallocated, 75 (46.3%) of whom were observed to survive to discharge. Among these 162 individuals, both the 88 individuals (54.3% of all reassessed as Blue, 55.7% observed survival) who had a worsening SOFA score less than 7 and the 61 individuals (37.7%, 41.0% survival) who had a SOFA score of 8-11 were more statistically likely to be discharged alive if provided with a ventilator than those 13 individuals (8.0%, 7.7% survival) who had a SOFA score greater than 11 (P = 0.001 and P = 0.02, respectively).

Step 3: Day seven reassessment

Of the 226 patients from Day 5, 5 individuals were extubated, 3 expired, and 162 were extubated due to the ventilator reallocation on the Day 5 Reassessment. As such, 56 patients were present to be reassessed after 7 days of mechanical ventilation. At this time, 7 individuals were prioritized for a ventilator. 49 patients (87.5%) were characterized as Blue and simulated to have their ventilator reallocated, 34 (69.4%) of whom were observed to survive to discharge. Amongst these 49 individuals, the 42 individuals (85.7% of all reassessed as Blue, 78.6% observed survival) who had a worsening SOFA score less than 7 were more statistically likely to be discharged alive if provided with a ventilator than and the 7 individuals (14.3%, 14.3% survival) who had a SOFA score of 8-11 (P < 0.001).

After 11 days of simulation, all 884 patients would have either been extubated and survived to hospital discharge, expired, had their ventilator reallocated, or placed onto ECMO; 742 patients would have had their ventilator reallocated (83.9% of the original cohort), 280 (37.7%) of whom were observed to survive to discharge if provided with a ventilator. Of the total 360 patients observed to survive to discharge, only 65 (18.1%) would have survived to discharge without having their ventilator reallocated in the simulation. There was no statistically significant ventilator reallocation by age, gender, race, or ethnicity at any day of assessment, among all who had their ventilator reallocated, or among those who were simulated to survive.

Missing data and additional analyses

Our cohort required 1852 patient-SOFA-days (a SOFA score for each day a ventilated patient was assessed). The majority of missing information was on the day of intubation, with 100 patients having no SOFA score on the day of intubation (Supplement Table 2). Additional modeling with adjusted or supplemented SOFA scores, as well as deferring NYSVAG Day 2 reassessment to Day 5, is shown in Figure 3. We failed to find a significant difference in simulated surviving individuals when normalizing the GCS or supplementing pulmonary SOFA subscores (72 GCS-nl versus 65; see Figure 3). A delay in the Day 2 reassessment to Day 5 (MVD 2to5) resulted in a 49.2% increase in simulated survival to discharge within NYSVAG criteria.

Figure 3.

Variations on the limited NYSVAG simulation on a cohort of critically ill COVID-19 patients. Solid represents total remaining patients. OG, original applied NYSVAG. GCS-nl, normalizing all GCS SOFA subscores. mSOFA-FiO2, missing pulmonary SOFA subscores are supplemented with mSOFA pulmonary subscore presuming an oxygen saturation greater than 85%. MVD 2to5, moving NYSVAG OG Day 2 reassessment to Day 5. Dashed lines represent the total number of simulated patients who were extubated & survived to hospital discharge (combined endpoint).

Discussion

Our findings simulate the performance of common triage criteria (SOFA cutoffs and ventilator time trials) on a large cohort of critically ill COVID-19 patients. Ventilator reallocation was simulated to occur for 83.9% (742/884) of the cohort, 37.7% (280/742) of whom would have survived to discharge if provided with a ventilator. Additionally, only 18% (65/362) of all individuals who were observed to survive to discharge were also simulated to survive to discharge within the simulated NYSVAG criteria. This raises a significant concern that under this ventilator allocation protocol, ventilators might be taken away from patients who would otherwise survive. This concern would be assuaged if the subsequent patients who received the reallocated ventilators would have fared much better, thereby improving overall group survival. This would satisfy the NYSVAG utilitarian goal of doing the most good for the most people, commonly interpreted as saving the most lives.

Others have articulated that any redistribution of lifesaving resources must be fair, transparent, and promote equity among vulnerable and marginalized populations. It is therefore encouraging that any ventilator distribution per NYSVAG was not statistically significant among any age, gender, ethnic, or racial groups.

Our results contrast with that reported by Wunsch et al., a similar retrospective simulation of applying NYSVAG to 40,439 intubated critically ill patients from 2014-2015. Our cohort of COVID-19 patients was sicker than their mixed medical/surgical population (initial COVID-19 SOFA of 7.4 ± 3.3 compared to 4.5 ± 3.7), which may account for the larger proportion triaged as ineligible for a ventilator (Blue, 13.7% vs 8.9%). On the second or fifth reassessment day, only 4.7% or 28.3% of their cohort was classified as ineligible for a ventilator (Blue), respectively. This contrasts concerningly to the 59.7% and 71.7% simulated in our cohort of COVID-19 patients. Moreover, their median (IQR) duration of mechanical ventilation was 1.94 (0.7, 5.1) days, whereas our COVID-19 patient cohort required a median of 10.2 (3.74, 23.73) days. COVID-19 patients do not recover on a similar trajectory to other critical respiratory illnesses, including influenza, and allocation guidelines may need to extend their reassessment intervals. Determination of the appropriate length of time for a trial of mechanical ventilation can be difficult since the physiologic trajectory of the disease—even with optimal treatment—may be unknown or variable. Too short a trial of ventilation and patients may have an inadequate length of time to demonstrate a clinical benefit. Short of objective evidence that a patient is failing therapy, premature reassessment that results in the removal of resources that might otherwise later benefit the patient may promote rapid cycling between patients irrespective of prognosis and worsen overall mortality. Too long between reassessments might lead to disproportionate resource utilization without optimizing overall survival.

Our results may signal that the recommended length of time for a ventilator time trial was too short to demonstrate improvement for COVID-19 patients. This is supported by the observation that ventilator reallocation would have occurred for over half of the simulated remaining patients on or after 2 days of mechanical ventilation, of which over one third were observed to survive to discharge (see Table 3). It is also corroborated by the 49.2% increase in patients who would survive to discharge if the second day of assessment of ventilator allocation was delayed to Day 5 (see Figure 3). This may suggest that the appropriate length of a time trial should be extended for COVID-19 patients compared to other typical causes of respiratory failure.

Potential Limitations

This study has several potential limitations. Although this simulation is not based on a fixed supply of ventilators and includes an unrealistic dynamic number of available ventilators, it provides an important analysis that best represents the true performance of commonly used SOFA cutoffs and reassessment intervals with respect to patient mortality. This design is crucial because guideline underperformance might be obscured under circumstances of low ventilator strain where the majority of patients are provided with a ventilator regardless of their eligibility. A true understanding of which patients are classified as ineligible for lifesaving resources and their chances of survival remains paramount to understanding how guidelines might perform with increasing resource strain. This cohort study also facilitates a unique opportunity to understand how many patients would have had their ventilator withheld (simulated) but would have otherwise survived if a ventilator was provided (because they were provided with a ventilator in the real world). Our simulation demonstrates the important concern that, according to the NYSVAG criteria studied, many patients who may have had their ventilators withdrawn would have subsequently survived. This concern would be mitigated, and NYSVAG indeed validated, if subsequent patients receiving reallocated ventilators had a higher rate of survival. We acknowledge that our study design cannot explicitly assess the mortality of subsequent patients receiving reallocated ventilators. However, our results suggest that patients ventilated for 5 or more days and selected to have their ventilator reallocated may have a higher chance of surviving to discharge with a ventilator (46.3-75.0%) than subsequent patients receiving their reallocated ventilator (43.4%) (see Table 3). This conjecture, if substantiated, may have significant implications as to how new patients should be triaged in the setting of existing ventilated patients. The survival of patients receiving reallocated ventilators must be explicitly studied before changes to existing protocols are proposed and adopted.

A simulation including a fixed number of available ventilators and patient throughput incorporating ventilator strain will produce different results and allow for the explicit study of patients receiving reallocated ventilators. In such a scenario, however, it remains concerning that the majority of patients classified as Blue on or after 5 days of mechanical ventilation had a SOFA score of less than 7. Randomization may not maximize patient survival if it occurs among all Blue individuals and does not prioritize subcategories that are more likely to survive (see Table 4). A triage officer or committee may provide a more sensitive selection process; however, NYSVAG only permits randomization within a priority color. (p. 68) Further subcategorization of all patients classified as ineligible for a ventilator should be considered in resource allocation protocols. Subsequent studies incorporating a fixed number of available ventilators, real patient throughput, and the comparison of alternative triage strategies such as first-come first-served should be simulated. Bhavani et al. performed a limited simulation; however, they do not incorporate patient throughput or dynamic real-world surge strain.

We believe our results are broadly generalizable. While NYSVAG was designed for an influenza pandemic, the NYSVAG Q&A approve its application for a broader respiratory crisis. Although we analyzed limited aspects of NYSVAG, these elements are similar to several other resource allocation policies. We believe our sizeable cohort is generalizable to other COVID-19 patients as our SOFA score and PaO2/FiO2 at intubation and duration of mechanical ventilation are consistent with other reports. Our high mortality is consistent with other local reports during the study period and likely related to the unprecedented surge and associated ICU strain that would be consistent in communities activating crisis standards of care. . Our NYSVAG simulation results are additionally similar to a 200-patient mixed COVID-19 cohort from a comparable local institution. While all ventilated patients would participate in the same resource allocation policy, non-COVID-19 patients were excluded from this simulation because they were marginally present during our institutions’ surge.

There remain specific limitations related to the general dependence on SOFA scores for reallocation policies. While an initial report suggested that SOFA scores could predict mortality among COVID-19 patients, subsequent studies have not been as supportive. Despite such shortcomings, using an objective score to prioritize resource allocation with the intent of saving the most lives achieves impartiality, preserves fairness toward patients, promotes transparency with the community, maintains reproducibility between providers, eliminates the encroachment of bias or prejudices by decision-makers, and may reduce provider distress about ventilator allocation decisions. Such scores may require further augmentation to strive for equity and not worsen existing health disparities, or to better reflect community values. While many critically ill COVID-19 patients are at risk of multiple organ sequelae, some remain “single organ” dysfunction with respiratory failure, whereupon SOFA scores may not be the most accurate reflection of the severity of their illness.

Dependence on SOFA scores can be additionally problematic as COVID-19 patients often require prone positioning, which can complicate the interpretation of the oxygenation component, or heavy sedation or paralytics, which can interfere with the neurological assessment. In our simulation, GCS scores were normalized when they were not documented, which was overwhelmingly common and may be related to patients’ clinical contraindications for interruption of sedation or paralytics versus incomplete documentation. The frequency of missing data in Supplemental Table 2 underscores this point and may only be partially mitigated by using the mSOFA score. Our normalization of missing information underestimates a patient’s true SOFA score and therefore may underestimate the frequency of ventilator reallocation; 11.3% of our cohort was intubated without any SOFA information, possibly suggesting that first responders may need to make triage decisions with incomplete information.

Conclusion

States across the nation continue to battle against surges in the COVID-19 pandemic. Federal, state, and local governments should continue to study and develop their disaster preparedness plans. Our results suggest that if NYS Ventilator Allocation Guidelines were implemented, patients might have their ventilator reallocated when they might otherwise survive. Further studies should determine whether subsequent patients receiving reallocated ventilators would have a higher survival than the original cohort before changes to the guidelines are enacted.