Development of an Electronic Medical Tool to Facilitate Allocation of Limited Resources in Times of Crisis.

The COVID-19 pandemic has made decisions about resource allocation and reallocation real possibilities even in high-resource settings. In April 2020, in preparation for such an eventuality, Atlantic Health System began to develop a real-time instrument built into the EMR to assist with such decisions. The instrument calculated the modified Sequential Organ Failure Assessment for all patients admitted, in real time, to assist triage teams make decisions if crisis standards of care were declared. A pilot assessment of the instrument was performed using retrospective data by nine members of the triage teams, who were asked to identify the six patients at highest risk of reallocation. Agreement about which patients were at highest risk of resource reallocation was good, but not perfect. All raters agreed on five of the six patients, but only seven of nine agreed on the final patient. Among the six consensus selections for reallocation, five died prior to hospital discharge. All patients at highest risk of reallocation had a predicted life expectancy of less than 1 year. In conclusion, the instrument was easy to use, and the concordance among raters was good but not perfect. Predicted life expectance was a major determinant of the triage score.

Since its emergence in late 2019[1,2] SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) has spread globally and, as of July 28, 2021, has infected more than 195 million people and caused the death of more than 4.1 million globally.[3] In the United States, an early first wave of infections occurred in the northeastern states, including New Jersey and New York, which peaked in mid-April 2020.[4,5] As hospital admissions and the need for scarce resources such as ICU beds, critical care staffing, and mechanical ventilators increased, concerns were expressed both in the medical literature and in the lay press that supply of critical care resources would be inadequate to meet demands.[6,7]

Resource allocation during times of scarcity is morally and ethically complex. This is especially true in countries such as the United States, where the right to medical care is usually considered a negative right of noninterference, meaning that there is the strong expectation that medical decisions made between an individual patient and his/her physician are a private matter and should be respected as such. However, during periods of scarcity, the question becomes not whether there should be a method of determining allocation of resources between competing individuals who all desire those resources, but how it should be done most ethically, equitably, and transparently.[8]

Although different countries have approached this problem in a variety of ways,[9] it is generally understood that allocation processes should serve the utilitarian purpose of maximizing benefits; therefore, scarce resources should be used where and when they will do the most good.[9] Such decisions may present themselves in different ways, including: (1) allocation: if two patients require a ventilator and only one is available, who should receive it? (2) Reallocation: should a ventilator be reallocated (taken from) to one person and provided to another who is more likely to benefit from it? In practice, however, these are morally equivalent decisions as both aim to use scarce resources to do the most good for the greatest number of patients.

Such ethical challenges are not unique to the COVID-19 pandemic. For example, in 2015, New York State published revised guidelines for ventilator allocation during a potential influenza pandemic.[10] These guidelines mapped a three-step process for future ventilator allocation: (1) identifying and excluding patients with other comorbidities that would preclude benefit from mechanical ventilation, (2) the use of an clinical acuity score to identify those unlikely to survive despite mechanical ventilation, and (3) an ongoing reassessment of the continuing value (or otherwise) of mechanical ventilation.[10] The clinical acuity score suggested in these guidelines[10] is the Sequential Organ Failure Assessment (SOFA) score.[11,12]

The Institute of Medicine has defined crisis standards of care as “a substantial change in usual healthcare operations and the level of care it is possible to deliver, which is made necessary by a pervasive (e.g., pandemic influenza) or catastrophic (e.g., earthquake, hurricane) disaster … and is formally declared by a state government … [this] enables specific legal/regulatory powers and protections for healthcare providers in the necessary tasks of allocating and using scarce medical resources and implementing alternate care facility operations.”[13] On April 11, 2020, the State of New Jersey published executive orders to facilitate healthcare institution's preparations for possible allocation/reallocation decisions should crisis standards of care be declared. If hospital or ventilator capacity were exceeded and crisis standards of care declared, limited legal protections were provided if institutions followed state guidelines.[14] This largely mirrored the guidelines from the University of Pittsburgh,[15,16] which, consistent with many other US state guidelines, utilized the SOFA score as the primary means by which acuity was defined.[10,17] However, extracting such information from the medical record is time-consuming and prone to human error, especially if a large number of patients are using scarce resources and are potentially eligible for reallocation of these resources. Therefore, in preparation for possible crisis standards of care, Atlantic Health System (AHS) began to develop a process that included a specially designed instrument that allowed instantaneous, real-time calculation of SOFA scores embedded within the EMR (Epic; Epic Systems, Madison, WI, USA).

In this article, we describe the development of such an instrument and its content validity: do different users of the instrument reach similar triage decisions?

METHODS

Setting

Atlantic Health System is a large not-for-profit healthcare system in Northern New Jersey. It includes five inpatient acute care medical facilities, including two large teaching hospitals: Morristown Medical Center (MMC), a 693-bed tertiary care academic medical center in Morristown, New Jersey, and Overlook Medical Center, a 504-bed teaching hospital in Summit, NJ; and three medium-sized community hospitals with fewer than 300 licensed beds: Chilton Medical Center, Newton Medical Center, and Hackettstown Medical Center. Each of the five hospitals has at least one or more ICUs that are medically managed by either in-house or contracted physician providers.

Triage Procedure

In April 2020, AHS adopted policies consistent with the executive order from the State of New Jersey Department of Health on the allocation of critical care resources during a public health emergency.[14] The purpose of this policy was to provide guidance for the triage of critically ill patients in the event that the State of New Jersey declared that a public health emergency had created a demand for critical care resources (including ventilators, critical care beds, or specialized medical and nursing care) that may exceed their availability.[18]

In preparation for the possibility that the COVID-19 pandemic would necessitate that the State of New Jersey would need to declare a public health emergency, contingency plans were developed. Atlantic Health System constituted four triage teams to be on call in rotating 12.5-hour shifts to provide coverage 24 hours a day for all sites to address questions of allocation and reallocation. Each triage team consisted of an intensive care physician, intensive care nurse, and a designated healthcare ethics consultant. Training and simulation classes and additional individual follow-up sessions were conducted, and the teams were ready to be deployed, had the need to implement allocation protocols occurred.

The state-mandated algorithm avoided the use of categorical exclusion criteria that would preclude the use of a mechanical ventilator (or any other scarce resource) in any given patient and aimed to be equitable, open, and transparent and avoid implicit biases based on patient factors that were unrelated to their likelihood of benefiting from the scarce resource (ie, unbiased by gender, ethnicity, religion, sexual orientation, social standing, insurance status, etc). At the core of the algorithm was the calculation of a triage score that combined the SOFA score (Table 1) as a measure for the likelihood that the scarce resource would save the person's life during this acute phase, and an assessment of the amount of life-years that might be saved as a result (Table 2). This generated a raw triage score between 1 and 8, with lower scores being those patients who would benefit most from the scarce resource both in terms of survival and length of additional life. The triage scores were used to categorize patients into three priority levels (Table 3). If several individuals were tied at the same priority level, the suggested tie-breaker was life stage with the highest priority given to individuals aged 0 to 17 years, followed in order by those aged 18 to 40, 41 to 60, 61 to 75 years, and finally to those older than 75 years. These criteria were not selected because older individuals were felt to be of less value than younger individuals, or to be less deserving of scarce recourses; rather, the guiding ethical principle was that of equity and allowing younger individuals an equal opportunity to pass through life stages that older individuals had already done.

Table 1

Components of the SOFA as Modified From Vincent et al[11,12]

Score		0	1	2	3	4
System	Parameter
Respiration	Pao2/Fio2 (mm Hg)	>400	≤400	≤300	≤200 and on mechanical ventilation	≤100 and on mechanical ventilation
Coagulation	Platelet count (×103/μL)	>150	≤150	≤100	≤50	≤20
Liver	Bilirubin (mg/dL)	<1.2	1.2–1.9	2.0–5.9	6.0–11.9	>12.0
Cardiovasculara	Hypotension	No hypotension	Mean arterial pressure <70 mm Hg	DA ≤5 or DB	DA >5 or epi ≤0.1 or ne ≤0.1	DA >15 or epi >0.1 or ne >0.1
CNS	Glasgow Coma Scale score	15	13–14	10–12	6–9	<6
Renal	Creatinine (mg/dL) or	<1.2	1–2 to 1.9	2.0–3.4	3.5–4.9	>5.0
	Urine output (mL/d)				<500	<200

Abbreviations: DA, dopamine; DB, dobutamine (any dose); epi, epinephrine; ne, norepinephrine.

aDoses of pressors are given as μg/kg per minute. The AHS version of the SOFA score did not include dobutamine.

Table 2

Calculation of the Triage Score

Points		Score
Principle	Factor	1	2	3	4
To save lives	Short-term prognosis for survival	SOFA score <6	SOFA score 6–8	SOFA score 9–11	SOFA score ≥12
To save life-years	Long-term prognosis for survival after hospital discharge		Life expectancy <5 y despite successful treatment of acute condition		Life expectancy <5 y despite successful treatment of acute condition

Table 3

Relationship Between Triage Scores and Priority Score/Code

Triage Score	Priority Code	Priority Description
1–3	Red	Highest priority (most likely to benefit)
4–5	Orange	Intermediate priority (reassess as needed)
6–8	Yellow	Lowest priority (reassess as needed)

When AHS began preparations for triage teams, and for potential allocation/reallocation decisions, there were more than 100 patients who were being mechanically ventilated across the AHS system, and all would be potential subjects for reallocation during a public health emergency. It was improbable that the triage teams would be able to assess all these patients in real time and calculate an accurate SOFA score and an accurate priority score, in a timely enough fashion to make real-world triage decisions. Even if this were possible, the likelihood of human error and miscalculation under such fraught and stressful conditions seemed very high. As the algorithm is objective and deterministic and avoids subjective factors or nuanced interpretation of information, it seemed suitable to be calculated automatically for all patients admitted, in real time, using data already present in the EMR; AHS therefore set about developing an EMR-based instrument to calculate SOFA scores and priority scores to assist decision-making by triage teams in the event that the demand for critical care resources outstrips the supply, and a public health emergency was declared.

Therefore, AHS developed an EMR-based algorithm to assist triage teams. The group tasked with ensuring successful completion of this project included a subgroup of an existing AHS group empaneled to examine and respond to the ethical implications of COVID-19 (I.G., Y.V., M.A.D.-R., G.P., D.M.), in addition to members of the AHS Information Technology team with specific experience of in EMR/Epic implementations (N.P., S.C.). The group first met on March 17, 2020, and a fully functioning Epic-based instrument and associated training materials were ready for use by April 9, 2020.

EMR-Based Instrument Development

This consisted of four parts:

Calculation of the SOFA score based on the most recently available clinical and biochemical data,

Determination of life expectancy (<1, <5, >5 years) should the patient survive the acute illness,

Combination of SOFA score and anticipated life expectancy to produce a composite triage score, and

Presentation of these data to triage team members in a format that would allow real-time allocation/reallocation decision-making in the form of recommendations to the clinical team.[17]

Determination of SOFA Score

The SOFA score consists of six components, each retrieving specific elements from database queries within the EMR (Table 1). Scores were automatically filed every hour from searches of laboratory results (Pao2, platelet count, total bilirubin), medications (dopamine, epinephrine; including prescribed dosage), and flowsheet documentation (Fio2, mean arterial pressure, Glasgow Coma Scale score, urine output). To calculate the contributing Pao2/Fio2 ratio, the system searched the most recent Pao2 laboratory value and Fio2 documentation in reverse chronological order over the last 6 days. Coagulation was calculated from the most recent platelet laboratory result in reverse chronological order with a time range of 8 days. Contributing points for the liver component were extracted from the patient's most recent total bilirubin laboratory value in the last 8 days. The cardiovascular component retrieved the maximum dose of dopamine, epinephrine, and norepinephrine in the last 28 hours. Atlantic Health System's SOFA adaptation did not consider dobutamine into the SOFA calculation. The minimum value for a patient's mean arterial pressure was considered in reverse chronological order with a time range of 8 days. Atlantic Health System's SOFA model accounts for both noninvasive and arterial line mean arterial pressure readings from clinician documentation. The Glasgow Coma Scale score was obtained from searching the most recent documentation in the past 8 days. Finally, a patient's renal value was calculated from a patient's documented urinary status and creatinine. If the patient was documented as anuric, the system awarded the highest point value. If the patient was not anuric, Epic calculated the score from the most recent creatinine laboratory value over the past 8 days.

The SOFA specifications utilized values within a 24-hour look-back window; however, the AHS triage team noted laboratory values sometimes resulted outside of the strict 24-hour window, creating inconsistencies in the scores. We therefore increased the look-back to 28 hours to account for this variation in workflow and later expanded to a 6- or 8-day window for most data.

Determination of Predicted Life Expectancy

Determination of predicted life expectancy, aside from the acute reason for admission, was assessed in two stages. The initial determination was made using operational definitions previously described in making triage decisions for the allocation of mechanical ventilators.[19] Examples of diagnoses predicting less than 1-year survival included cancer being treated only with palliative care, severe Alzheimer disease (or related dementia), cirrhosis of the liver with MELD score ≥20 and ineligible for liver transplantation, heart failure with New York Heart Association class IV disease, and evidence of frailty, severe chronic lung disease with evidence of frailty, or end-stage renal failure in patients older than 75 years. Examples of Concept Diagnostic Groupers codes predicting survival of 1 to 5 years included moderate Alzheimer disease (or related dementia), cirrhosis of the liver with a history of decompensation, heart failure (New York Heart Association class III), moderately severe chronic lung disease (chronic obstructive pulmonary disease or idiopathic pulmonary fibrosis), severe multivessel coronary artery disease, and end-stage renal failure in patients aged younger than 75 years.[19] These definitions were mapped to Epic (Epic Systems Corporation, Verona, WI, USA) Concept Diagnostic Groupers, using the problem list grouper based on International Classification of Diseases (9th and 10th revisions) and SNOMED CT terms to classify patients based on their expected survival (Table 2). The second stage of the procedure was envisaged as an in-person, or virtual, discussion between the triage team and the patient's physician to determine that the screening assessment based on diagnostic codes accurately reflected that individual patient's prognosis as determined by his/her physician using normal clinical judgment (see below).

Determination of Triage Score

The data extracted for the SOFA score components and the patient's life expectancy were converted to their numerical equivalents (Table 3) and summed to produce the raw triage score. When presented to triage team members within Epic, they were color coded as red, orange, or yellow to simplify data interpretation.

Discussion With Primary Care Team

The final step in the process was to be direct conversations with the patient's primary physician. As the 1- and 5-year survival data were important in driving allocation/reallocation decision, it was felt to be vital to confirm that the prognosis determined using Epic Concept Diagnostic Groupers and SNOMED codes was in agreement with the physician’s best assessment of the patient's life expectancy. Similarly, the primary care team could review and correct the calculation of the SOFA score if they thought that clinical data had been misinterpreted.

Timeline

Early in the pandemic, AHS recognized that it may present challenging ethical problems and convened a COVID-specific ethics committee to address any such issues. This committee first met on March 17, 2021, and identified ethical issues around allocation of scarce resources to be a priority. A subgroup was formed to address the open, transparent, and timely implementation of allocation/reallocation decisions, which first met on March 25, 2021, and identified the need for an EMR-based triage instrument to assist in timely decision making. The triage instrument was completed, training documents produced, and plans for its operationalization (if needed) made by April 9, 2021. Training of members of the triage team began shortly thereafter. Ethical approval of the triage instrument was not required. The pilot study of content validity of the instrument was exempted from institutional review board oversight.

Training of Team Members in Application of the Algorithm

The triage teams consisted of a critical care physician, a critical care nurse, and a healthcare ethicist. It was felt that the inclusion of healthcare ethics in the triage team would provide an additional layer of oversight based on the ethical principles utilized by the policy/protocol, as well as support for the clinicians who may be personally and morally challenged by the role they were asked to fulfill. The bioethicist's role was not to clinically evaluate patients per se but instead ensure that ethical principles were followed, bias was controlled for, conflict resolution was managed within the teams, and appropriate decision making was applied in an equivalent manner throughout. The ethicists who participated in the triage teams had experience as critical care clinicians (whether as nurses or physicians), except for one who was nationally certified in healthcare ethics by the American Society of Bioethics and Humanities. Training was carried out virtually (either by telephone or by video conference). The triage process was explained in detail, and the appropriate approach to applying the algorithm. Triage team members were shown where the different components (ventilator settings, medications, etc) could be found in the EMR. They were also oriented to the EMR-based environment for applying the algorithm, and worked examples of triage assessments were demonstrated.

Content Validity of the Instrument

Nine members of the planned triage teams were asked to individually assess which patients might hypothetically be eligible for ventilator reallocation. They were provided access to a cached version of the SOFA instrument for the same day and hour in mid-April 2020 when the hospital census of COVID-19 patients was at its highest (Figure 1). They were asked to consider all patients receiving mechanical ventilation (for whatever reason) for more than 120 hours and identify the top three candidates for reallocation at MMC, the top three candidates for reallocation at Overlook Medical Center, and the top candidate for reallocation at each of the three remaining institutions. Those patients who were categorized in yellow with a score of between 6 and 8, according to the protocol, were deemed to have the lowest priority for the scarce resources in the total census because they had the lowest likelihood of benefit from the continued use of the scarce resource. Therefore, patients with yellow priority scores were considered potentially eligible for resource reallocation.

FIGURE 1

Inpatient census of COVID-19–positive patients within the Atlantic Health System (March 2020 to June 2020 inclusive).

External Validity of the Instrument

Although not the primary purpose of this report, the nature of the dataset used in the study allowed short-term outcomes of the patients identified as possible candidates for reallocation of scarce resources to be assessed. These outcomes included whether patients survived to hospital discharge and, if so, where they were discharged to and the level of care they required after discharge.

RESULTS

Development of Epic Instrument

The results of the SOFA/triage instrument were available to members of the triage teams within the “My Reports” workspace of the EMR (Epic; Epic Systems Corporation). By default, data were collected on all patients who were inpatients within AHS and were updated hourly. As patients with COVID-19 often worsen before they improve, it was decided that all patients would be allowed 120-hour ventilation as a “trial of therapy” before being considered for reallocation. Therefore, results were filtered to limit patients to those who had received mechanical ventilation for more than 120 hours. Each patient had data summarized for a variety of variables including facility/unit, age, length of stay, SOFA score, life expectancy (<1 year, <5 years), raw triage score, and so on (Supplemental Digital Content 1, http://links.lww.com/CIN/A148), and data could be ordered using any of these variables. When an individual row was highlighted, a secondary panel below the summary panel presented more detailed data, for example, whether any component used to calculate the SOFA score was missing (in which case it was flagged and highlighted in orange), the total SOFA score, the individual components of the score, the change in SOFA score over the previous 72 hours, a summary of how the SOFA score was calculated, and relevant laboratory and flowsheet data used to calculate the score, and a summary of the patient’s problem list and medications (in order to assess for medications that might confound the interpretation of the Glasgow Coma Scale score) (Supplemental Digital Contents 2 and 3 http://links.lww.com/CIN/A149, http://links.lww.com/CIN/A150).

Reallocation

The study was carried out using archived data for 213 patients across the five hospitals of the AHS system that were ventilated for any reason, of whom 198 were known or suspected to have COVID-19. Twenty-six patients were in the yellow priority code (and therefore potentially eligible to have their ventilators reallocated); all had expected life expectancies of less than 1 year. Conversely, 87% of patients with a life expectancy of less than 1 year were in the yellow priority group (Table 4). The situation was similar for the 126 patients who had been ventilated for more than 120 hours. All the patients in the yellow priority group had life expectancies of less than 1 year, and 77% of the patients with life expectancies of less than 1 year were in the yellow priority group (Table 5).

Table 4

Distribution of Expected Survival of <1, <5, and >5 Years Among the Red (Triage Score 1–3), Orange (Triage Score 4–5), and Yellow (Triage Score 6–8) for Patients Ventilated at the Time of the Study Across the Entire Atlantic Health System

Priority Code (Triage Score)	Life Expectancy <1 ya	Life Expectancy <5 y	Life Expectancy >5 y	Total
Red (1–3)	0	2	154	156
Orange (4–5)	4	8	23	31
Yellow (6–8)	26	0	0	26
Total	30	10	177	213

aBy definition, all patients with a life expectancy of <1 year also had a life expectancy of <5 years.

Table 5

Distribution of Expected Survival of <1, <5, and >5 Years Among the Red (Triage Score 1–3), Orange (Triage Score 4–5), and Yellow (Triage Score 6–8) for Patients Ventilated for ≥120 Hours at the Time of the Study Across the Entire Atlantic Health System

Priority Code (Triage Score)	Life Expectancy <1 ya	Life Expectancy <5 y	Life Expectancy >5 y	Total
Red (1–3)	0	2	95	97
Orange (4–5)	3	4	15	19
Yellow (6–8)	10	10	0	10
Total	13	16	110	126

aBy definition all patients with a life expectancy of <1 y also had a life expectancy of <5 y.

Nine raters (four RNs, four physicians, and one medical ethicist) assessed which patients were candidates for ventilator reallocation. One rater chose not to follow the algorithm for patients at MMC, so their data for MMC were excluded. At MMC, all remaining raters agreed on the three candidates for ventilator reallocation. At Overlook Medical Center, all nine raters agreed on the first two candidates for reallocation, and seven of nine agreed on the third candidate. The two other raters selected two different patients. No patients at Newton Medical Center, Chilton Medical Center, or Hackettstown Medical Center were candidates for reallocation as none were in the yellow priority group, although three raters identified candidates who were closest to meeting the reallocation criteria at Chilton Medical Center and Hackettstown Medical Center.

Of the six patients who were determined to be likely candidates for reallocation at MMC and Overlook Medical Center, only one survived to hospital discharge. The remaining patients had do-not-attempt-resuscitation orders placed after consultation between their physicians and surrogates, which occurred many months before the current analysis was conducted. Of those patients, two were palliatively extubated. The one surviving patient was transferred to a long-term acute care hospital with a tracheostomy, technology-dependent, and percutaneous gastrostomy tube. To date, that patient has been discharged and readmitted twice, once with a dislodged tracheostomy tube and once with a decubitus ulcer. Among the two patients selected by a minority of raters, one died before hospital discharge after a do-not-attempt-resuscitation order was placed in the chart, and one survived with poor outcome (tracheostomy, percutaneous gastrostomy tube, technology-dependent, and anoxic brain injury).

DISCUSSION

The emergence of the COVID-19 pandemic has caused many resource-rich countries to address critical and imminent issues of allocation of limited healthcare resources that they have previously not been required to. Such a circumstance was felt possible in New Jersey in April/May 2020. To assist with allocation/reallocation decisions, an EMR-based algorithm was developed. Although the instrument was ultimately not needed or used, we conducted exercises to test its validity.

It is interesting that one evaluator chose to ignore the algorithm at one of the hospitals. The reasons for this are unclear but may speak to a reluctance to cede such decision-making to impersonal algorithms and a preference for a more nuanced approach. However, off-protocol decision making would not happen in actual practice as triage groups were designed to include three members, and it is to be hoped that the other two members would ensure compliance with the intended methodology. Apart from this instance, there was a high level of concordance between different raters. This concordance was, however, not perfect and again highlights the importance of triage teams including multiple members to limit misinterpretation of the algorithm.

A critical factor in determining which patients were in the yellow priority group (and therefore potentially eligible for ventilator reallocation) was the predicted life expectancy of the patient. Specifically, all patients in the yellow priority group had a predicted life expectancy of less than 1 year. Conversely, the majority of patients with a life expectancy of less than 1 year were in the yellow priority group. For these reasons, it is important that the assessment of life expectancy is as accurate as it can be. The correlation between life expectancy and triage score is not unexpected. Patients with a life expectancy of less than 1 year score a minimum of 5 (out of 8) even if doing well. Conversely, patients whose life expectancy is between 1 and 5 years will have at most a triage score of 6, placing them at the lowest risk among patients eligible for reallocation, whereas a life expectancy of greater than 5 years will preclude reallocation as the maximum triage score will be 4 (Orange priority). Not only was this relationship expected, but it was also intended by the developers of this triage instrument. In order to do the most good for the largest number, resources need to be preferentially available to those with the highest chance of survival and the longest length of survival.[16] Even if one accepts these predicates, justice and fairness require that such assessments of life expectancy are open, accurate, and transparent. One method we had intended to ensure this was by sharing the results of the algorithm with the primary care team looking after the patient and giving them the opportunity to correct any misinterpretations of the patient's life expectancy. For this reason, the identification of limited life expectancy was intended to be overinclusive rather than underinclusive as the former could be corrected by such in-person discussion about patients identified for possible resource allocation, whereas the latter would require discussions about all patients, including those not felt to be eligible for reallocation.

Our algorithm relied on diagnostic codes to predict life expectancy. An alternative would be that this becomes a routine part of the admission assessment or the assessment prior to transfer to an ICU, or before intubation. This assessment could be made by the attending or accepting physician or made a shared assessment as part of multidisciplinary rounds. Whether this would perform better than the assessment based on diagnostic codes is unclear. An advantage may be that in inclusion of a “human” factor would be more acceptable to families and that all patients would have data for possible resource allocation collected in the same manner. However, disadvantages include the possibility that subconscious or conscious biases may influence the decision or that the healthcare team would feel responsible for failing to ensure the patient's ongoing access to the scarce resource.

We used some modification of the usual SOFA criteria in our algorithm. For example, every patient was allowed a 120-hour trial of mechanical ventilation before being considered for reallocation. Anecdotally, it seemed that patients with COVID-19 often deteriorate in the period immediately after the institution of mechanical ventilation, so a protected period of therapy was felt to be necessary to determine whether patients were likely to improve over time. Although the SOFA score may not be perfect for triaging patients with COVID-19, our study did not consider other approaches to triage. The decision to use the SOFA score if crisis standards of care were declared had already been made by the State of New Jersey. Our intention was not to reinvent the triage criteria, but to find a way to effectively and efficiently apply the criteria we expected to be mandated to use.

Our algorithm was designed to assist in reallocation of ventilators, as that was the most limited resource at the time. Presently, as the available number of ventilators has increased, other resources, for example, skilled nursing, medical, and other staff, may be more limiting.[20] However, the method we have described could be easily modified to assist with decision-making around allocation and reallocation of other resources.

It is important to understand that the EMR-based algorithm was only the first stage in the allocation/reallocation process. If the process was to be applied in practice, the next step would be a direct conversation among the triage team, the bedside care team, the family, and any healthcare proxies. This would provide an opportunity to ensure that the predicted life expectancy of the patient (a major driver of triage decisions) had been correctly classified and was consistent with the healthcare team's opinions, as well as ensuring that the physiological data used to calculate the SOFA score have been applied correctly. Such discussion would clearly be challenging, but it is worth noting that the concept of crisis standards of care[21] and the decision about resource allocation[6,20,22] are being increasingly discussed in the lay press as a result of the pandemic.

The algorithm has some advantages: it is rapid, efficient, and transparent and is also blinded to nonrelevant factors (such as the patient's ethnicity, sexual orientation, marital status, etc). One concern may be the relatively limited input from primary care providers. However, providers are subject to subconscious biases that may cause their advocacy for patients to be affected, in part, by clinically irrelevant factors. In addition, it imposes a significant emotional burden on providers to “successfully” advocate for their patients in regard to allocation/reallocation decisions.

Although it is to be hoped that such allocation/reallocation decisions will not need to be made, it is important to prepare for that contingency. It is also important to emphasize that such algorithms are neither a replacement for a careful and considered discussion about personal preferences in healthcare with all patients (or potential patients), nor honest and frank discussions about goals of care with family members of patients who have lost medical decision-making capacity.

In conclusion, we have developed a robust EMR-based algorithm to applied triage decisions in times of crisis, should they be required. Going forward, attitudes of healthcare providers, families, and patients should be considered and incorporated into the triage process. As predicted life expectancy was a principal driver of triage decision, and accuracy and reliability of the evaluation of different methods of predicting life expectancy needs to be examined.