Functionality of Scarce Healthcare Resource Triage Teams During the COVID-19 Pandemic: A Multi-Institutional Simulation Study.

Plans for allocating scarce healthcare resources during the COVID-19 pandemic commonly involve the activation of institutional triage teams. These teams would be responsible for selecting patients who are most likely to survive to be prioritized to receive scarce resources. However, there is little empirical support for this approach.
DESIGN: High-fidelity triage-team simulation study.
SETTING: Healthcare institutions in Washington state.
SUBJECTS: Triage teams, consisting of at least two senior clinicians and a bioethicist.
INTERVENTIONS: Participants reviewed a limited amount of deidentified information for a diverse sample of critically ill patients. Teams then assigned each patient to one of five prioritization categories defined by likelihood of survival to hospital discharge. The process was refined based on observation and participant feedback after which a second phase of simulations was conducted.
MEASUREMENTS AND MAIN RESULTS: Feasibility was assessed by the time required for teams to perform their task. Prognostic accuracy was assessed by comparing teams' prediction about likelihood of survival to hospital discharge with real-world discharge outcomes. Agreement between the teams on prognostic categorization was evaluated using kappa statistics. Eleven triage team simulations (eight in phase 1 and three in phase 2) were conducted from December 2020 to February 2021. Overall, teams reviewed a median of 23 patient cases in each session (interquartile range [IQR], 17-29) and spent a median of 102 seconds (IQR, 50-268) per case. The concordance between expected survival and real-world survival to discharge was 71% (IQR, 64-76%). The overall agreement between teams for placement of patients into prognostic categories was moderate (weighted kappa = 0.53).
CONCLUSIONS: These findings support the potential feasibility, accuracy, and effectiveness of institutional triage teams informed by a limited set of patient information items as part of a strategy for allocating scarce resources in healthcare emergencies. Additional work is needed to refine the process and adapt it to local contexts.

If lifesaving resources become scarce in a healthcare crisis, such as the COVID-19 pandemic, these should be allocated in a way that supports fair distribution across the population of people in need (1–3). Although the ethical implications of this tragic work are complex, deliberation among clinicians, ethicists, and the public has led to a consensus that the primary goal of resource distribution under such circumstances would be to maximize the overall benefit of a scarce resource by prioritizing patients who are most likely to survive (1, 4). A triage team-based strategy has been proposed as a way of operationalizing this goal and is a cornerstone of many regional, national, and international emergency response plans (5–9).

Under the triage team approach to healthcare resource allocation, a triage team would be provided with a limited set of patient information items that they would use to identify patients with the greatest likelihood of survival to hospital discharge, and these patients would receive priority for scarce resources (10, 11). Teams would typically include senior clinicians with critical care experience, an ethicist, and/or other content experts. Information that is not relevant to prognostication about short-term survival—such as gender, race, and physical or cognitive ability—would be intentionally excluded from triage decision-making to limit the impact of implicit bias (12).

The triage team approach relies on clinical judgment, making it potentially flexible and adaptable to unanticipated emergency settings. Alternative and simpler triage strategies, such as random allocation, first-come/first-served, and algorithms based on summary physiology scores (e.g., Sequential Organ Failure Assessment [SOFA] score), have been criticized on grounds of both fairness and effectiveness (13–17). Specifically, the SOFA score has shown poor accuracy in the context of COVID-19 and confers limited ability to clinically differentiate between prognostic categories (17). Further, this score may perpetuate biases against underserved racial groups (18). For these reasons, the SOFA score has been removed from many triage algorithms, including for Washington State. However, there are also potential limitations to the triage team approach, which is more complex than default strategies such as random allocation or first-come, first-served, and there is little empirical evidence to support effectiveness (14).

Experience during the COVID-19 pandemic has reinforced the need for further development and testing of existing theoretical approaches to healthcare emergency response to ensure that these can be operationalized in real-world contexts (11, 19–21). The pandemic also offers the unique opportunity to simulate the triage process with clinicians who are embedded in an active healthcare emergency setting and have ongoing experience with the relevant patient population. We conducted a high-fidelity simulation study to test whether a triage team approach to scarce resource allocation could be feasible and could produce consistent and accurate results during the COVID-19 pandemic.

MATERIALS AND METHODS

Recruitment and Data collection

Triage teams were recruited from healthcare institutions across Washington State and composed of at least two clinicians (physicians and nurses) with critical care, emergency medicine, or inpatient clinical experience and one team member with training and/or experience in clinical bioethics. E-mails with information about the study were sent to members of the Washington State Disaster Medical Advisory Committee, a group that represents diverse clinical backgrounds and affiliations with institutions across the state. Members were invited to participate and also to identify colleagues who were involved in institutional planning for crisis during the COVID-19 pandemic and/or who had volunteered to act as hospital triage team members, and these potential participants were similarly invited to participate. Teams were often, but not always, composed of clinicians from a common institution. Each participant took part in only one triage simulation.

A patient information form consisting of the minimum necessary data needed to support triage team decision-making was developed and refined in an asynchronous Delphi study among healthcare disaster preparedness experts, which is described in detail elsewhere (22). Briefly, these patient information items included age, severe or end-stage comorbidities, reason for and timing of hospitalization, indications for ICU admission, severity of acute respiratory distress syndrome, and trajectory of illness (Supplemental Fig. 1, http://links.lww.com/CCX/A902). The triage team was tasked with assigning patients to one of four prognostic categories defined by likelihood of survival to discharge (red [> 75% predicted likelihood of survival and highest priority to receive scarce resource], orange (50–75% predicted survival), yellow (25–49% predicted survival), or blue [< 25% predicted survival and lowest priority to receive scarce resources]) or to a “striped” category defined by a list of discrete conditions that had been previously determined through clinical and community stakeholder consensus to carry uniquely poor short- and long-term prognoses (8, 23) (i.e., severe acute trauma, severe burns with low survival rate (24), or persistent vegetative state/coma) (Supplemental Fig. 2, http://links.lww.com/CCX/A902).

Patient cases were selected from the set of all adult patients requiring ICU admission between March and November 2020 at a large academic institution (Virginia Mason Medical Center) in Seattle, Washington. From an initial set of 13,000 patients admitted to the ICU from April 1, 2020, to November 30, 2020, 75 patients were purposively sampled to include a range of survival outcomes (35% survival), COVID-19 statuses, and clinical diagnoses. One study team member then performed a manual chart review to populate patient information forms (L.B.W.). For each patient case, the team member selected a random day in the patient’s ICU course and limited patient data extraction to the information that would have been available on this day. We also collected information on sex, ethnicity, race, SOFA score, and the outcome of hospitalization (death vs hospital discharge) that was not provided to triage teams. At this time, hospital admissions related to COVID-19 were common, but case load at our local institution did not exceed capacity, and patient outcomes were not expected to have been impacted by resource limitations.

Triage Simulation

Simulations were conducted by video conferencing and observed by at least three study team members. At the beginning of each simulation, triage team participants were given an orientation by one of the study team members (L.B.W. or V.L.S.) in which goals and guiding ethical principles of healthcare triage (Supplemental Table 1, http://links.lww.com/CCX/A902) and the origin and structure of the patient information and reporting forms were reviewed (22). Teams were instructed to focus exclusively on categorizing patients based on their likelihood of survival to hospital discharge and to avoid subjective judgments of patients’ quality of life or other external considerations. Although triage teams were given autonomy to choose how to approach decisions, the study team suggested that each team appoint a team leader to guide discussion and explained the role of the ethicist in navigating ethical conflicts that might arise during deliberation.

Teams were presented with a series of patient cases and encouraged to sequentially review as many cases as they felt was appropriate within a 90-minute session. For each patient case, teams were instructed to review the patient information form, answer the binary question: “Is the patient likely to survive to hospital discharge if they receive all needed resources?,” and assign each patient to a prognostic category. The duration of discussion and decision-making for each patient was recorded. Any time that teams spent discussing general process or other issues not related to a specific patient was not included in the analysis of timing.

Given the lack of established precedent for triage team process design, we allowed for the possibility that the study team might identify a need to refine features of the process and hold a second phase of simulations. Each simulation was followed by a 30–45 minute participant debriefing session led by an experienced clinician facilitator (L.B.W. or V.L.S.). Participants were probed about their perceptions and experience of the triage process including the logistics of triaging, the value of information items, approach to reporting, team member roles and dynamics, and any other thoughts or concerns, which could be used to refine the process. After each simulation and debriefing session, the study team met to review observation notes and to discuss opportunities to improve orientation materials, tools, and the format of the patient information form.

Statistical Analysis

The characteristics of triage team members and patient cases were described using means (sd), medians (interquartile range [IQR]), and proportions as appropriate. The median (IQR) seconds that teams took to make decisions for each patient case were determined for each simulation phase and for both simulation phases combined.

Prognostic accuracy was determined by comparing teams’ responses to the binary question: “Is the patient likely to survive to discharge?” with each patient’s real-world survival to hospital discharge. Specifically, if the team responded that a patient was likely to survive and the patient survived to hospital discharge or if the team responded that the patient was unlikely to survive and the patient died before discharge, the response was considered to be accurate. Test statistics (sensitivity, specificity, positive predictive value, and negative predictive value) were derived based on how an answer of “no” to the binary question about expected survival functioned to predict death before discharge. We also determined the proportion of patients who survived to discharge among those assigned to each of the five prognostic categories.

The proportion of patient cases placed in each prognostic category was described for each team. Consistency between teams in the assignment of patient cases to prognostic categories was determined using the weighted kappa statistic (25). Feasibility of the triage team approach was defined by the median duration of time required to assign patient cases to prognostic categories.

The Benroya Research Institute at Virginia Mason Institutional Review Board (IRB) waived the need for IRB review because the work did not fall under the definition of human subjects research. The Research Electronic Data Capture online database management platform was used to collect and organize data (University of Washington Institute of Translational Health Sciences). Statistical analyses were performed using Stata 16.0 statistical software (Stata Corp, College Station, TX).

RESULTS

Twelve triage simulations were conducted from December 2020 to February 2021. Of these, the results of one simulation were excluded from analysis due to an error in communicating instructions to this triage team (i.e., a list of example cases were misunderstood to be prescriptive definitions of prognostic categories). The 11 triage simulations included in the analysis involved 37 total participants. Teams were composed of a median of 3 (IQR, 3–4) team members. Participants had an average age of 49.7 years (sd, 11.2 yr), the majority were White (83.8%), and 45.9% were women (Table ). Most participants were practicing in community clinical settings (59.5%) and had an average of 20.6 years (sd, 10.6 yr) of healthcare work experience.

A median of 23 patient cases (IQR, 17–29) and maximum of 38 cases were reviewed in each simulation session. Of the 38 unique patient cases reviewed by at least one team, patients’ average age was 60.2 years (sd, 16.0 yr), 13 were female (34.2%), and 17 were Hispanic and/or of a minoritized racial group (44.7%) (Table ). Most of patients (82%) were receiving mechanical ventilation at the time of data collection. Overall, teams spent a median of 102 seconds (IQR, 50–268 s) discussing each patient case (Supplemental Table 2, http://links.lww.com/CCX/A902).

After review of observation notes from the first eight simulations and deliberation among study team members, there was consensus that several adaptations to the instructions, presentation of data, and data reporting form were needed to refine the process before a second phase of simulations. First, triage team members perceived that prognostication was most reliable for patients who were either very likely to survive or very unlikely to survive, and there was a greater degree of uncertainty in assigning prognostic categories to those with intermediate prognosis (i.e., the yellow and orange categories). In order to better represent this variable degree of uncertainty, prognostic category definitions were changed from quartiles to greater than or equal to 90% (red), 50–89% (orange), 11–49% (yellow), and less than or equal to 10% (blue) (Supplemental Fig. 2, http://links.lww.com/CCX/A902). Second, several teams were unsure of how to use the striped category, so the definition was made more specific (Supplemental Fig. 2B, http://links.lww.com/CCX/A902). Third, teams wished to have additional contextual information about how the patient information form was developed and how triage decisions would be implemented. The introductory session was adapted to include this information as well as description of how triage would be operationalized within hospital settings. Fourth, teams were observed to often reason through patient cases by comparing a case under consideration with earlier cases. To support this cognitive approach (i.e., case-based reasoning or casuistry [26]), triage teams were provided with a set of exemplar cases for each prognostic category as a tool to ground deliberation in a common understanding (Supplemental Fig. 3, http://links.lww.com/CCX/A902).

Prognostic Accuracy

Teams’ response about whether a patient was likely to survive to discharge was concordant with real-world survival to discharge for a median of 68% (IQR, 60–73%) of cases in phase 1 simulations, 76% (IQR, 72–80%) in phase 2 simulations, and 71% (IQR, 64–76%) for all simulations combined. Teams’ determination that a patient was unlikely to survive to discharge predicated death before discharge with a sensitivity of 55.5%, specificity of 80.3%, positive predictive value of 67.4%, and negative predictive value of 71.1%. This response was better than chance for predicting survival to discharge for 10 of 11 teams (Fig. ).

Receiver-operating curve for survival to discharge predictions. Each triage team is represented by a dot. The dotted line indicates what would be expected if predictions were no more accurate than chance, and points above this line indicate prediction that is better than chance.

In phase 1, a median of 78% (IQR, 64–93) patients placed in the red category survived, 42% (IQR, 33-64) in the orange category, 29% (IQR, 19-43) in the yellow category, 33% (IQR, 32–43) in the blue category, and 50% (IQR, 42–58) in the striped category (Fig. ). In phase 2, a median of 75% (IQR, 67–88) patients placed in the red category survived, 67% (IQR, 61–73) in the orange category, 0% (IQR, 0–25) in the yellow category, and 20% (IQR, 10–60) in the blue category. The striped category was not used by any team in phase 2.

Proportion of patients who survived to discharge among those assigned by a team to each prognostic category. *Only two teams in phase 1 and no teams in phase 2 assigned patients to the striped category. n = 8 teams in phase 1 and n = 3 teams in phase 2. IQR = interquartile range, N/A = not applicable.

Agreement

The proportion of patients assigned to each prognostic category varied between teams (Fig. ). The weighted kappa statistic for agreement between teams in prognostic category assignment for each patient case was 0.51 for phase 1 simulations (moderate agreement), 0.67 for phase 2 simulations (substantial agreement), and 0.53 for the combined phases 1 and 2 (moderate agreement) (25).

Proportions of patient cases assigned to each prognostic category.

DISCUSSION

This multi-institutional triage team simulation study, conducted in real time among clinicians imbedded in clinical settings during the COVID-19 pandemic, offers valuable insights into the function of triage teams as a component of scarce healthcare resource triage. Participating triage teams were able to efficiently employ a limited set of deidentified patient information items to group patient cases within prognostic categories with considerable accuracy and consistency. These findings support the potential utility of triage teams as an approach to prioritizing patients for allocation of scarce ICU resources during the pandemic.

Although the triage team model is more complex than alternative strategies for scarce resource allocation, such as random allocation, our findings suggest that this approach may achieve the intended goal of improving the overall benefit of resource allocation by effectively and consistently prioritizing patients with a greater likelihood of survival to receive scarce resources. Advance training and practice, such as conducting routine or just-in-time local triage simulations, may further support triage team members in effectively performing this unique task. Our results also compare favorably with modeling of triage strategies based on alternative criteria such as admission SOFA score (27). Consistent with existing reports (17), SOFA scores for our cohort were clustered in a relatively limited range, which would likely limit the utility of this score in making clinically meaningful prognostic distinctions between the patients.

Our results suggest that an institutional scarce resource triage team can function with a limited set of information items that exclude many factors known to be shaped by implicit biases, such as race, sex, socioeconomic status, and functional status. Nonetheless, several characteristics known to be associated with negative implicit biases, such as advanced age and comorbidities, were provided to triage teams because this information has been considered to be central to accurate prognostication (22). The challenges of disentangling the impact of implicit biases from prognostic relevance of patient information items underscore the importance of implicit bias training for triage team members and active self-monitoring during triage team decision-making (28). These concerns also reinforce the utility of including a team member specifically responsible for facilitating an ethical approach to decision-making as well as systems of oversight at both a local and regional levels (23). A system of triage prioritizing those most likely to survive will systematically disadvantage underserved populations because these groups tend to have a greater burden of comorbidities (29, 30). Whether and how to incorporate features that aim to actively promote equity in resource allocation into the triage process is an important topic of ongoing discussion (29, 31).

The triage team constitutes only one component of a broader process of scarce resource allocation and processes upstream and downstream of triage team decisions that will also impact feasibility, effectiveness, and fairness of the system overall (21). Our prior work supports the efficiency of collecting patient information items from an institutional electronic health record (22), but local planning within each institution for how to collect this information will be important to support operationalizability. Additional work is also needed to develop and refine downstream processes of communicating triage decisions to bedside teams and families, ensuring alternative and palliative care for patients who are not selected to receive scarce resources (32), and monitoring the process for fairness. Strategies for supporting triage team members and bedside clinicians in the face of extreme tragedy and moral distress would also likely be critical to the feasibility of this approach in practice (14, 33–35).

This study has several strengths, but also limitations. Embedding this work in real-time clinical settings during the COVID-19 pandemic offered the opportunity to conduct a simulation under the most realistic conditions possible. Nonetheless, participants may have approached the simulation differently than they would in a real-world situation. The number of simulations we were able to conduct was limited by constraints on participants’ availability during the pandemic, which limited our ability to study a broader array of variations in team structure, approach to education, patient information form items, and reporting framework. A relatively small sample size also limited our ability determine whether triage team performance differed between patient gender and racial groups, but this type of monitoring could be incorporated into a real-world triage approach to identify sources of bias. The study was conducted in Washington state, which may limit generalizability to other countries or regions of the United States. Heterogeneity in operations between institutions, differing hospital culture, and unpredictable pressures of the pandemic and other healthcare emergencies mean that this approach may need to be tailored to local circumstances. Finally, functionality of the entire triage process, including recruiting triage team members, completing patient information forms, enacting triage decisions, and process oversight are also critical components of the process for which additional study is needed.

CONCLUSIONS

A triage team model may offer a feasible and effective approach to healthcare resource allocation in settings of critical care resource limitation during the COVID-19 pandemic. Additional work is needed to refine processes surrounding triage team decisions, and any planned triage strategy should be accompanied by a system of data collection and monitoring to identify opportunities to iteratively improve the process in real time.

ACKNOWLEDGMENTS

We thank Mr. Josh Edrich for his exceptional administrative and technical support and Dr. Craige Blackmore for his statistical guidance.

TABLE 1.

Triage Team Participant Characteristics

Participant Characteristic	Participants (n = 37)
Age, yr, mean (sd)	49.7 (11.2)
Race (%)
Asian	5 (13.5)
White	31 (83.8)
More than one race	1 (2.7)
Black or African American, American Indian/Alaska Native, Native Hawaiian or Other Pacific Islander, Prefer to self-identify, or Prefer not to say	0 (0)
Hispanic or Latino (%)	1 (2.7)
Gender identity (%)
Woman	17 (45.9)
Man	20 (54.1)
Other or prefer not to say	0 (0)
Years in clinical practice, yr (sd)	20.6 (10.6)
Type of primary institution (%)
Academic	10 (27.0)
Private	7 (18.9)
Community	22 (5.5)
Other	2 (5.4)
Primary practice setting (%)
Urban	32 (86.5)
Rural	6 (16.2)
Other	3 (8.1)
Primary work site (%)[a]
Clinic or outpatient	5 (13.5)
Acute-care hospital setting	19 (51.4)
Intensive care hospital setting	9 (24.3)
Emergency department	6 (16.2)
Nonclinical setting	4 (10.8)
Research	0 (0)
Other setting	2 (5.4)
Clinical ethics experience (%)	16 (43.2)

Participants could select multiple answers.

TABLE 2.

Patient Case Characteristics

Patient Characteristic[b]	Patient Cases (n = 38)[a]
Age, yr, mean (sd)	60.2 (16.0)
Severe or end-stage comorbidities (%)
Chronic kidney disease	8 (21.1)
Chronic lung disease	5 (13.2)
Heart failure	11 (28.9)
Coronary artery disease	11 (28.9)
Malignancy	2 (5.3)
Other severe or end-stage condition	2 (5.3)
Duration of hospitalization, d, median (IQR)	1.5 (0.0–6.3)
ARDS severity (%)
No ARDS	20 (52.6)
Mild (Pao2/Fio2 = 200–300)	2 (5.3)
Moderate (Pao2/Fio2 = 100–200)	8 (21.1)
Severe (Pao2/Fio2 < 100)	8 (21.1)
COVID-19 positive (%)	20 (52.6)
Information not available to the triage team
Female (%)	13 (34.2)
Race and ethnicity (%)
American Indian or Alaska Native	2 (5.3)
Asian	4 (10.5)
Black or African American	3 (7.9)
Native Hawaiian or Pacific Islander	1 (2.6)
White	17 (44.7)
Hispanic or Latinx	7 (18.4)
Declined to answer or unknown	4 (10.5)
Sequential Organ Failure Assessment score, median (IQR)	5 (4–7)

ARDS = acute respiratory distress syndrome, IQR = interquartile range.

Only includes those patient cases that were viewed by at least one triage team.

At the randomly assigned day of triage.