Outcomes considered most important by emergency physicians when determining disposition of patients with pulmonary embolism.

PURPOSE: Clinical decision rules for the disposition of patients with pulmonary embolism (PE) are typically validated against an outcome of 30-day mortality or disease recurrence. There is little justification for this time frame, nor is it clear whether this outcome reflects emergency department (ED) decision making. AIMS: To determine which outcomes emergency physicians (EP) consider most relevant to disposition decisions.
METHODS: Survey of attending EPs in geographically diverse US states using acute PE as the diagnostic framework. Responses required single-answer multiple choice, a numerical percentage, rank-ordered responses, or a five-point Likert scale. We distributed the survey via e-mail to 608 EPs.
RESULTS: We received responses from 292 (48%) EPs: 88% board certified, 91% trained in emergency medicine, and 70% work in academics. Respondents reported discharging 1% of patients with PE from the ED, but 21% reported being asked to do so by an admitting service. EPs were more interested in knowing 5-day (in hospital) outcomes [192/265, 72% (95% exact CI = 66%-78%)] than 30-day outcomes [39/261, 15% (95% exact CI = 11%-20%)] or 90-day outcomes [29/263, 11% (95% exact CI = 8%-15%)]. On a Likert scale, 212/241 (88%, 95% exact CI = 83%-92%) agreed or strongly agreed that they considered 5-day (in hospital) clinical deterioration when making a decision to admit or discharge a patient from the ED compared to 184/242 (76%, 95% exact CI = 70%-81%) and 73/242 (30%, 95% exact CI = 24%-36%) for 30 and 90 days, respectively. A wide variety of clinical outcomes beyond death or recurrent PE were considered indicative of clinical deterioration.
CONCLUSIONS: Five-day (in hospital) outcomes that incorporate a variety of clinical deterioration events are of interest to EPs when determining the disposition of ED patients with PE. Researchers should consider this when developing and validating clinical decision rules.

Introduction

With every patient encounter, emergency physicians (EPs) must decide on an appropriate disposition. In some cases, disposition decisions are obvious—clinically unstable patients must be admitted and patients with minor problems may be safely discharged from the Emergency Department (ED). However, for a large number of conditions, determining which patients are safe for outpatient treatment is more complex.

Prospectively validated decision rules are available to help clinicians determine which patients with pneumonia, syncope, transient neurological attacks, pulmonary embolism (PE), and other conditions should be admitted to the hospital and which patients are safe for discharge [1-7]. It is common for these rules to be validated against outcomes such as 30-day mortality or disease recurrence. However, there is little justification for this in the literature. Since few patients are hospitalized for 30 days, it is not clear how this time frame informs the decision to admit or discharge a patient. Narrowly defined outcomes such as death and disease recurrence may not reflect the complexity of the disposition decision either. Moreover, statistical models that predict all-cause mortality may unduly reflect factors with a high fatality rate (e.g., cancer), whether or not they are associated with the diagnosis in question. Given these issues, it is not surprising that physicians tend not to be familiar with clinical decision rules and use them infrequently in practice [8].

We sought to determine which of three time frames EPs considered most relevant to the disposition of patients with PE. We also sought to determine whether different outcomes (including cardiopulmonary arrest, hypotension, hypoxia, need for respiratory support, need for thrombolysis, development of a cardiac dysrhythmia, and bleeding) were considered more relevant to disposition decisions than others. A better understanding of the clinical decision making of EPs may improve our ability to develop decision rules that are useful to practicing clinicians.

Methods

We developed a survey to determine how different outcomes influence an EP’s decision to admit or discharge patients from the ED. The survey was developed by two of the authors (Weston Sacco and Christopher Kabrhel). The survey was designed to be anonymous, with respondents being asked to create a coded unique identifier. Substantive questions required: single-answer multiple choice; yes/no; input of a numerical percentage (0-100%); rank ordering of a series of six responses (from most important to least important); or completion of a five-point Likert scale. Distribution of the survey was approved by the Institutional Review Board of Partners Healthcare Inc.

Demographic information gathered included respondent age, training, and practice setting. We chose to use a diagnosis of PE as the diagnostic framework, and respondents were asked to consider a patient with a diagnosis of acute PE when determining whether an event would represent clinical deterioration. The main goals of the survey were to determine: (1) which of three time frames (5, 30, and 90 days) EPs consider most relevant to determining ED disposition; (2) which outcomes represent clinical deterioration and (3) the degree to which these outcomes are considered relevant to disposition decisions. For questions regarding the most appropriate time frame, respondents were provided the general instruction: “We are trying to determine the most appropriate time frame for considering risk.” For questions assessing clinical deterioration, respondents were provided the general instruction: “We are trying to determine what events or interventions define ‘clinical deterioration.’” In addition, for questions assessing specific clinical events, and whether they represent clinical deterioration, respondents were provided with the following clinical scenario: “You have diagnosed an Emergency Department patient with Pulmonary Embolism (PE). The patient is currently stable, and you are deciding whether to admit the patient or to discharge him/her from the Emergency Department” and the general instruction “We would like you to consider ‘clinical deterioration’ events that would, if predictable, impact the ED disposition of a patient with PE.” Respondents were then asked to complete the sentence: “I would consider a patient to have had a clinical deterioration if….”

We inquired about outcomes occurring within three specific time frames: 5 days, 30 days, and 90 days. We chose 5 days to reflect an average length hospitalization for a patient with PE [9-11]. We chose 30 and 90 days because these time frames have been used to validate the pulmonary embolism severity index in prior studies [7, 12–14].

During development, the survey was piloted serially on six board-certified EPs using an iterative process to assess question clarity and completeness. Physicians in the pilot group were asked to identify any question that was unclear or vague, and to suggest improvements. Suggested changes were incorporated until the physician felt each question was clear and addressed the goal of the survey. Pilot physicians took the survey, with revisions incorporated, two to three times each. The final survey was then uploaded to a commercially available online survey site (www.surveymonkey.com).

The survey was distributed to 608 attending EPs at 16 institutions, including a mix of academic and community practices, located in 13 geographically diverse US states. Potential respondents were contacted three times in order encourage completion of the survey. Informed consent was implied with completion of the survey.

Demographics are presented as simple means and proportions. Comparative analysis was performed by comparing binomial proportions and exact 95% confidence intervals (CI), with non-overlapping CIs considered significantly different. In order to assess generalizability across academic and community practice settings, we performed a sensitivity analysis, limiting our analysis to respondents (n = 86) who described their practice setting as a “community medical center” or “combined academic/community.” All statistical analyses were performed using SAS version 9.1 (SAS Institute, Cary, NC).

Results

We received survey responses from 292 EPs, or 48% of those contacted. Demographic data describing respondents are provided in Table 1. The vast majority of respondents did residency training in Emergency Medicine and were board certified. The majority of respondents worked in academic medical centers, though a sizable percentage worked in combined academic/ community settings.

Table 1

Demographic characteristics of respondents

Descriptor	Number responding	(%)	Characteristic	(%)
Age	290	99.3%
<30 years			11	3.8%
30–39 years			141	48.6%
40–49 years			80	27.6%
50–59 years			39	13.4%
60–69 years			18	6.2%
≥70 years			1	0.3%
Board status	288	98.6%
Board certified			252	87.5%
Board eligible			36	12.5%
Residency training	283	96.9%
Emergency medicine			257	90.8%
General surgery or surgical specialty			4	1.4%
Internal medicine or medical specialty			18	6.4%
Pediatrics or pediatric specialty			4	1.4%
Other			11	3.9%
Clinical workload	289	99.0%
Full time			115	39.8%
75%–99% of full time			45	15.6%
50%–74% of full time			77	26.6%
25%–49% of full time			42	14.5%
<25% of full time			10	3.5%
Practice setting	287	98.3%
Academic medical center			201	70.0%
Community medical center			7	2.4%
Combination academic and community			79	27.5%
Emergency department observation unit	289	99.0%
Yes			194	67.1%
No			95	32.9%

Respondents reported substantial experience diagnosing PE. Only 3/281 (1%) respondents had not diagnosed a PE within the last year, whereas 71 (25%) had diagnosed 1–5 PEs, 100 (36%) had diagnosed 6–10 PEs, 45 (16%) had diagnosed 11–15 PEs, 36 (13%) had diagnosed 15–20 PEs, and 26 (9%) had diagnosed more than 20 PEs in the past year. Respondents reported discharging a mean of 1% of patients with PE from the ED after diagnosis. Of the 83/281 (30%) respondents who reported that they do sometimes discharge patients with PE from the ED, only 9 (11%) said they use a scoring system to decide which patients with PE are safe for discharge. When asked whether an admitting service had ever suggested discharging a patient with PE from the ED, 59/282 (21%) said “Yes.”

When asked to rank order 5-, 30- and 90-day outcomes, 192/265 (72%, 95% exact CI = 66%–78%) ranked 5-day (in hospital) outcomes “most important,” whereas 39/261 (15%, 95% exact CI = 11%–20%) said 30 days, and 29/263 (11%, 95% exact CI = 8%–15%) said 90 days (Fig. 1). Similarly, on a five-point Likert scale, 212/241 (88%, 95% exact CI = 83%–92%) agreed/strongly agreed that they considered 5-day (in hospital) clinical deterioration when making a decision to admit or discharge a patient from the ED compared to 184/242 (76%, 95% exact CI = 70%–81%) who agreed/strongly agreed that they considered 30-day clinical deterioration, and 73/242 (30%, 95% exact CI = 24%–36%) who agreed/strongly agreed that they considered 90-day clinical deterioration. When respondents were asked to consider a patient who had no clinical deterioration throughout their hospitalization, but subsequently deteriorated after discharge, 148/241 (61%, 95% exact CI = 55%–68%) agreed/strongly agreed that the hospitalization was justified. Only 29/244 (12%, 95% exact CI = 8%–17%) of respondents agreed/strongly agreed that clinical deterioration was “only important if it required treatment,” while 192/244 (79%, 95% exact CI = 73%–84%) disagreed/strongly disagreed with that statement.

Fig. 1

Outcome time frame considered “most important” by respondents. *While 273/292 respondents ranked at least one time frame (i.e., 5, 30, or 90 days), 13 did not list any of the time frames at “most important,” leaving 260 responses available for this analysis

We asked respondents which clinical events are indicative of clinical deterioration that would influence the disposition decision for a patient with PE. All respondents [242/242, 100% (95% exact CI = 98%–100%)] said that cardiopulmonary arrest within 5 days represented clinical deterioration, though fewer said so if cardiopulmonary arrest occurred within 30 days [210/241, 87% (95% exact CI = 82%–91%)] or 90 days [127/238, 53% (95% exact CI = 47%–60%)]. Hypotension was considered clinical deterioration by 237/237 (100%, 95% exact CI = 98%–100%) if it required vasopressor therapy, 232/237 (98%, 95% exact CI = 95%–99%) if it required volume resuscitation, and by 168/232 (72%, 95% exact CI = 66%–77%) if no treatment was required. The greatest number of respondents said that if a patient’s room air oxygen saturation became less than 90%, they would consider the patient to have had a clinical deterioration (Fig. 2a). Most respondents said that if a subject required any supplemental oxygen to maintain the SaO2, either at rest or with exercise, it represented clinical deterioration (Fig. 2b). Treatment with thrombolysis was considered clinical deterioration by 214/242 (88%, 95% exact CI = 84%–92%). Bleeding (intracranial, gastrointestinal, retroperitoneal, other major bleeding, or minor bleeding) was considered clinical deterioration by >80% of respondents, regardless of type or whether treatment was required, with the exception of minor bleeding not requiring treatment [46/203, 23% (95% exact CI = 17%–28%)]. Similarly, all cardiac dysrhythmias (bradycardia requiring treatment, reentrant supraventricular tachycardia requiring treatment, atrial fibrillation/flutter, ventricular tachycardia) were considered clinical deterioration by more than 90% of respondents, with the exception of reentrant supraventricular tachycardia not requiring treatment [158/207, 77% (95% exact CI = 71%–82%)] and bradycardia not requiring treatment [80/204, 38% (95% exact CI = 32%–45%)].

Fig. 2

a Oxygen saturation (SaO2) considered indicative of clinical deterioration. b Supplemental oxygen considered indicative of clinical deterioration. Abbreviations: L = liters, NRB = non-rebreather mask, PPV = positive pressure ventilation. *Results are standardized according to percentage of respondents answering question. In a, 266 and 240 respondents provided oxygen saturations indicative of clinical deterioration at rest and with exercise, respectively. In b, between 198 and 241 respondents stated whether a given level of supplemental oxygen constituted clinical deterioration. **Supplemental oxygen required to maintain the oxygen saturation (SaO2) named by the respondent as indicative of clinical deterioration (see Fig. 1). ***Supplemental oxygen required to make the patient “subjectively more comfortable” even though not required to maintain oxygen saturation (SaO2)

Finally, we asked respondents about factors that, while not indicative of clinical deterioration, might influence disposition. On a five-point Likert scale, respondents said they were more/much more likely to admit patients >70 years old [200/240, 83% (95% exact CI = 78%–88%)] and 50–69 years old [179/241, 74% (95% exact CI = 68%–80%)], but not patients 30–49 years old [60/240, 25% (95% exact CI = 19%–31%)] or <30 years old [42/241, 17% (95% exact CI = 13%–23%)]. Other factors that made admission more/much more likely were: lack of family/friend support at home [213/241, 88% (95% exact CI = 84%–92%)]; inability to return to hospital if further problems arise [232/241, 96% (95% exact CI = 93%–98%)]; pregnancy [202/241, 84% (95% exact CI = 79%–88%)]; recent surgery [182/239, 76% (95% exact CI = 70%–81%)]; non-English speaking [138/239, 58% (95% exact CI = 51%–64%)]. Lack of insurance was less influential, with most respondents [131/241, 54% (95% exact CI = 48%–61%)] being neutral on the question.

Sensitivity analysis

When we limited our analysis to the 86 respondents who described their practice setting as either a community medical center or a combined academic/community setting, results were similar. When asked to rank order 5-, 30- and 90-day outcomes, 65/83 (78%, 95% exact CI = 68%–87%) ranked 5 day (in hospital) outcomes “most important,” while fewer [4/73, 5% (95% exact CI = 2%–13%)] said 30 days, and 2/69 (3%, 95% exact CI = 0%–10%) said 90 days. The results of all other analyses were similar to responses overall (data not shown).

Discussion

In our survey, when determining the most appropriate disposition of patients with PE, EPs were more interested in knowing 5-day (in hospital) clinical outcomes than 30- or 90-day outcomes. This sentiment was apparent when the question was asked directly—with 5-day outcomes more than four times more likely to be considered “most important” than 30-day outcomes. It was also apparent when the question was asked indirectly—with only 61% answering that hospitalization was justified if a patient remained stable while hospitalized but deteriorated after discharge. The importance of 5-day (in hospital) outcomes was also apparent in the responses to questions about specific clinical deterioration events. For example, the percentage of EPs who responded that the risk of cardiopulmonary arrest would influence the disposition decision declined from 100% when considering 5-day risk to 87% when considering 30-day risk and 53% when considering 90-day risk. Our results suggest that while 30- and 90-day outcomes have value, EPs find time frames that reflect the length of a typical hospitalization more relevant to their decision making.

Although the importance of in-hospital outcomes is intuitive and apparent from our survey, the use of such short-term clinical outcomes in the medical literature is rare [3, 4, 15]. Studies demonstrating that clinical factors and biomarkers are associated short-term (10-15 day) adverse clinical events after PE have been published [16, 17], though to the authors’ knowledge, there are no outcome studies in the PE literature that use a time frame reflective of a typical hospitalization. Our data suggest that EPs would consider studies of outcomes occurring during an average length hospitalization highly relevant to their disposition decisions.

We also found that EPs consider a broad range of clinical events to be indicative of clinical deterioration and that these events inform the disposition decision. It is our feeling that studies that limit their outcomes to death or disease recurrence oversimplify the disposition decision made by EPs every day. Our results suggest that in addition to outcomes that clearly represent clinical instability (e.g., cardiopulmonary arrest), when determining the best disposition of a patient with PE, EPs consider the need for respiratory support, the risk of developing a dysrhythmia, hypoxia, hypotension, and, bleeding, even if those events do not require treatment. In contrast, 12% of respondents did not consider treatment with thrombolysis indicative of clinical deterioration. We did not ask respondents to explain the rationale for their responses, so the reason for this remains unclear. One possibility is that treatment with thrombolysis was felt to reflect a clinical decision rather than a measure of patient status. This finding warrants further exploration.

We felt that questions related to clinical deterioration would be difficult to answer unless physicians were provided with a diagnostic framework within which to consider their decision. We chose to use a patient with PE as the diagnostic framework. PE is a common diagnosis and is an entity with which most EPs are familiar [18, 19]. PE is usually treated in the hospital, though outpatient treatment has gained traction in Europe and Canada, and patients with deep vein thromboses are commonly treated as outpatients in the US [7, 13, 20–23]. The fact that our respondents reported discharging a mean of 1% of patients with PE from the ED supports this and shows that while ED discharge after PE remains rare, some patients may be considered appropriate for discharge.

Understanding the factors that impact emergency department decision making, especially with regards to patient disposition, is relevant for several reasons. Systems designed to aid clinical decision making, improve patient care, and increase cost-effectiveness are only useful if they are adopted by practitioners. However, studies including ours have shown that clinical decision instruments are infrequently applied to practice [8]. While clinical rules are undoubtedly difficult to use for a variety of reasons, we believe that a clinical rule that poorly or partially reflects the concerns of the physician making the decision is unlikely to be adopted. The need for new outcomes specific to the practice of emergency medicine was highlighted at the Future of Emergency Medicine Research Conference more than a decade ago [24, 25]. Despite this, we believe that our survey is the first to document the importance of a broad range of clinical outcomes, occurring in a clinically relevant time frame, to EP decision making.

Limitations

There are several limitations of this study methodology that deserve mention. First, the recipients of this survey were chosen because they work in institutions that have a history of working relationships with study investigators. There is disproportionate representation towards academic centers in the eastern portion of the United States. This may limit the generalizability of our results as survey responses may not reflect the opinions of community EPs. However, when we limited our analysis to respondents practicing in community or combined academic/community settings, results were similar. Our response rate was moderate at 48%, but we acknowledge that our results may be biased towards those physicians inclined to complete surveys or otherwise participate in research. We acknowledge that the relevance of our data is particular to the question of disposition of patients with PE and does not necessarily inform disposition decisions for patients with other diagnoses. Also, for clinical decision rules that address different concerns than disposition, such as the likelihood of diagnosis, different outcomes may be relevant. We only compared 5-, 30-, and 90-day outcomes, so it is possible that there is a threshold time frame that would be more important to clinicians than 5 days. It is also possible emergency physicians simply chose the time frame that was temporally closest to their evaluation. However, we chose our time periods to reflect a typical hospitalization for PE (5 days) and time frames (30 and 90 days) used previously in the medical literature to validate clinical decision rules. Each time respondents were asked about a 5-day outcome, it was made clear that this reflected an in-hospital outcome. Thus, we feel that our results reflect physicians’ impressions of outcomes occurring during a typical hospitalization versus those occurring after discharge. While the vast majority of responses to our questions yielded consistent answers, we did find 34 respondents who gave inconsistent responses on questions about the time frame they would most like to know (e.g., choosing 5 days as the “most important” time frame, but strongly disagreeing with the idea of considering a patient’s risk of clinical deterioration while in the hospital). It is likely the labeling of the ordinal system was misinterpreted by some respondents. To assess the impact of this, we performed a sensitivity analysis excluding such inconsistent responses, and again, a greater percentage chose 5-day outcomes as “most important” (data not shown). We also sought to assess the importance of certain clinical outcomes specific to PE. Most of these were amenable to discrete categorization. However, when piloting the survey we found that respiratory parameters were particularly intertwined with other factors (e.g., need for respiratory support is determined by oxygen saturation, which is influenced by exercise and patient comfort). We therefore chose to divide our questions about respiratory parameters into several questions, but acknowledge that the lack of a single descriptor of respiratory clinical deterioration is complex.

Conclusions

Five-day (in hospital) outcomes are of more interest to EPs when determining the disposition of ED patients with PE than longer time frames. EPs also consider a wide variety of events indicative of clinical deterioration. Researchers should consider this when developing and validating clinical decision rules.