Predicting outcome for ambulance patients with dyspnea: a prospective cohort study.

OBJECTIVE: To validate the discrimination and classification accuracy of a novel acute dyspnea scale for identifying outcomes of out-of-hospital patients with acute dyspnea.
METHODS: Prospective observational population-based study in the North Denmark Region. We included patients from July 1, 2017 to September 24, 2019 assessed as having acute dyspnea by the emergency dispatcher or by emergency medical services (EMS) personnel. Patients rated dyspnea using the 11-point acute dyspnea scale. The primary outcomes were hospitalization >2 days, ICU admission within 48 hours of ambulance run, and 30-day mortality. We used 5-fold cross-validation and area under receiver operating curves (AUC) to assess predictive properties of the acute dyspnea scale score alone and combined with vital data, age, and sex.
RESULTS: We included 3144 EMS patients with reported dyspnea. Median acute dyspnea scale score was 7 (interquartile range 5 to 8). The outcomes were: 1966 (63%) hospitalized, 164 (5%) ICU stay, and 224 (9%) died within 30 days of calling the ambulance. The acute dyspnea scale score alone showed poor discrimination for hospitalization (AUC 0.56, 95% confidence intervals: 0.54-0.58), intensive care unit admission (0.58, 0.53-0.62), and mortality (0.46, 0.41-0.50). Vital signs (respiratory rate, blood oxygen saturation, blood pressure, and heart rate) showed similarly poor discrimination for all outcomes. The combination of [vital signs + acute dyspnea scale score] showed better discrimination for hospitalization, ICU admission, and mortality (AUC 0.71-0.72). Patients not able to report an acute dyspnea scale score worse outcomes on all parameters.
CONCLUSION: The dyspnea scale showed poor accuracy and discrimination when predicting hospitalization, stay at intensive care unit, and mortality on its own. However, the dyspnea scale may be beneficial as performance measure and indicator of out-of-hospital care.

INTRODUCTION

Background

Breathing difficulty is common symptom associated with high morbidity and mortality. , , , Outcomes among patients with acute dyspnea/acute respiratory diseases have not improved compared with other emergencies such as myocardial infarction, stroke, and cardiac arrest. , The severity of acute dyspnea is difficult to assess objectively, and among vital signs, the respiratory rate and oxygen saturation only to some extent correlate to the perceived degree of dyspnea. , , Moreover, health care professionals’ assessment of the degree of dyspnea does not correlate well with the patients’ perceived dyspnea. ,

Recently, a simple verbal numeric rating scale, similar to scales assessing pain, was introduced in an Australian emergency department as a novel method to assess the patients’ perceived degree of acute dyspnea. , Using the scale, patients are asked to verbally assess their dyspnea on a scale from 0 to 10, which is easy and quick to perform in emergency care. In addition to a subjective rating scale, other tools for assessing dyspnea such as the Medical Research Council and Borg scale also incorporate elements such as a questionnaire on respiratory symptoms and verbal descriptions of breathing. , , ,

Importance

Symptom scoring may lead the ambulance paramedics and the ED to focus on the initial treatment and thus better relief of the patients’ acute dyspnea. Only limited data describe the association of emergency medical service (EMS)‐assessed dyspnea score with outcomes such as hospitalization, ICU stay, and mortality. , , To improve outcomes in critical illness, assessing the severity of the patients’ dyspnea plays an important role. No single vital sign in itself provides a good measurement of the degree of dyspnea. The 11‐point verbal dyspnea scale is simple and quick to use in emergency care, to monitor the changes during transport, but validation studies assessing the accuracy of the dyspnea scale for identifying outcome studies are scarce. It is important to investigate the accuracy and the discrimination for identifying outcomes using the dyspnea scale alone and in combination with vital signs.

Goals of this investigation

We sought to validate the discrimination and classification accuracy of an acute dyspnea scale for identifying outcomes among ambulance patients with acute dyspnea.

METHODS

Study design and setting

We conducted a prospective cohort study. The study is reported according to the guideline Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.

The study was approved by the Danish Data Protection Agency (North Denmark region record number 2008‐58‐0028 and project ID 2017‐128). Likewise, The Danish Patient Safety Authority approved access to out‐of‐hospital patient medical records (3‐3013‐2270/1). According to Danish legislation, registry‐based studies that do not involve biological material do not require approval from the National Committee on Health Research Ethics.

Setting

The study was carried out in the North Denmark Region, a 7885 km² rural and urban area with a widespread population of 589,731 citizens. , In Denmark, requests for emergency medical calls are handled through a national emergency number and forwarded to Emergency Medical Coordination Centres. Specialized nurses assess the urgency and the main reason for the call with the aid of a criteria‐based dispatch guideline, Danish Index for Emergency Care and dispatch the appropriate type of unit. Danish ambulances are manned with two paramedics and provide treatment and monitoring. Doctors in cars or helicopters are involved in prehospital care and dispatched in the most severe cases. Data from the ambulance is entered into a portable tablet computer, which transfers data to an electronic prehospital medical record (Amphi Systems A/S, Aalborg, Denmark). A monitor/defibrillator (LIFEPAK 15) can electronically measure blood pressure, heart rate, blood oxygen saturation, and respiratory rate (when monitoring end‐tidal CO2 using capnography, respiratory rate is otherwise measured manually). These measurements and accompanying timestamps are automatically transferred to the electronic out‐of‐hospital medical record.

Linkage of healthcare data

All Danish citizens have a personal civil registration number, which is used by all public authorities for unique identification, and healthcare contacts of any kind are registered using the citizen's personal civil registration number. , The National Patient Registry contains information on specifically all hospital contacts, including time and place of the contact, diagnoses, treatments, and procedures, but it does not include, for example, information of the patient's vital status (alive/dead). This information is available in the Danish civil registration system, which also contains information on age and sex, among other things.

Since 2015, all Danish medical emergency vehicles (except helicopter emergency medical services) have used the electronic out‐of‐hospital medical record, which is identical for the entire country. The record contains several hundred variables, including patient assessments, out‐of‐hospital treatments and medicine administrations, vital sign measurements, electrocardiograms, time of arrival, and more. Limited regional additions to the out‐of‐hospital medical record are possible.

Apart from the National Patient Registry, the Danish civil registration system, and the prehospital medical record, several other extensive regional and national registers exist, all linkable by the personal civil registration number. , This effectively enables the possibility of following the entire patient pathway and provides a complete follow‐up.

Selection of participants

We included all ambulance patients over the age of 18 years with acute dyspnea as the main symptom assessed, either by healthcare professionals at the emergency call or by ambulance professionals on seeing the patient, in the period July 1, 2017 to September 14, 2019.

Intervention—application of the dyspnea scale

We implemented a verbal numeric rating scale for assessing dyspnea (the acute dyspnea scale) in the North Denmark Regions ambulances. Ambulance professionals ask patients in Danish “On a scale from 0 to 10, where 0 is no difficulty breathing at all and 10 is the worst possible breathlessness imaginable, how are you experiencing your breathing now?” The patients’ dyspnea scores are registered in the electronic out‐of‐hospital medical record as numerical values 0–10, accompanied by either an automatic generated timestamp or a manually entered time if registration has been postponed (eg, by acute situation). If a patient is unable to use the acute dyspnea scale, ambulance professionals can enter the patient as unable because of (1) acute medical situation, or (2) other reasons (eg, language barriers). The ambulance professionals are instructed to ask patients to assess dyspnea at first and last contact (eg, patient released on scene or arrival at hospital). Patients were otherwise treated according to the ambulance professionals’ usual guidelines.

Outcomes

Our primary outcome measure was hospitalization for 2 or more days following the ambulance run. We chose 2 or more days to exclude patients only requiring brief treatment (eg, in the ED). Secondary outcomes were (1) admittance to intensive care unit within 48 hours following the ambulance run, and (2) mortality within 30 days following the ambulance run (ie, 30‐day mortality). Outcomes were determined according to the date of hospital admittance and release and whether there was a stay at ICU, according to the regional Patient Administrative System. Thirty‐day mortality was determined using possible date of death according to the Danish civil registration system.

Analysis

We obtained the patients’ acute dyspnea scale scores and vital sign measurements (respiratory rate [breaths/min], blood oxygen saturation [SpO2, %], blood pressure [mm Hg], and heart rate [beats/min]) from the electronic prehospital medical record. We included the vital sign values measured closest in time to the registered acute dyspnea scale scores. We included the first and last measured acute dyspnea scale scores and examined changes in dyspnea scores and vital signs. If a patient had any missing vital sign measurements, we omitted the patient from the statistical analysis including the related vital sign. Because one of our inclusion criteria was an assessment made on the phone at the emergency call, it is possible patients with no acute dyspnea were included in the study. We therefore excluded patients who had an initial (ie, first measured) acute dyspnea scale score of 0. For patients with >1 ambulance run in the study period, we only included the patients’ first contact when analyzing the outcome mortality.

To assess ability of the acute dyspnea scale to predict an outcome, we performed a 5‐fold cross‐validation. , We chose 5‐fold cross‐validation over regular logistic regression because it uses data for both training and validation, reducing variability and thereby giving a more accurate estimate of the abilities of the acute dyspnea scale. We initially randomized data into five equal groups. A logistic regression was then carried out, with one group used as validation data set, and the remaining four groups used as training data set, thereby providing 20% of the validation results. This was done five times, giving a combined 100% validation result from the logistic models (Figure 1). Using the combined 100% validation results from the five logistic models, we drew receiver operating characteristic curves and calculated area under the curve (AUC). We used the 5‐fold cross‐validation to assess the ability of the acute dyspnea scale to predict hospitalization for >2 days, as we computed a crude model for each individual parameter (Table 1). To assess whether the acute dyspnea scale scores could contribute to a prediction model, we first made a model adjusted for all variables, and then a model adjusted for all variables excluding the acute dyspnea scale scores. Likewise, we made the same model computations for assessments of the ability of the acute dyspnea scale to predict admittance to the ICU within 48 hours following the ambulance run, and 30‐day mortality following the ambulance run (Table 1). However, we reduced the number of variables included in the latter two assessments to reduce possible bias, because we assumed a lower number of patients would have these two outcomes.

FIGURE 1

An example of the 5‐fold cross‐validation used in the study.

TABLE 1

Included variables

Hospitalization for >days	Stay at intensive care unit within 48 h	Mortality within 30 days
Age	Age	Age
Sex	Sex	Sex
First measured dyspnea score	First measured dyspnea score	First measured dyspnea score
Delta dyspnea score	Delta dyspnea score	Delta dyspnea score
First measured respiratory rate (breaths/min)	First measured respiratory rate (breaths/min)	First measured respiratory rate (breaths/min)
Delta respiratory rate (breaths/min)	Delta respiratory rate (breaths/min)	Delta respiratory rate (breaths/min)
First measured blood oxygen saturation (%)	First measured blood oxygen saturation (%)	First measured blood oxygen saturation (%)
Delta blood oxygen saturation (%)	Delta blood oxygen saturation (%)	Delta blood oxygen saturation (%)
First measured systolic blood pressure (mm Hg)
Delta systolic blood pressure (mm Hg)
First measured heart rate (beats/min)
Delta heart rate (beats/min)

Included variables in the 5‐fold cross‐validations for each outcome.

We performed a supplementary logistic regression analysis of patients unable to use the acute dyspnea scale, either because of an acute medical situation or because of other reasons. Stata/MP 16.0 (StataCorp LLC, College Station, TX) were used for all statistical analyses.

RESULTS

Characteristics of study subjects

In the study period, there was a total of 116,409 ambulance runs, of which 4261 included patients reporting dyspnea. We excluded 897 patients unable to use the acute dyspnea scale (9% because of acute medical severity and 12% because of other reasons) and 220 (5%) patients with an initial acute dyspnea scale score of 0, leaving 3144 ambulance runs in the analysis. The 3144 ambulance encounters corresponded to 2461 individual patients (Figure 2; Table 2).

FIGURE 2

Flowchart for the patients included in the study.

TABLE 2

Patient characteristics

Characteristics
Age (mean years, SD, range	70 years (14.6, range: 18–104)
Sex (% female)	50%
Initial dyspnea score (median, interquartile range)	7 (5–8)
Ambulance run ended with:
Brought to hospital (n, %)	3031 (96%)
Treat‐and‐release (n, %)	31 (1%)
Other (n, %)	82 (3%)
Diagnoses given in hospital:
X: Respiratory diseases (n, %)	1743 (59%)
J44.1 Chronic obstructive pulmonarydisease with acute exacerbation,unspecified	769 (26%)
J18.9 Pneumonia, unspecified	301 (10%)
J96.0 Acute respiratory failure	150 (5%)
J44.9 Chronic obstructive pulmonarydisease, unspecified	146 (5%)
J45.9 Asthma, unspecified	80 (3%)
IX: Circulatory diseases (n, %)	408 (14%)
I48.9 Atrial fibrillation and atrial flutter, unspecified	63 (2%)
I50.9 Heart failure, unspecified	44 (1%)
I26.9 Pulmonary embolism without mention of acute cor pulmonale	29 (1%)
I35.0 Aortic (valve) stenosis	25 (1%)
I48.0 Paroxysmal atrial fibrillation	23 (1%)
XVIII: Symptoms and signs (n, %)	318 (11%)
R06.0 Dyspnea	176 (6%)
R07.4 Chest pain, unspecified	53 (2%)
R53 Malaise and fatigue	19 (1%)
R05 Cough	12 (<1%)
R06.4 Hyperventilation	6 (1.89)
XXI: Other factors (n, %)	174 (6%)
Z03.9 Observation for suspected disease or condition, unspecified	119 (4%)
Z76.8 Persons encountering health services in other specified circumstances	14 (<1%)
Z03.5 Observation for other suspected cardiovascular diseases	12 (<1%)
Z03.8 Observation for other suspected diseases and conditions	10 (<1%)
Z51.5 Palliative care	5 (<1%)
Remaining (n, %)	337 (10%)
Patient outcome
Hospitalization for >2 days (n, %)	1966 (63%)
Intensive care unit stay within 48 h (n, %)	164 (5%)
30‐day mortality (n, %)a	224 (9%)

The characteristics of the included 3144 patients. Diagnoses given in hospital according to International Statistical Classification of Diseases and Related Health Problems, 10th Revision, main chapters, and related specific diagnoses. Note diagnoses are only registered after release from hospital, patients still hospitalized at the end of the study period have no registered diagnosis available.

Only includes the patients’ first ambulance run in the study period.

Regarding the outcome, 1966 (63%) patients were admitted to the hospital for >2 days, 164 (5%) patients had an ICU stay within 48 hours, and 224 (9%) patients died within 30 days of calling the ambulance. The number of missing vital signs did not exeed >4% for any patient.

Main results

When exploring the predictive properties of the acute dyspnea scale regarding hospitalization for >2 days, the AUC for first measured acute dyspnea scale score was 0.56 (95% confidence interval, 0.54–0.58). This AUC was not different from first measured respiratory rate (0.60 [0.58–0.62]), age (0.60 [0.57–0.62]) and first measured heart rate (0.57 [0.55–0.59]). Only the first measured blood oxygen saturation had higher AUC (0.65 [0.63–0.67]) (Table 3). The adjusted model containing all other variables increased AUC to 0.71 (0.69–0.73), adding the acute dyspnea scale scores did not change the AUC (Figure 3A; Table 3). Concerning the admittance to the ICU within 48 hours of the ambulance call, the first measured acute dyspnea scale score showed similar a AUC (0.58 [0.53–0.62])(Table 3). When adding the acute dyspnea scores to the adjusted combined model, AUC increased to 0.73 (0.69–0.77) from 0.70 (0.66–0.75) (Figure 3B; Table 3).

TABLE 3

Outcome and AUC

Hospitalization for >2 days	Observations	AUC (95% CI)
All variables	2963	0.71 (0.69–0.73)
All variables excluding dyspnea scores	2963	0.71 (0.69–0.73)
First measured blood oxygen saturation (%)	2963	0.65 (0.63–0.67)
First measured respiratory rate (breaths/min)	2963	0.60 (0.58–0.62)
Age	2963	0.60 (0.57–0.62)
First measured heart rate (beats/min)	2963	0.57 (0.55–0.59)
First measured dyspnea score	2963	0.56 (0.54–0.58)
Delta blood oxygen saturation (%)	2963	0.56 (0.54–0.58)
First measured systolic blood pressure (mm Hg)	2963	0.53 (0.51–0.55)
Delta systolic blood pressure (mm Hg)	2963	0.52 (0.50–0.54)
Delta respiratory rate (breaths/min)	2963	0.51 (0.49–0.53)
Sex	2963	0.51 (0.49–0.53)
Delta heart rate (beats/min)	2963	0.49 (0.47–0.52)
Delta dyspnea score	2963	0.49 (0.47–0.51)
Stay at intensive care unit within 48 h of the ambulance run
All variables	2998	0.73 (0.69–0.77)
All variables excluding dyspnea scores	2998	0.70 (0.66–0.75)
First measured blood oxygen saturation (%)	2998	0.65 (0.60–0.70)
First measured respiratory rate (breaths/min)	2998	0.61 (0.57–0.66)
First measured dyspnea score	2998	0.58 (0.53–0.62)
Delta dyspnea score	2998	0.57 (0.53–0.62)
Delta blood oxygen saturation (%)	2998	0.53 (0.48–0.58)
Age	2998	0.52 (0.47–0.56)
Sex	2998	0.51 (0.46–0.56)
Delta respiratory rate (breaths/min)	2998	0.50 (0.45–0.55)
Mortality within 30 days a
All variables	2298	0.71 (0.67–0.75)
All variables excluding dyspnea scores	2298	0.71 (0.67–0.75)
Age	2298	0.66 (0.63–0.70)
First measured blood oxygen saturation (%)	2298	0.64 (0.60–0.68)
Delta blood oxygen saturation (%)	2298	0.56 (0.52–0.60)
First measured respiratory rate (breaths/min)	2298	0.53 (0.49–0.57)
Delta respiratory rate (breaths/min)	2298	0.52 (0.48–0.56)
Sex	2298	0.51 (0.47–0.55)
Delta dyspnea score	2298	0.47 (0.43–0.51)
First measured dyspnea score	2298	0.46 (0.41–0.50)

Abbreviations: 95% CI, 95% confidence interval; AUC, area under the curve; SD, standard deviation.

The area under the receiver operating curve for the adjusted model with all variables, the adjusted model with all variables barring the dyspnea scale scores, and the individual variables.

Only includes the patients’ first ambulance run in the study period.

FIGURE 3

Receiver operating characteristic curves. Curves including the first measured dyspnea scale score and the two individually best performing variables. The adjusted model with all variables, and the adjusted model with all variables barring dyspnea scale scores. All curves for the outcomes include (A) hospitalization for 2 or more days, (B) stay at intensive care unit within 48 hours, and (C) 30‐day mortality for the patients’ first ambulance run in the study period

When predicting mortality within 30 days of the ambulance run, the acute dyspnea scale scores had the lowest AUC of all variables with the first measured score at 0.46 (0.41–0.50). Blood oxygen saturation was the only vital sign that showed a higher AUC with 0.64 (0.60–0.68) alongside age with an AUC of 0.66 (0.63–0.70). The adjusted combined model had higher AUC 0.71 (0.67–0.75), and the acute dyspnea scale scores did not contribute when added to the model (Table 3 and Figure 3C)

Among patients unable to use the acute dyspnea scale because of an acute medical situation and because of other reasons, 67% were hospitalized, 10% had an ICU stay, and 27% died within 30 days of the ambulance run. The patients did not have a significantly increased likelihood of hospitalization for 2 or more days, when adjusting for respiratory rate, blood oxygen saturation, systolic blood pressure, heart rate, age, and sex. The patients did have an increased likelihood of stay in the ICU, if they were unable to use the score because of an acute medical situation (odds ratio [OR] = 3.25 [2.33–4.54]) when adjusting for respiratory rate, blood oxygen saturation, age, and sex. For mortality, patients unable to use the acute dyspnea scale because of an acute medical situation or because of other reasons had a significantly increased likelihood of 30‐day mortality (OR = 2.89 [2.15–3.90] and 1.84 [1.40 to 2.43]) when adjusting for respiratory rate, blood oxygen saturation, age, and sex.

Limitations

The inclusion criteria and potential inclusion bias are regarded as the main limitation in this study.

First, 21% of the dyspnea patients were not included because they were unable to use the acute dyspnea scale, hereof 9% due to an acute medical situation. Only patients able to provide an acute dyspnea scale score were included in our analyses for predicting outcomes. Therefore, the acute dyspnea scale score in itself cannot be generalized to the most severely impaired patients (eg, those confused or unconscious). However, our study showed patients unable to use the acute dyspnea scale had an increased likelihood of stay in the ICU and 30‐day mortality. The fact that they are unable to use the acute dyspnea scale can therefore be considered the initial measurement of the patients’ medical severity.

Second, it is possible that patients with no dyspnea were included in the study at the emergency call assessment. When met by ambulance professionals, the patients’ actual medical situation may differ from the assessment made over the telephone. We attempted to limit this bias by excluding patients with an initial acute dyspnea scale score of 0.

Our outcome measure of hospitalization for 2 or more days was chosen to exclude patients in need of only short treatment before being released from the hospital (eg, exacerbation of chronic obstructive lung disease and treatment at ED). This outcome relates to a more severe patient group, in contrast to patients with any hospital contact.

Only the patients’ first contacts were included when analyzing the predictive value on mortality, which introduces a possibility of underestimating the mortality. We chose this method to obtain a conservative estimate, in contrast to the risk of overestimating the patients’ mortality by only including the patients’ last contact.

We did not include descriptive variables such as lung function, history of smoking, and comorbidity in the analysis. The inclusion of these variables could have affected the outcome. However, this information will not usually be available in the emergency out‐of‐hospital setting.

The 5‐fold cross‐validation efficiently used data for both training and validation. In combination with the large number of patients, the very low number of missing scores, and the close proximity of registred vital signs and the acute dyspnea scale score were all major strengths of the study.

DISCUSSION

In summary, we found that the acute dyspnea scale showed poor discrimination for hospital admission, ICU admission, and mortality. Vital signs similarly demonstrated poor discrimination for hospital outcomes. However, models combining acute dyspnea scale scores with vital signs showed better discrimination for hospital outcomes.

Despite overlapping CIs for the outcome stay in the ICU, blood oxygen saturation outperformed acute dyspnea scale scores for all three outcomes, with an AUC of 0.65–0.64 versus 0.58–0.46. However, as all other vital signs, it had AUCs suggesting poor performance and only the combined models managed to obtain a fair accuracy with an AUC ranging from 0.71–0.73. Including acute dyspnea scale scores to the adjusted models containing all variables improved the AUC for stay in the ICU, but did not contribute to predicting hospitalization and mortality. These findings emphasize an important point, namely the need for including several parameters when assessing outcome, and not only symptom scores or vital signs.

A similar verbal dyspnea scale to the one used in the current study was used in an Australian study that included fewer patients (n = 249) presenting with shortness of breath in an ED. The study found that the dyspnea score by itself could predict the need for inpatient admission, from low to high according to dyspnea scores 0–3, 4–5, 6–7, and 8–10. Furthermore, the study indicated that a dyspnea score above 6 combined with heart rate above 94 beats/min, and arrival by ambulance predicted the need for inpatient admission. In contrast to our study, they used cut‐off points to improve the predictive abilities of the verbal dyspnea score. It is possible that identification of ideal cut‐off points for the acute dyspnea scale scores, both by itself and in combination with other variables, could improve the predictive performance in prehospital acute dyspnea patients.

Our findings suggest the acute dyspnea scale score by itself has limited relevance in predicting patient outcome among ambulance patients. However, the combined acute dyspnea scale score and vital signs had a higher AUC (0.70–0.73) than any measure by itself, making it fairly useful—identifying ideal cut‐off points for a combined acute dyspnea scale and vital sign score could result in a Danish Dyspnea Scale score that could be implemented in the prehospital medical record. We consider future studies to investigate this element.

Despite the limited relevance of the acute dyspnea scale score by itself, it does provide highly clinically relevant information of the patent's experience of symptoms that might be unavailable otherwise. As such, it could be used as a performance measure and indicator of prehospital treatment (eg, does the patient experience a relief of symptom following specific treatment). Future studies investigating the clinical relevance of the dyspnea scale in this regard are planned.

In summary, the acute dyspnea scale in itself showed poor accuracy and discrimination, and on a similar level as the usually measured vital signs, when predicting hospitalization, stay in the ICU, and mortality.

CONFLICT OF INTEREST

The authors declare no conflict of interest.