Hospital mortality is associated with ICU admission time.

INTRODUCTION: Previous studies have shown that patients admitted to the intensive care unit (ICU) after "office hours" are more likely to die. However these results have been challenged by numerous other studies. We therefore analysed this possible relationship between ICU admission time and in-hospital mortality in The Netherlands.
METHODS: This article relates time of ICU admission to hospital mortality for all patients who were included in the Dutch national ICU registry (National Intensive Care Evaluation, NICE) from 2002 to 2008. We defined office hours as 08:00-22:00 hours during weekdays and 09:00-18:00 hours during weekend days. The weekend was defined as from Saturday 00:00 hours until Sunday 24:00 hours. We corrected hospital mortality for illness severity at admission using Acute Physiology and Chronic Health Evaluation II (APACHE II) score, reason for admission, admission type, age and gender.
RESULTS: A total of 149,894 patients were included in this analysis. The relative risk (RR) for mortality outside office hours was 1.059 (1.031-1.088). Mortality varied with time but was consistently higher than expected during "off hours" and lower during office hours. There was no significant difference in mortality between different weekdays of Monday to Thursday, but mortality increased slightly on Friday (RR 1.046; 1.001-1.092). During the weekend the RR was 1.103 (1.071-1.136) in comparison with the rest of the week.
CONCLUSIONS: Hospital mortality in The Netherlands appears to be increased outside office hours and during the weekends, even when corrected for illness severity at admission. However, incomplete adjustment for certain confounders might still play an important role. Further research is needed to fully explain this difference.

Introduction

Ideally, care for critically ill patients is optimal, 24 h per day. Unfortunately, this is not the case. During so-called off hours, staffing is often reduced, and diagnostic and therapeutic procedures might take longer or are postponed until office hours. Because treatment in the first hours after admission to the intensive care unit (ICU) is related to outcome [1, 2], admission outside office hours might be associated with increased mortality [3, 4].

However, this increased mortality outside office hours has been questioned by several other studies [5-10], suggesting that the association was confounded by differences in the definition of office hours, differences in study population, study size, healthcare organisation or insufficient correction for case mix and illness severity. As a consequence of these differences in methodology and organisational differences between countries, it remains unclear whether there is a relationship between admission outside office hours and increased hospital mortality.

We hypothesised that mortality during off hours is higher than during office hours and that previous studies that did not find this result were probably underpowered or defined office hours too broadly. We therefore analysed hospital mortality in relation to admission time in a very large database with stringent case-mix correction. We also analysed whether sample size could have influenced the results of previous studies. This approach enables us to analyse the relationship between admission time and hospital mortality in the most robust way thus far.

Methods

Patient data

Since 1996 the National Intensive Care Evaluation (NICE) foundation has collected data on admissions to ICUs in The Netherlands. NICE started with 6 ICUs in 1996, and in 2008 more than 70 ICUs were participating in this registry, accounting for 80% of all Dutch ICUs, and approximately 50,000 admissions were included in this registry in 2008. Details about inclusions and exclusions in the Dutch registry have been published previously [11]. In short, the participating ICUs are mixed medical-surgical units located in university hospitals (n = 7), teaching hospitals (n = 25) or non-teaching hospitals (n = 38). A data set of about 100 items is collected for each individual patient. Based upon this data several prediction models can be calculated, such as APACHE II [12], which are used to correct the crude hospital mortality for illness severity at admission.

Data collection takes place in a prospective and standardised manner according to strict and uniform definitions and is subject to stringent data quality checks. Additionally, site visits are performed to ensure the quality of the collected data. This has been shown to ensure high quality of data [13]. The data is aggregated at a central point (NICE Foundation, Amsterdam, The Netherlands). All patient identifying information, such as name and patient identification number, is encrypted. Data are analysed and stored in an anonymous way and are not traceable to any patient, and therefore informed consent was not needed.

All patient records in this database between 2002 and 2008 were included in our analysis. According to the original APACHE II exclusion criteria, cardiopulmonary surgical patients, re-admissions to the ICU, patients who were discharged or died within 8 h after admission and burn patients were excluded.

Defining office hours

Office hours were defined by the presence of a fully qualified intensivist available for patient care. In The Netherlands, three levels of intensive care are identified. The minimal demands for the three levels of intensive care are available in the ESM. Despite organisational differences, commonly intensivists are available from 08:00 to 22:00 hours. Therefore, we defined 08:00–22:00 hours during weekdays and 09:00–18:00 hours during the weekends as office hours. A weekday was defined as a day from 00:00 hours until 24:00 hours, and the weekend was defined as from Saturday 00:00 hours until Sunday 24:00 hours.

Data analysis

Hospital mortality by day of week, in or outside weekend and during office hours versus off hours were examined first using univariate analysis and then with multiple logistic regression adjusted for case mix. Adjustment for case mix was undertaken by using the Dutch APACHE II model. To date, there have been no studies that have shown that prognostic models are stable over time, in a new setting, and with different case mixes [14, 15]. As a result of medical progress and advancement of science, it is expected that the model’s performance will decline over time [16]. To account for this decline in performance we recalibrated the APACHE II before using it for case-mix correction. This involved fitting a new logistic regression equation with in-hospital mortality as dependent variable and the original covariates (APACHE II score, APACHE II reason for admission and admission type) as independent variables. Accordingly, customisation does not change the influence of individual covariates included in the model but modifies their joint influence on the observed mortality in the external dataset [17-19].

Three different recalibrated logistic regression models were used to analyse the association between in-hospital mortality and off hours versus office hours (model 1), weekdays versus weekend (model 2) and day of week (model 3). For better risk adjustment, patients admitted during office hours versus off hours, weekend versus weekday, and day of the week were matched based on a calculated propensity score. This score expresses the probability that a patient falls in the office hours or off hours, weekday or weekend, or day of week group, and its log odds was added as a covariate in the recalibrated APACHE II model for adjustment.

When the incidence of an outcome of interest is common in the study population (>10%), the adjusted odds ratio derived from the logistic regression could under- or overestimate the risk ratio. Therefore, the associations between admission time and mortality are reported as relative risk and their 95% confidence interval (95% CI). The relative risks were approximated from the adjusted odds ratio derived from the three logistic regression models [18] and considered statistically significant if the confidence interval did not contain zero.

Categorising a continuous variable (admission time) into office hours and off hours may conceal the behaviour of the variables over time. Therefore, the relationship between admission time and hospital mortality was also graphically inspected using locally weighted scatterplot smoothing (LOWESS). In simple terms, at each time point, e.g. 15:00 hours, simple models are fitted to the mortality data in the “vicinity” of the point, e.g. from 13:00 to 17:00 hours, with points closer to 15:00 hours having more influence than those farther away. This procedure is performed for all data points. Instead of the scatter plot of 0 and 1 values of mortality over time we now have a smoothed curve that reflects the underlying structure of the data.

To investigate whether smaller sample size could explain the results of previous articles on this subject that showed no relation between mortality and time of admission, we used 300 random sub-samples from our research dataset with sample size equal to that in two previous articles: 56,250 and 6,725 patients [6, 10]. Data were analysed using the R2.6.2 statistical environment and SPSS 16.0 (Chicago, IL, USA).

Results

From January 2002 to January 2009, a total of 149,894 patients eligible according to the APACHE II inclusion criteria were included in the analyses. Baseline characteristics of these patients are shown in Table 1. Table 2 shows the distribution of patients according to admission day, weekend versus weekdays and off hours versus office hours. This table shows that the predicted and observed mortality of patients during the weekends and off hours is higher than during the week and office hours, respectively.

Table 1

Baseline characteristics of the study population (N = 149,894)

Patient characteristic	N (%)
Hospital mortality	28,135 (19)
ICU mortality	18,051 (12)
Male	86,977 (58)
Age (years), mean (SD)	61.9 (17)
Ventilated in first 24 h of admission	71,294 (48)
Median APACHE II score (25–75%)	15.0 (10-21)
Type of admission
Non-surgical/medical	71,181 (47)
Emergency surgery	26,298 (18)
Planned surgery	52,415 (35)

Table 2

Characteristics of the patients admitted per time variable (N = 149,894)

Time variable	Admissions N	Male (%)	Age, mean (SD)	APACHE II score, median (25–75%)	Expected mortality, median (25–75%)	Expected mortality, mean (SD)	Observed hospital mortality, N (%)	SMR (95% CI)
Sunday	13,518	57	59.9 (18.7)	18 (12–24)	26.6 (11.2–51.5)	33.3 (26.3)	3,517 (26.0)	0.78 (0.76–0.81)
Monday	24,499	59	61.9 (16.4)	15 (10–21)	15.2 (6.2–35.9)	24.6 (24.2)	4,336 (17.7)	0.72 (0.7–0.74)
Tuesday	25,394	58	62.6 (16.0)	14 (10–20)	14.6 (6.2–34.8)	24.1 (24.0)	4,352 (17.1)	0.71 (0.69–0.73)
Wednesday	25,627	58	62.5 (16.1)	14 (10–20)	14.3 (6.1–34.6)	23.8 (23.7)	4,333 (16.9)	0.71 (0.69–0.73)
Thursday	24,222	58	62.2 (16.4)	15 (10–20)	14.8 (6.3–35.3)	24.2 (23.7)	4,066 (16.8)	0.69 (0.67–0.72)
Friday	23,259	58	62.4 (16.5)	15 (10–21)	15.9 (6.7–36.9)	25.1 (24.1)	4,255 (18.3)	0.73 (0.71–0.75)
Saturday	13,375	57	60.3 (18.4)	17 (12–24)	25.9 (11.2–49.5)	32.7 (26.0)	3,276 (24.5)	0.75 (0.72–0.78)
Weekend	26,893	57	60.1 (18.6)	18 (12–24)	26.2 (11.2–50.1)	33.0 (26.1)	6,793 (25.3)	0.77 (0.75–0.78)
Week	123,001	58	62.3 (16.3)	15 (10–21)	14.8 (6.3–35.5)	24.4 (23.9)	21,342 (17.4)	0.71 (0.70–0.72)
Office hours	107,762	58	62.8 (15.9)	14 (10–20)	14.3 (6.2–34.6)	23.9 (23.8)	18,452 (17.1)	0.72 (0.71–0.73)
Off hours	42,132	56	59.6 (18.5)	17 (11–23)	23.6 (9.6–47.1)	30.9 (25.8)	9,683 (23.0)	0.74 (0.73–0.76)

Standardised mortality ratio (SMR) calculated as observed mortality divided by mortality predicted by the APACHE II model

Figure 1 shows the actual observed mortality and the predicted mortality probability of the recalibrated APACHE II model in relation to admission time. The lowest line shows the difference between the observed and predicted mortality proportion. This figure shows qualitatively that the APACHE II model predictions closely follow the observed mortality proportion over all admission times. The difference between the observed and predicted mortality is minimal (the lowest line is almost horizontal) except for the hours close to midnight. The lowest mortality is seen during office hours, especially during the afternoon. Then, mortality slowly increases again, with a peak around 05:00–06:00 hours in the morning. The figure shows that the observed proportion minus the predicted probability during office hours was lower than during off hours (17.5% versus 22.7%). In the whole off hours region the difference between observed and predicted mortality was positive (meaning that there was more mortality than predicted), which was not the case in the office hours region. This observation provides face validity for treating office hours versus off hours as a dichotomous variable in the logistic regression model. The graph also suggests that the effect of off hours on mortality would have been even stronger if off hours had been defined as, say, 20:00–08:00 hours.

Fig. 1

Relationship between observed and predicted mortality (in percentages) in relationship to admission time. The predicted mortality is based upon the APACHE II model and observed mortality. The APACHE II model has been recalibrated to better fit the Dutch ICU population (see “Methods” section for explanation). The upper lines are overlapping, which shows that the model correctly predicts mortality in the general Dutch ICU population. The difference between the upper lines is minimal (almost horizontal lower line). To illustrate the pattern of the difference between the lines, the lower figure blows up this difference. Both the model as well as the observed mortality change with admission time. The lowest mortality is seen during office hours (08:00–22:00 hours), and the highest mortality is seen during off hours (22:00–08:00 hours)

Table 3 shows the relative risk and 95% confidence intervals of the three different models, investigating the days of weeks, weekend versus weekdays and off hours versus office hours after case-mix correction. Admissions during off hours have a significantly higher mortality risk [RR 1.059 (1.031–1.088)] than admissions during office hours. The same holds for admissions during the weekend [RR 1.103 (1.071–1.136)] in comparison with weekdays, although the mortality risk is already increased for patients admitted on Friday [RR 1.046 (1.001–1.092)].

Table 3

Relative risk for hospital mortality during off hours, the weekend and during the week

Time	Relative risk	95% Confidence interval
Model 1
Office hours	1.0	–
Off hours	1.059	1.031–1.088*
Model 2
Week	1.0
Weekend	1.103	1.071–1.136*
Model 3
Monday	1.006	0.962–1.051
Tuesday	0.999	0.956–1.044
Wednesday	1.0	–
Thursday	0.977	0.934–1.022
Friday	1.046	1.001–1.092*
Saturday	1.062	1.012–1.122*
Sunday	1.113	1.062–1.166*

The relative risk for mortality was corrected for age, gender, APACHE score, admission type and reason for admission. Mortality after correction for these variables is significantly higher during off hours (22:00–08:00 hours during weekdays and 18:00–08:00 hours during weekend days). Mortality is also significantly higher during the entire weekend in comparison with office hours during the weekdays. However, mortality did not differ much between weekdays during the office hours, except on Friday

* Significant difference

We also analysed whether smaller sample size would have influenced our results. The 300 random sub-samples from our research dataset based on the largest sample size among studies reporting negative findings (N = 56,250) [6] showed a significant difference between office hours versus off hours in 85% of our samples. However, the analyses on 300 random sub-samples based on the smallest sample size among studies reporting negative findings (N = 6,725) [10] only showed significantly higher mortality during off hours in 15% of samples, and significantly higher mortality during office hours in 1% of samples.

Discussion

This study showed an increase in the risk of hospital mortality for patients admitted during off hours compared with patients admitted during office hours (RR 1.059), and an increase of hospital mortality risk for patients admitted during the weekend compared with patients admitted during the week (RR 1.103).

Several analyses that describe the difference in mortality between patients admitted during office hours and those admitted to the ICU during off hours have been published [5–10, 20–23]. Unfortunately, all of these studies defined office hours differently. We defined working hours as those hours when a qualified intensivist was available for direct patient care. For most ICUs in The Netherlands this is from 08:00 to 22:00 hours. This definition is in accordance with another recent Dutch publication [10]. However, our results contradict some of these more recent publications on this subject. For example, Meynaar et al. [10] analysed the difference in mortality of 6,725 patients admitted during office hours and outside office hours. They could not detect a difference in mortality after correction for case mix and illness severity. As their sample size of only 6,725 patients was much smaller than our research sample, we used 300 random sub-samples from our research dataset to replicate their sample size. We found a statistically significant difference in only 49 of 300 samples (15%). Although the sub-samples are overlapping, this suggests that their sample size might have been too small to detect these differences in mortality. However, they analysed only three ICUs located in teaching hospitals, which are possibly much more homogenous in performance over time compared with our mixed set of 70 participating ICUs.

On the other hand, our results also contradict the largest analysis thus far [6]. In a UK database of 56,250 ICU patients, Wunsch et al. found increased mortality for the weekends (Friday–Sunday) and during the evening and night. However, after correction for case mix, this difference disappeared. They concluded that there was no difference in outcome between office hours and off hours. However, they defined office hours differently, choosing three shifts (08:00–18:00, 18:00–24:00 and 24:00–08:00 hours) that best reflect ICU care in the UK. An analysis on 300 random sub-samples from our dataset sample with their sample size (n = 56,250) showed a significant difference in mortality between office hours and off hours in 256 of the 300 samples. Although the sub-samples are overlapping, this suggests that the difference in conclusion is not likely based upon limited power in the study by Wunsch et al. We corrected in a similar way for potential confounders, and therefore our analyses are comparable. This suggests that the increased mortality in our study might be based on differences in staffing or logistics between the UK and The Netherlands.

Of course, ICU performance is influenced by an intricate interplay of various factors. Besides the ICU organisation during off hours there is intensive interaction with other medical disciplines, and changes in their quality of care during off hours might influence ICU outcome as well, which is not reflected by illness severity at admission. Such unknown confounders might be stronger in smaller hospitals, which often have less staff to fill the roster and/or have less sophisticated diagnostics than larger hospitals. Furthermore, the performance of health care workers (physicians and nurses) varies during the day. Although speculative, the detrimental effect of the circadian biorhythm on human performance during the night shift and especially at the end of the night shift is a known factor [24]. We also found the highest (predicted and observed) mortality at the end of the night shift (05:00–06:00 hours), when both health care workers and patients perform at their worst.

This study has several limitations. Although the data were collected in prospective fashion during the first 24 h of admission to the ICU and the outcome (discharged alive or dead) was not influenced by subjective assessment, this remains a retrospective analysis. Therefore, true cause-and-effect relationships cannot be ascertained, and unmeasured confounders still might play a role. For example, differences of care beyond the ICU might be confounders in the association between admission timing and mortality. However, all surviving patients are discharged to the same wards, and it is reasonable to assume that they all experience the same quality of care on the wards.

Although APACHE II was used to correct for illness severity and patients were matched based upon a propensity score (age, gender, APACHE score, admission type, reason for admission), this still does not fully exclude the influence of case-mix differences. Surgical patients admitted in the middle of the night are different from patients admitted during office hours. For example, does waiting for emergency surgery during off hours result in higher acute physiology score and subsequently to higher APACHE II score (so-called lead-time bias)? Other, possibly very important, confounders might be differences in organisational aspects. Unfortunately, the data in this study could not be corrected for these organisational aspects, and we assumed office hours to be 08:00–22:00. Figure 1 shows that there is an increase in the difference between observed and predicted mortality from 16:00 hours onwards. The effect of off hours on mortality would have been stronger if off hours had been defined as, say, 20:00–08:00 hours. As of 2008, all ICUs have collected data on quality parameters, such as nurse-to-patient ratio, physician-to-patient ratio, availability of ICU beds, etc. Such information might explain the differences between ICUs and explain why some ICUs apparently have equal performance during the entire day while others perform worse during off hours.

However, a strong feature of this analysis is its size and its power to detect these differences. This is one of the largest analyses of admission timing and survival. Additionally, this analysis is performed in a large sample of the Dutch ICUs (up to 80% of the Dutch ICUs in 2008), and therefore these results can be extrapolated to represent the level of critical care in The Netherlands. Previous studies were often performed in a smaller subset of ICUs and might represent the better-performing ICUs. Such decreased external validity might explain a lack of difference between office hours and off hours in these studies.

We conclude that admission timing is associated with differences in outcome, even when mortality is corrected for illness severity by means of recalibrated APACHE II score. Patients admitted during the night (22:00–08:00 hours) or weekend days have a decreased chance of survival in comparison with patients admitted during office hours. However, the cause of this association needs further analysis that corrects for more potential confounders. This investigation has been started in The Netherlands with the registration of various quality-of-care variables.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 25 kb)

Supplementary material 2 (DOC 43.5 kb)