An observational study: associations between nurse-reported hospital characteristics and estimated 30-day survival probabilities.

BACKGROUND: There is a growing body of evidence for associations between the work environment and patient outcomes. A good work environment may maximise healthcare workers' efforts to avoid failures and to facilitate quality care that is focused on patient safety. Several studies use nurse-reported quality measures, but it is uncertain whether these outcomes are correlated with clinical outcomes. The aim of this study was to determine the correlations between hospital-aggregated, nurse-assessed quality and safety, and estimated probabilities for 30-day survival in and out of hospital.
METHODS: In a multicentre study involving almost all Norwegian hospitals with more than 85 beds (sample size=30, information about nurses' perceptions of organisational characteristics were collected. Subscales from this survey were used to describe properties of the organisations: quality system, patient safety management, nurse-physician relationship, staffing adequacy, quality of nursing and patient safety. The average scores for these organisational characteristics were aggregated to hospital level, and merged with estimated probabilities for 30-day survival in and out of hospital (survival probabilities) from a national database. In this observational, ecological study, the relationships between the organisational characteristics (independent variables) and clinical outcomes (survival probabilities) were examined.
RESULTS: Survival probabilities were correlated with nurse-assessed quality of nursing. Furthermore, the subjective perception of staffing adequacy was correlated with overall survival.
CONCLUSIONS: This study showed that perceived staffing adequacy and nurses' assessments of quality of nursing were correlated with survival probabilities. It is suggested that the way nurses characterise the microsystems they belong to, also reflects the general performance of hospitals. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Healthcare organisations are complex systems that require multifaceted approaches to achieve the goal of patient safety by minimising adverse events that may lead to patient harm.1 One approach towards this goal has been to examine how the properties of the organisation can become latent conditions for hazards. Knowledge about how organisational properties work together may be helpful in designing the work environment and the organisation so as to enhance patient safety. The aspect of design may be illustrated by the Swiss cheese model, in which the holes in the Swiss cheese slices represent the aspects of design that allow errors to occur, and how changes in the way the slices are arranged may close these holes and thus prevent those errors from occurring.2 Care is provided by frontline healthcare workers, and their ability to ensure quality of care and to discover and close the ‘holes’ may be facilitated or constrained by the organisational properties.2

Donabedian's structure–process–outcome model serves as a framework for identifying organisational properties that have an impact on clinical outcomes.1 3 Organisational structures describe the setting of healthcare delivery. Structure measures, such as nurse staffing levels, teaching status and volume are associated with clinical outcomes,4–8 but evidence for impact on clinical outcomes is inconsistent.9 Other more changeable and dynamic properties, such as the organisational processes,10 11 should, therefore, be addressed when attempting to design patient-safe organisations.9 Processes may be defined and operationalised as the care delivered (clinical process measures) or as organisational processes measures that describe the work environemnt.3 12 13 It has been suggested that the impact of safety culture vary with the work environment,14 and several studies have shown that nursing work environments are associated with clinical and nurse-reported quality.5 15 16 17 18 Improvements in nurses’ perceptions of organisational process measures in their work environment are likely to have an impact on nurse-reported quality. Two studies have shown that nurse-reported quality was associated with clinical outcomes such as hospital standardised mortality rates (HSMR), 30-day mortality, failure to rescue and patient satisfaction.19 20 However, the evidence for relationships between self-reported quality and clinical outcomes is sparse, and a confirmation of these relationships may enhance the potential to learn from studies that use quality measures reported by healthcare workers as outcome measures.10 17–20

In a previous study, we found that four subscales from a questionnaire measuring nurses’ work environment were associated with nurse-assessed patient safety and quality: (1) quality system, (2) staffing adequacy, (3) nurse–physician relationship and (4) patient safety management.11 These variables with their interactions were regarded as mutually dependent elements and will be referred to as a ‘complex parameter’ that represents one way to describe the organisation. In the present study, the complex parameter was applied to explore the associations between nurse-reported work environment, nurse-assessed quality and safety, and clinical outcomes.

Objectives

The aim of this study was to determine the correlations between hospital-aggregated, nurse-assessed quality and safety measures, and estimated probabilities for 30-day survival in and out of hospital. The complex parameter described in the introduction was applied to examine the associations between nurse-reported organisational process measures and hospital-level clinical outcomes.

Methods

Design

This observational, ecological study combined data from two sources: (1) a multicentre cross-sectional nurse survey in Norwegian hospitals conducted as part of the international project entitled European Nurse Forecasting: Human Resources Planning in Nursing (RN4Cast) and (2) estimated probabilities for 30-day survival in and out of hospital computed by The Norwegian Knowledge Centre for the Health Services based on patient administrative data collected from Norwegian hospitals. Estimated probabilities for 30-day survival in and out of hospital will hereafter be referred to as ‘survival probabilities’. Hospital averages for the nurses’ responses were computed and merged with hospital-level survival probabilities for 30 hospitals. This study included hospital-level data (survival probabilities) to compare different properties of hospital performance. Since the unit of observation was hospitals, the ecological design was most suitable.

Discharge data

Survival measures are commonly accepted quality indicators, for example, the HSMR that is corresponding to overall survival described in this study. However, the survival probabilities in the present study include a standardised 30-day follow-up during and after hospital admission. Time is an essential element in mortality measures, and this standardisation prevents that variation in length of stay between the patients compromise the validity of the quality indicator.21 In Norway, 30-day survival probabilities are estimated in and out of hospitals for patients admitted for acute myocardial infarction (AMI), stroke and hip fracture. To limit the scope of this study, survival after hip fracture was not included in the present study. The survival probabilities were computed for patients discharged in the period 2005–2009. The discharge data provide information about age, sex, diagnoses, procedure codes, length of stay, departments, municipality of residence, type of admission (acute or elective) and in-hospital mortality. The database was linked to the National Population Register to obtain the exact date of death, so that the cases were followed-up for 30 days from admission during hospitalisation and after discharge (in and out of hospital). Norwegian hospitals’ trusts often contain several hospitals, and the survival probabilities were computed at the hospital level. In the present study, survival probabilities after AMI (survival AMI), after stroke (survival stroke) and overall survival were included as clinical outcome measures. Overall survival for 2009 was computed for diagnoses covering 80% of hospital deaths in hospitals with more than 800 admissions. Inclusion of diagnoses covering 80% of hospital deaths was made according to procedures for estimation of HSMR.22 Cases for AMI and stroke were identified by admission codes according to International Classification of Diseases tenth revision.23 Survival probabilities were computed for hospitals with more than 100 cases in the period 2001–2005 and more than 20 for 2009.

A case was defined as starting with first admission of the patient and was followed within the hospital and when transferred to other hospitals (with delay <8 h). To attribute the outcome to the hospitals involved in a case, hospital weighting proportional with the length of stay for each hospital was implemented.21 Risk adjustments were made for age, sex, number of admissions (during the previous 2 years) and Charlson comorbidity (during the previous 3 years), and are more thoroughly described elsewhere.21 The present study made use of survival probabilities that were published as national quality indicators, and so ethical approval was not necessary.

Nurse survey

The Norwegian branch of RN4Cast was conducted during autumn 2009 involving a survey among nurses working bedside in surgical and medical wards in 35 Norwegian hospitals (with more than 85 beds). The data collection procedure is described elsewhere, and the method of data collection and handling was approved by the Data Protection Official for Research.11 Two global questions from the RN4Cast questionnaire were used to compare nurses’ assessments of quality of nursing and patient safety with clinical outcomes:

Quality of nursing: ‘In general, how would you describe the quality of nursing care delivered to patients on your unit/ward?’ (Responses were on a 4-point Likert-type scale, where 1=poor, 2=fair, 3=good and 4=excellent)

Patient safety: ‘Please give your department an overall grade on patient safety’ (Responses were on a 5-point Likert-type scale, where 1=failing, 2=poor, 3=acceptable, 4=very good and 5=excellent)

The complex parameter was based on nurses’ descriptions of their work environment and patient safety management. From the instrument Nursing Work Index—Revised (NWI-R), three subscales that are described in table 1 were included (responses were on a 4-point Likert-type scale, where 1=strongly disagree, 2=somewhat disagree, 3=somewhat agree and 4=strongly agree). Patient safety management was defined as a composite score including three items that were originally included in the Hospital Survey on Patient Safety Culture (HSOPSC) (responses were on a 5-point Likert-type scale, where 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree and 5=strongly agree) (see table 1).24

Table 1

Construction of the subscales: presentation of the single items and the instrument origin (Nursing Work Index—Revised (NWI-R), Hospital Survey on Patient Safety Culture (HSOPSC)

Subscales	Single items
Staffing adequacy (NWI-R)	Enough time and opportunity to discuss patient care problems with other nurses
	Enough registered nurses to provide quality patient care
	Enough staff to get the work done
Nurse–physician relationship (NWI-R)	Physicians and nurses have good working relationships
	A lot of team work between nurses and physicians
	Collaboration (joint practice) between nurses and physicians
	Physicians value nurses’ observations and judgments
	Physicians recognise nurses’ contributions to patient care
	Physicians respect nurses as professionals
	Physicians hold nurses in high esteem
Quality system (NWI-R)	A clear philosophy of nursing that pervades the patient care environment
	Working with nurses who are clinically competent
	An active quality assurance programme
	A preceptor programme for newly hired registered nurses
	Nursing care is based on a nursing model, rather than a medical model
	Written, up-to-date nursing care plans for all patients
	Patient care assignments that foster continuity of care (ie, the same nurse cares for the patient from one day to the next)
Patient safety management (HSOPSC)	In this unit, we discuss ways to prevent errors from happening again
	We are given feedback about changes put into place based on event reports
	The actions of hospital management show that patient safety is a top priority

The Norwegian version of NWI-R was translated and tested according to acknowledged procedures for questionnaire modifications between cultures, and the instrument's reliability and construct validity were assessed in a pilot test.25 In a previous paper, we have described the exploratory factor analysis that identified a factor structure of NWI-R for use in a Norwegian context. If more than 50% of scores in the single items were missing, the scale scores were noted as missing. Internal consistency for the four subscales included in the present study ranged from 0.71 to 0.88 (Cronbach's α) in the reliability test. Internal consistency of the three items from HSOPSC was 0.72 (Cronbach's α).11

The sample

Based on recommendations for cut-off points for response rates, four hospitals were excluded because the response rate was below 40%.26 Additionally, one hospital that specialises in cancer care was excluded, and the final sample was 30 hospitals. Responses from 3556 nurses working in surgical or medical wards in these 30 hospitals were included as averaged scores for each hospital. The mean response rate for the present study was 58.9%. Hospitals were categorised into regional university hospitals (university hospitals with national and regional responsibilities), local university hospitals (university hospitals without national responsibilities) and local hospitals (without university functions) that made the reference value in analyses.

Statistical analysis

The complex parameter and nurse-assessed quality of nursing and patient safety were transformed into a scale of 0–100, where high scores represent positive descriptions. Descriptive statistics for the outcome measures in this study are presented as means, 95% CIs, minimum and maximum. Means and 95 CIs for the nurse-reported measures are described per hospital type. The nurse-assessed quality and safety measures and survival probabilities were included in a correlation analysis, from which the results are reported as Pearson's correlation coefficients and p values. The complex parameter and hospital type were included in a regression analysis to examine the associations with nurse-assessed quality and safety, and survival probabilities that are reported as regression coefficients and p values. Initial univariate analyses were not performed to determine which variables to include in the logistic model, since the variables were included as subcomponents of a complex parameter based on findings from a previous paper.11 In the additional regression analysis, hospital-level nurse-to-patient ratios collected from Statistics Norway were included (number of nurse-years per 10 000 patient days).27

SPSS V.18.0 Software (SPSS, Chicago, Illinois, USA) for Windows was used for correlation analysis, and R 2.15.1, free software available at (http://www.r-project.org/), was used to conduct the multivariate stepwise regression (conditional backwards with the Akaike information criterion) on the hospital-level dataset.

Results

Survival probabilities varied between hospitals, and the hospital mean for overall survival was 95.1% (95% CI 94.8% to 95.4%, minimum=93.8%, maximum=96.2%). The hospital mean for survival AMI was 87.1% (95% CI 86.2% to 88.1%, minimum=83.6%, maximum=93.8%) and for survival stroke 85.5% (95% CI 84.4% to 86.7%, minimum=74.8%, maximum=89.1%). The mean score for nurses’ assessments of quality of nursing was 66.9 (95% CI 65.2 to 68.7, minimum=54.8, maximum=76.0). The mean score for patient safety was 60.3 (95% CI 58.9 to 61.8, minimum=48.5, maximum=67.4).

Most of the hospitals were categorised as local (n=22), but three hospitals were considered to be local university hospitals and further five were regional university hospitals. The variation in independent variables is presented per hospital type in table 2.

Table 2

Mean and CIs for organisational characteristics per hospital type

Hospital type		Staffing adequacy*	Nurse–physician relationship*	Quality system*	Patient safety management*	Nurse-to-patient ratio†
Local hospital (n=22)	Mean	43.9	68.0	52.0	55.1	51.3
	Lower 95% CI	40.5	66.2	49.3	52.9	47.2
	Upper 95% CI	47.3	69.7	54.7	57.3	55.1
Local university hospital (n=3)	Mean	42.0	65.1	53.1	53.7	53.2
	Lower 95% CI	28.1	57.9	48.3	47.1	41.6
	Upper 95% CI	55.9	72.3	57.8	60.2	64.8
Regional university hospital (n=5)	Mean	49.0	65.8	51.6	54.9	68.2
	Lower 95% CI	37.0	62.9	43.5	48.1	52.7
	Upper 95% CI	61.2	68.7	59.6	61.6	83.7

*The original 4- or 5-point Likert scales have been transformed into 0–100 scale, where high scores represent positive descriptions in this table.

†Number of nurse-years per 10 000 patient days.

The bivariate correlation analysis showed that nurse-assessed quality of nursing was correlated with nurse-assessed patient safety, overall survival and survival after AMI (table 3).

Table 3

Correlation coefficients between nurse-assessed outcomes and estimated survival probabilities

	Survival after AMI	Survival after stroke	Overall survival	Quality of nursing	Patient safety
	Pearson's correlation (p value)
Survival after AMI	–
Survival after stroke	0.01 (0.960)	–
Overall survival	0.56 (0.001)	0.30 (0.106)	–
Quality of nursing	0.40 (0.028)	0.00 (0.990)	0.49 (0.006)	–
Patient safety	0.30 (0.114)	0.02 (0.918)	0.32 (0.088)	0.82 (<0.001)	–

AMI, acute myocardial infarction.

Nurses’ assessments of patient safety were not correlated with the survival probabilities, and survival after stroke was not correlated with any of the other measures (table 3).

The multivariate stepwise regression analysis showed that nurses’ assessments of staffing adequacy were positively associated with overall survival, while there was a negative association between patient safety management and overall survival (table 4). A positive perception of staffing adequacy and regional university hospitals was associated with higher survival after AMI. There was no association between the organisational measures and survival after stroke. Staffing adequacy was associated with the two nurse-assessed outcomes, and patient safety management was associated with patient safety (table 4). The variables that remained significant at 5% level in the linear stepwise regression are presented in the table.

Table 4

Multivariate stepwise regression model (backwards conditional) showing associations between organisational characteristics and nurse-assessed outcomes and estimated survival probabilities

	Overall survival	Survival after AMI	Survival after stroke	Quality of nursing	Patient safety
	Regression coefficient (p value)
Patient safety management	–0.09 (0.048)*	–	–	–	0.36 (0.011)*
Staffing adequacy	0.09 (0.002)**	0.08 (0.079)	–	0.44 (<0.001)**	0.24 (0.005)**
Nurse–physician relationship	–	–	–	–	–
Quality system	–	–	–	–	–
Local university hospital†	–	1.10 (0.346)	–	–	–
Regional university hospital†	–	4.05 (<0.001)**	–	–	–
Adjusted R2	0.30	0.45	–	0.57	0.79

*Significant at the 0.01% level.

**Significant at the 0.05% level.

†Reference value is all other hospitals.

AMI, acute myocardial infarction.

Nurse-reported staffing adequacy was associated with three of the outcome measures, and so we repeated the model analysis and included nurse-to-patient ratios based on figures from official statistics (ie, number of nurse man-years per 10 000 patient days, in 2009).27 This did not result in significant changes.

Discussion

In the present study, we found that overall survival is correlated with nurse-assessed quality of nursing. Furthermore, one of the diagnosis-specific survival measures, survival after AMI, is correlated with overall survival and quality of nursing. The subjective impression of staffing adequacy was correlated with overall survival, whereas the nurse-to-patient ratio was not associated with survival probabilities.

Hospitals are hierarchical systems consisting of wards that represent microsystems in which interactions (or care delivery) take place between healthcare workers and patients. According to Nelson et al,28 these microsystems are ‘essential building blocks that combine to form the care continuum’. In the present study, quality of care at the microsystem level was reflected in nurses’ assessments of quality and, to some extent, nurses’ assessments of patient safety, whereas quality of care at the hospital level was reflected in survival probabilities. Nurses’ assessments of quality of nursing in their wards are strongly correlated with overall survival, while their assessments of patient safety are not. The presence of a relationship between quality of nursing and overall survival is supported by a similar study, from whose results it was concluded that staff feedback is weakly associated with HSMR.20 The question regarding patient safety called for the nurses’ perceptions at the department level, which is one level above their ward level (a department in Norwegian hospitals most often consists of several wards). The department is a more distant entity, and the question about patient safety may reflect the nurses’ judgement of the patient safety systems administered by people outside their own ward and their own microsystem.29 30 This may also be one way to understand why patient safety management was negatively associated with overall survival. Another way to understand this odd result is that patient safety may have had a high priority in the wards where staffing was perceived as adequate. In wards where the attention towards patient safety is high, the attitudes towards safety and quality among the respondents might have been more conscious and, therefore, more critical. Moreover, the cross-sectional design of this study did not allow adjustments for variations over time. If interventions prior to the nurse survey changed the respondents’ perceptions of the work environment, this may not have had impact on the survival probabilities that are based on data from a long period of time.

We also examined the survival of two diagnosis groups: AMI and stroke. Survival after AMI was correlated with overall survival and with nurses’ assessments of quality of care. The different processes implemented for the treatment of AMI and stroke may explain this finding. Survival after stroke is dependent on acute functions in primary healthcare, a multifaceted intervention within specialist healthcare and a good rehabilitation plan in primary healthcare after discharge from hospital. A crucial factor is the time from symptom onset to medical treatment and admission to hospital. The accessibility of healthcare, distance from the hospital, and patient paths within and between the hospitals are perhaps more important organisational aspects for survival after stroke. The management of AMI patients was revised in Norway from 2000, and standardised procedures are now followed for the acute care provided by hospitals managing these patients. This is reflected in better survival data for AMI in a comparative national survey, as recorded in the Organization for Economic Co-operation and Development's document ‘Health at a Glance’.31 It may not be surprising that nurses assess the quality of care as being high in those hospitals with well-functioning acute care. However, a limitation of the present study is that the nurses’ assessments of quality of care was not broken down and stratified according to AMI and stroke wards. Therefore, it was not possible to examine the correlation between quality of care and survival at the ward level. Another limitation of the present study is that responses from nurses working in critical care units in which these patients may be cared for at their most vulnerable state, was not included.

The finding that survival after AMI was higher in regional university hospitals than in the other hospital types may indicate that the former are pioneers at improving the processes of care, which consequently reduces mortality.32 The introduction of percutaneous coronary intervention in regional university hospitals during the period studied may have had a major effect on survival for patients with AMI. Higher volume, higher personnel ratios and other structural features of teaching hospitals may also explain the better outcomes. However, we did not find the association between the nurse-to-patient ratio and survival that has been found in other studies.5 33–35 Hence, the association between nurse-reported staffing adequacy and overall survival indicates that the way resources are organised impacts patient outcomes. It is therefore important to emphasise that nurse-reported staffing adequacy is a composite variable that reflects factors additional to simply the number of nurses. This finding is consistent with another study that showed that nurse-reported staffing adequacy is not correlated with the nurse-to-patient ratio.36 On the other hand, it has been shown that nurse-reported staffing adequacy is associated with the accessibility of assistive personnel,37 and qualitative interviews indicate that the content of staffing adequacy includes how nurses assess the personnel mix, the cohesiveness of the staff, the delivery systems and ‘knowing’ the patients.38 In a recent literature review, perceived staffing adequacy was associated with clinical outcomes such as mortality, failure to rescue and adverse events.17 From our perspective, together these results emphasise the importance of having organisational processes that facilitate the quality of care and patient safety. This is particularly important considering our findings indicating that the way personnel resources are organised and allocated is more vital for quality and safety than the number of personnel. However, further research is required on what makes healthcare personnel perceive staffing as adequate.

Several studies have found associations between clinical outcomes and other organisational process measures, such as the nurse–physician relationship, the general work environment and educational level. 18 34 39–40 The finding that staffing adequacy was the only process measure of significance in the present study may be attributable to several factors. First, we used a selection of organisational process measures identified in a previous study; the measures used in other studies may not be compatible with our complex parameter. Second, other studies used patients as units of analysis; our ecological approach provides another perspective and involves a relatively small number of cases. Third, the outcome measures in the present study, survival probabilities, were weighted for hospital length of stay, and followed the cases after hospital discharge. This approach differs from similar studies using in-hospital mortality, or mortality during 30 days in hospital.

Strengths and limitations

The use of mortality measures for comparison purposes has been criticised because the different ways to calculate mortality may provide ‘large variation in hospital ranking and outlier detection’.21 Transfer of patients between hospitals might explain this problem, and the survival probabilities used in the present study have been recommended as indicators for hospital performance.21 Another strength of our study is the comprehensiveness of the nurse survey, for which there was a response rate of 58.9%, and which involved the majority of Norwegian general hospitals. A comparative study has shown that nurses’ ratings of their work environment, staffing levels, educational level and skill mix are generally high in Norway compared with other countries.41 This makes Norway a unique arena in which to explore the issues of patient safety, because the findings may point to organisational properties that are crucial for patient safety, and that may be concealed in other contexts.

Ecological studies are criticised because data at the individual level are aggregated to the group level. However, as the intention of this study was to compare hospital performance including hospital-level data, the ecological design was necessary.

The survey method is a cost-effective way of investigating the organisational properties of healthcare organisations, and the subjective description of quality made by nurses appears to be a valid measure.19 20 However, the healthcare workers’ reports on quality and safety may be biased for several reasons, and the discrepancy between healthcare workers’ assessments and clinical outcomes may be great.17 Still, nurse-assessed quality has been used to describe the role of nursing in hospitals, and to identify properties of high-performing work systems. Our study may serve as a support for studies using healthcare workers assessments as outcome measures to examine organisational properties that promote quality of care and patient safety.

Conclusion

In this study, perceived staffing adequacy, but not nurse-patient ratio was associated with overall survival. This result suggests that the way resources are organised is a critical factor to achieve the goal of quality and safety. Furthermore, nurses’ assessments of quality of nursing were correlated with overall survival and survival after AMI, suggesting that the way nurses characterise the microsystems they belong to also reflects the general performance of hospitals.