Safety climate in Swiss hospital units: Swiss version of the Safety Climate Survey.

RATIONALE, AIMS AND
OBJECTIVES: Safety climate measurements are a broadly used element of improvement initiatives. In order to provide a sound and easy-to-administer instrument for the use in Swiss hospitals, we translated the Safety Climate Survey into German and French.
METHODS: After translating the Safety Climate Survey into French and German, a cross-sectional survey study was conducted with health care professionals (HCPs) in operating room (OR) teams and on OR-related wards in 10 Swiss hospitals. Validity of the instrument was examined by means of Cronbach's alpha and missing rates of the single items. Item-descriptive statistics group differences and percentage of 'problematic responses' (PPR) were calculated.
RESULTS: 3153 HCPs completed the survey (response rate: 63.4%). 1308 individuals were excluded from the analyses because of a profession other than doctor or nurse or invalid answers (n = 1845; nurses = 1321, doctors = 523). Internal consistency of the translated Safety Climate Survey was good (Cronbach's alpha German  = 0.86; Cronbach's alpha French  = 0.84). Missing rates at item level were rather low (0.23-4.3%). We found significant group differences in safety climate values regarding profession, managerial function, work area and time spent in direct patient care. At item level, 14 out of 21 items showed a PPR higher than 10%.
CONCLUSIONS: Results indicate that the French and German translations of the Safety Climate Survey might be a useful measurement instrument for safety climate in Swiss hospital units. Analyses at item level allow for differentiating facets of safety climate into more positive and critical safety climate aspects.

Introduction

The measurement of patient safety climate, that is, ‘the measurable components of safety culture’ [1], is a broadly used element in evaluating improvement initiatives and conducting quality assessments in health care. This trend bases on the assumption that a high level of safety climate builds the basis for providing safe care to patients. Accordingly, one important area of research focuses on the development of measurement instruments of safety climate as well as research on the relationship between safety climate and patient outcomes, which draws a heterogeneous picture of results so far [2, 3].

Even though there are different ways of measuring safety climate (e.g. interviewing, observation), it is mainly done by means of standardized questionnaires. Many instruments have been developed to measure safety climate recently [4, 5, 6]. The instruments differ regarding the dimensions of safety climate considered, the area of application [e.g. primary care, hospital, operating room (OR)], length, level of validation and national context of development or validation of translations.

Among the instruments broadly known and tested [7] are the ‘Hospital Survey on Patient Safety Culture’ (HSoPS) [8], the ‘Safety Attitudes Questionnaire’ (SAQ) [9], and the ‘Safety Climate Survey’ (SCS) [10, 11]. The HSoPS and SAQ are probably the most frequently used questionnaires on an international level. Comparing the instruments, the SCS is shortest in length and thus takes less time to complete. Moreover, the SCS is a one‐dimensional scale in contrast to the others. This might be helpful for the purpose of cross‐cultural transfer, as validation studies of questionnaire translations frequently report differences in dimensional structure between the original instruments and translations [12, 13, 14] or exclude items from factor analysis when replicating questionnaire dimensions [15]. Such results raise the difficult question whether discrepancies between language versions are a matter of cultural differences, of translation problems or of both. Besides shortness and one dimensionality, the Safety Climate Survey shows good psychometric properties [10, 11, 16]. For these reasons, we decided to use the Safety Climate Survey – an instrument recommended by the Institute for Healthcare Improvement – for the purpose of our project: providing a sound and easy‐to‐administer measurement instrument of safety climate for the use in Swiss hospitals.

In this study, we describe the results of a validation study applying the Safety Climate Survey to the context of Swiss hospital units. We provide a French and German version of the Safety Climate Survey, which was missing so far. Furthermore, we present results at item and scale level of a multisite use of the instrument surveying members of OR teams and staff on OR‐related wards. Based on prior research [16, 17, 18, 19], we analysed group differences and hypothesized that doctors show more positive safety climate values than nurses and professionals with managerial function perceive a more positive safety climate than staff without. Significant differences based on work areas and time spent per week in direct patient contact were assumed without formulating a clear direction of group differences.

Methods

Participants and setting of the study

A cross‐sectional survey study was conducted in October/November 2013 in the context of a larger implementation intervention in 10 hospitals in French‐ and German‐speaking parts of Switzerland. The hospitals were selected for the purpose of their participation in a project to comprehensively implement the use of the World Health Organization (WHO) surgical checklist. The questionnaire was part of a larger data collection episode in which two separate questionnaires were distributed at the hospitals. Hospitals did participate with different numbers and types of surgical departments in the implementation intervention. Accordingly, the survey sample consisted of all members of the OR teams of the respective hospital departments (doctors, nurses, nurses with special education in anaesthesia nursing or intensive care, attendants for surgical positioning and surgical technicians) and ward staff (doctors, nurses, nursing assistants and further professionals who were subsumed under ‘others’) involved in the pre‐ and post‐operative care of surgical patients care process of OR patients. The sample included 3153 individuals. For the purposes of this analysis, only doctors and nurses were included.

Translation

The original version of the Safety Climate Survey was translated by a professional translator from English to German and back‐translated to English by a second translator. Differences in translation and back‐translation were discussed and resolved within the research team. In cases that showed to be difficult to resolve, the translators were consulted to find an appropriate solution. Two versions ‘OR’ and ‘ward’ were developed that differed in the wording of single items, which referred to the specific working area, that is, OR or ward. The translated Safety Climate Survey was tested in a sample of 60 health care professionals (HCPs) from non‐participating hospitals. Only minor modifications followed from this pre‐test mainly referring to the socio‐demographic variables added to the survey. The pre‐test also revealed that completing the questionnaire took approximately 12 minutes including seven additional pre‐test questions on the questionnaire itself. Thus, completing the Safety Climate Survey and some socio‐demographic questions should not exceed 10 minutes.

In a second translation step, the finalized German version of the Safety Climate Survey was translated to French and back‐translated by two independent translators. Two bilingual experts from healthcare and research compared the resulting French version with the German version of the survey. Corrections to the French translation were made to match closest the original English questionnaire items.

Survey instrument

The questionnaire consisted of the 19 items of the Safety Climate Survey, to be rated on a 5‐point Likert‐scale from 1 = ‘disagree strongly’ to 5 = ‘agree strongly’. The category ‘not applicable’, which is presented in the original instrument, was not included. In addition, participants were asked to answer some questions on their personal and professional background (age, gender, training, years in practice, managerial function).

Data collection

A print version of the questionnaire was sent to the hospitals and distributed locally to the HCPs included in the sample. HCPs were invited to participate by the local project leaders at each hospital and repeatedly reminded to participate throughout the data‐collecting period. HCPs received the questionnaire together with a prepaid return envelope to send the questionnaires back individually.

Data analyses

Negatively worded items were recoded to insure that higher scores indicated a more positive assessment of safety climate for every item. Internal consistency of the instrument was tested by means of Cronbach's alpha and compared with the results published by other authors. Alpha values above 0.7 denote an acceptable scale. Besides the examination of Cronbach's alpha, the French and German questionnaire versions have been analysed together for data analyses.

On item level, we calculated a percentage of ‘problematic response’ (PPR) following the approach of Singer et al. [20]. PPR refers to the number of individuals that scored low on the respective item, indicating a low level of reported safety climate. Negative answers, that is, answers ≤ 2 on the 5‐point Likert scale were treated as ‘problematic’ response. Accordingly, ‘a low PPR is indicative of a high safety climate’ [18]. Furthermore, a PPR higher than 10% is assumed to be inconsistent with an optimal level of safety climate within an organization, which points to a need of enhancing safety climate.

On scale level, group differences between individuals of different profession, managerial function, and work areas were examined using a t‐test. Time spent per week in direct patient contact was examined using analysis of variance. Statistical tests were two‐sided and considered significant at the P < 0.05 level. Effect sizes were calculated. Analyses were conducted using STATA v13.1 [21].

Results

3153 HCPs completed the survey, leading to a response rate of 63.4%. 1308 individuals were excluded from the analyses of this study because of a profession other than doctor or nurse (n = 619), missing values and invalid answers on any of the items (n = 689), leaving a final sample of 1845 (523 doctors and 1321 nurses).

Sample characteristics are presented in Table 1. The sample consisted mainly of nurses (72% of the sample) without managerial function (71% of the sample). 37% of the participating HCPs worked in an OR team, 63% on ward.

Table 1

Sample characteristics (n = 1845)

Characteristic	n	%
Survey language
German	1239	67
French	606	33
Gender
Male	535	29
Female	1300	71
Mean age in years	39 (10.36)
Work area
OR	688	37
Ward	1156	63
Profession
Doctor	523	28
Nurse	1322	72
Managerial function
Yes	526	29
No	1319	71
Years of professional experience
0–2	215	12
2–5	306	17
5–10	401	22
10–20	469	25
More than 20	453	25
Hours of direct patient care per week
0	295	16
0–8	240	13
8–16	236	13
16–24	271	15
24–32	276	15
32–40	310	17
More than 40	217	12

The translated Safety Climate Survey showed good internal consistency: Cronbach's alphaTotal = 0.85 (95% CI: 0.84–0.86); Cronbach's alphaGerman = 0.86 (95% CI: 0.85–0.88); Cronbach's alphaFrench = 0.84 (CI: 0.82–0.86). Mean differences on item level between language versions were found to be significant for some items.

Table 2 shows mean scores, standard deviations and PPR at item and scale level. Missing values for the individual items are also presented in this table. Missing values at item level ranged from 0.23% (‘Patient safety is constantly reinforced as the priority in this clinical area’) to 4.3% (‘This institution is doing more for patient safety now than it did 1 year ago’), with a mean missing rate of 0.94%.

Table 2

Mean, standard deviation, missings and PPR at item and scale level (n = 1845)

	Mean	SD	PPR (%)	Missings (%)
Safety Climate Scale	3.8	0.53	11.76
Items
(1) The culture of this clinical area makes it easy to learn from the mistakes of others.	3.78	0.92	12.09	0.68
(2) Medical errors* are handled appropriately in this clinical area.	3.77	0.94	12.90	0.68
(3) The senior leaders in my hospital listen to me and care about my concerns.	3.86	0.92	10.24	0.36
(4) The doctor and nurse leaders in my area listen to me and care about my concerns.	3.73	0.99	14.25	0.41
(5) Leadership is driving us to be a safety‐ centered institution.	3.57	0.97	14.42	0.72
(6) My suggestions about safety would be acted upon if I expressed them to management.	3.80	0.94	9.49	0.86
(7) Management/leadership does not knowingly compromise safety concerns for productivity.	3.53	1.19	21.84	1.54
(8) I am encouraged by my colleagues to report any safety concerns I may have.	3.95	0.86	6.45	0.27
(9) I know the proper channels to direct questions regarding patient safety.	3.83	0.97	11.98	0.90
(10) I receive appropriate feedback about my performance.	3.57	1.02	17.29	0.63
(11) I would feel safe being treated here as a patient.	3.72	0.95	13.12	0.77
(12) Briefing personnel before the start of a shift (i.e. to plan for possible contingencies) is an important part of patient safety.	4.38	0.84	4.50	0.59
(13) Briefings are common here.	3.65	1.14	18.64	1.31
(14) I am satisfied with the availability of clinical leadership.
(14a) Doctor	3.78	1.03	14.91	0.59
(14b) Nursing	4.15	0.78	4.28	1.09
(14c) Pharmacy	3.85	0.90	8.13	2.26
(15) This institution is doing more for patient safety now than it did 1 year ago.	3.41	0.95	14.31	4.30
(16) I believe that most adverse events occur as a result of multiple system failures and are not attributable to one individual's actions.	3.18	1.01	28.40	0.77
(17) The personnel in this clinical area take responsibility for patient safety.	4.18	0.78	3.25	0.32
(18) Personnel frequently disregard rules or guidelines that are established for this clinical area.	3.92	1.01	13.01	0.36
(19) Patient safety is constantly reinforced as the priority in this clinical area.	4.11	0.89	7.64	0.23

Note. PPR (percentage of ‘problematic responses’) refers to the number of individuals that scored low on the respective item, that is, answered ≤ 2 on the 5‐point Likert‐scale.

*Medical error is defined as any mistake in the delivery of care, by any health care professional, regardless of outcome.

The mean value of the Safety Climate Survey was 3.8 (SD = 0.53). At item level means ranged from 3.18 to 4.38. The two items with the lowest scores are ‘I believe that most adverse events occur as a result of multiple system failures and are not attributable to one individual's action’ (3.18, SD = 1.01) and ‘This institution is doing more for patient safety now than it did 1 year ago’ (3.41, SD = 0.95). Items showing the highest ratings are ‘Briefing personnel before the start of a shift is an important part of patient safety’ (4.38, SD = 0.84) and ‘The personnel in this clinical area take responsibility for patient safety’ (4.18, SD = 0.78). Comparing the mean scores at item level showed that items referring to the institutional level/hospital leadership (items no 5, 7, 15) were rated more negatively than items referring to the safety climate in the individual's work area, that is, ward/OR (items no 17, 19). Furthermore, there was a clear discrepancy between the importance attached to briefings as part of patient safety (item no 12: mean = 4.38) and the actual practice of briefings in daily work (item no 13: mean = 3.65).

The PPR of the Safety Climate Survey was 11.75%, that is, nearly 12% of the HCPs reported a ‘problematic safety climate’. At item level, 14 out of 21 items showed a PPR higher than 10%, and two items returned a PPR higher than 20% (‘Management/leadership does not knowingly compromise safety concerns for productivity’, ‘I believe that most adverse events occur as a result of multiple system failures and are not attributable to one individual's action’).

Group differences have been analysed at scale level and are presented in Table 3. Doctors scored significantly higher compared with nurses and staff with managerial function scored higher compared with staff without. Moreover, HCPs working in the OR reported significantly higher values compared with staff working on wards. Comparing HCPs regarding the time they spend with patients also shows significant differences. Staff with no patient contact and staff with more than 40 hours per week showed lower values compared with staff with 1 to 40 hours worked in direct patient care.

Table 3

Group difference in mean scores of the Safety Climate Scale

	Mean	n	Effect sizes*	P‐value
Profession			0.35
Nurse	3.75	1322		<0.001
Doctor	3.93	523
Managerial function			0.26
Yes	3.90	526		0.001
No	3.77	1319
Work area			0.12	0.01
OR	3.84	688
Ward	3.78	1157
Work with patients (hours per week)			0.01	0.001
0	3.75	295
1–less than 16	3.88	476
16–less than 40	3.79	856
40 and more	3.77	218

*We report Cohen's d for differences between two groups and eta‐squared for ‘work with patients (hours per week)’.

The analysis of group differences at item level showed that there is a limited number of items showing larger differences and thus underlying the observed group differences at scale level. Depending on the group difference in focus (profession, managerial function, work area), there is a varying pattern of items with largest differences observable. However, one item showed to be of relevance in all analysed group differences at item level, that is, item 14a. In reference to the estimates of doctors and nurses, the largest differences could be found in item 4 (nurses = 3.6; doctors = 4.07), item 7 (nurses = 3.35; doctors = 3.99), item 11 (nurses = 3.61; doctors = 4) and item 14a (nurses = 3.54; doctors = 4.4), Comparing staff with and without managerial function showed largest differences regarding the following items: item 7 (with = 3.71, without = 3.46), item 9 (with = 4.07; without = 3.73), item 14a (with = 3.97, without = 3.71) and item 15 (with = 3.59, without = 3.34). Referring to the work area of HCPs item 10 (OR = 3.38; ward = 3.68), item 14a (OR = 4.17, ward = 3.56), item 12 (OR = 4.7; ward = 4.19) and item 13 (OR = 3.8; ward = 3.56) showed larger differences than the other scale items.

Discussion

The goal of this study was to develop French and German language versions of the Safety Climate Survey and to apply the survey to Swiss hospital units. As far as we know, it is the first time the Safety Climate Survey was used in a sample of HCPs in Switzerland.

Swiss version of the Safety Climate Survey

Overall, the Safety Climate Survey proved to be a valuable measurement instrument of safety climate. For both language versions the instrument demonstrates a good internal consistency. Cronbach's alpha (αTotal = 0.85; αGerman = 0.86; αFrench = 0.84) corresponds to those reported by other authors (Kho et al., Cronbach's alpha = 0.86) [16].

Furthermore, the items showed low rates of missing values indicating that the survey questions are easy to answer. Adding the shortness of the instrument and the information that time to fill in the questionnaire should not exceed 10 minutes, we provide a reasonable and easy‐to‐administer measurement instrument of safety climate to be applied in Swiss hospitals.

Group differences

Group differences found in our study confirm results from other studies. Doctors assessed safety climate more positive than nurses and staff with managerial function rated safety climate in a more positive way than staff without [16, 18, 20]. This pattern seems to be consistent across different countries and different survey instruments. We also found that staff working in ORs reported a more positive safety climate than staff from wards and that the assessments differ in correspondence with the time HCPs spend in direct patient care. These group differences emphasize the importance to consider and to differentiate staff groups when analysing safety climate and planning activities to improve patient safety. However, in order to identify the specific aspects that are of relevance for the different staff groups, analysis at item level might be valuable. As our results show, depending on the specific group differences, a different set of items was underlying the observed mean differences at scale level. Further research might be valuable that systematically analyses group differences at item level in order to find characteristic item sets that are of relevance for specific staff groups. Moreover, as we found stronger differences in mean values and PPR at item level than at scale level, the detailed analyses at item levels seems a valuable approach also from this point of view.

Mean values and PPR

Having a closer look at means and PPR at item level, the results revealed differences in values that are worth discussing. Theory based on high reliability organizations suggests that even a small number of staff that is not committed to safety may result in higher risk in the organization [18, 22]. For this reason, earlier research defined a rate of 10% problematic responses (PPR) as a cut‐off to differentiate between high reliability organizations with clear need for improvement on safety climate from others. In our study, two‐third of the items showed PPR above 10%, 2 of those items had a PPR above 20%. Items with highest PPR and thus indicating a low level of safety climate tend to refer to the leadership of the hospital (‘Leadership is driving us to be a safety centred institution’, ‘Management/leadership does not knowingly compromise safety concerns for productivity’). On the other hand, items with PPR below 10%, indicating a positive safety climate, tend to be related to the direct work environment of respondents. Respective items, for example, refer to colleagues (‘I am encouraged by my colleagues to report any safety concerns I may have’) or to respondents' work area (‘Patient safety is constantly reinforced as the priority in this clinical area’). This tendency is consistent with mean values at item level. Items referring to respondents' area of work show a more positive safety climate. In sum, HCPs in our sample tended to rate the local safety climate more positive than safety climate at the level of the organization/management. The high PPR of items like ‘Leadership is driving us to be a safety‐centred institution’ and ‘Management/leadership does not knowingly compromise safety concerns for productivity’ might also be interpreted in a more political context. Diagnosis‐related groups have been introduced to Swiss hospital care just 3 years ago and consequences for quality and safety of care are still under strong discussion.

The result indicates that HCPs perceive and assess safety climate in a differentiated way. Depending on the area addressed (local ward/OR vs. institution) in the survey items and depending on the group of people answering the questions (e.g. nurses vs. doctors; HCPs with or without managerial function; HCPs from wards or OR teams) safety climate scores differ systematically. Accordingly, it seems essential to have a differentiated look at a hospital and its units when planning and implementing safety climate improvements.

One item (no. 16: ‘I believe that most adverse events occur as a result of multiple system failures and are not attributable to one individual's actions’) clearly showed the highest PPR. As the item is rated so different than most of the other items, the question arises whether there might be an additional reason for this result that is different from believes in individual responsibility for failure. As the formulation of the item is rather complicated respondents might had difficulties in understanding. The understanding of the item should be examined before applying the instrument the next time.

As said before, regarding the interpretation of group differences, the detailed analyses of results at item level seems a helpful way to work with the Safety Climate Survey. This is also visible in another finding. Items 12 and 13 both refer to team briefings. Results show that HCPs in our sample consider the relevance of team briefings generally quite high (item 12). This contrasts with a lower rate of actually conducted briefings on the wards and in the ORs (item 13). Interpreting the results at item level indicates that hospital management might think of promoting briefings in daily routine of the local units. Using the Safety Climate Survey only by means of calculating a single overall safety climate score would not disclose such details.

Limitations

The study was conducted in the context of a broader implementation intervention on the WHO surgical checklist. This resulted in a study sample of OR teams and staff from surgical wards. Even though different departments were involved and the participating hospitals represented a broad range of institutions, our sample is not representative and further research may examine the application of the translated Safety Climate Survey in different clinical areas and health care institutions. Furthermore, as we have no information about HCPs that did not return the questionnaire, we are not able to discuss any possible non‐responder bias.

Other limitations relate to the gathering of socio‐demographic information that was used to analyse group differences. Participants assigned themselves to work in the OR area or on ward. There might be some work places that are hard to clearly assign to OR area or ward or there might be staff that works in both areas. This may lead to some inaccuracy in differentiating the two staff groups. Besides, participants rated themselves as being staff with or without managerial function without any guiding or definition provided in the questionnaire. Even though the term ‘managerial function’ is commonly used in Swiss hospitals, there might be some inaccuracy in this classification.

Another point to be critically discussed is the rather small differences in means, especially with regard to group differences at scale level. Although the group differences reported proved to be significant, absolute differences are very small and thus the clinical relevance for patient safety is questionable. However, as the observed differences are supported by other research, the interpretation of the group differences seems meaningful. Future research may address other group differences that might be of relevance to learn more about safety climate in hospital units. Examples might be the use of a surgical checklist, participation in quality activities or regularity of team meetings that showed to be important in other research [23]. Further research might also focus on the question of relevant effect sizes. For example, is there a threshold that must be reached in group differences in order to meaningfully interpret observed differences and to draw conclusions for patient safety strategies?

Finally, we did not collect any data on outcomes that could be related to the safety climate measurements. As research shows unclear results on this issue so far, it would be helpful to discuss data on safety climate in this context. In a next step, research may further examine the relationship of safety climate and outcome variables.

Conclusions

The results of our study suggest that the French and German translations of the Safety Climate Survey seem to be useful measurement instruments to assess safety climate in Swiss hospital units. The analyses of ratings at item level allow for differentiation of more positive and rather critical safety climate facets. Further use and test of the questionnaire might prove whether the Safety Climate Survey or single survey items serve as a valuable monitoring instrument accompanying improvement activities in Swiss hospitals. Moreover, further research should examine whether the instrument can be usefully linked to measurement of different outcome variables.

Conflict of interest

The authors declare no conflict of interest.

Ethical approval

This study was conducted as part of a quality improvement project. The study designed and the data collected did not require approval of an ethical committee in Switzerland referring to Article 1 and 2 of the Federal Act on Research involving Human Beings (Human Research Act, HRA).