Improving the culture of safety among healthcare workers: Integration of different instruments to gain major insights and drive effective changes.

The Safety Attitude Questionnaire (SAQ) and Manchester Patient Safety Framework (MaPSaF) are known as effective tools to assess patient safety culture and climate and develop targeted strategies. However, they are seldom applied in an integrated way. The aim of this study was to conduct an implementation project through a novel use of both instruments to gain unique insights. The Italian version of MaPSaF and SAQ were administered to 1,759 healthcare workers from three Italian hospitals (response rate: MaPSaF 70.5%, SAQ 61.6%). MaPSaF evaluation proved an overall bureaucratic level of patient safety culture. SAQ scores showed a predominance of neutral scores (75.99%). The dimension perception of management gained the lowest mean score (53.32), while Stress recognition obtained the highest (75.17). Safety climate perception differed significantly among groups: working in a small hospital, in a medical department, and being a physician were associated with the most positive results. The majority (67.1%) of responders to both MaPSaF and SAQ considered the two instruments as providing with different and complementary information. Overall, results showed that an integrated approach in the evaluation of an organisation's safety culture may result useful for an in-depth analysis of the criticalities and the adoption of appropriate improvement strategies.

INTRODUCTION

Accurate measurement of patient safety culture is the first step to define main critical issues and develop effective, ad hoc improvement strategies. The U.K. Health and Safety Commission defines safety culture as ‘the product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation's health and safety management’. An active collaboration between hospital management and healthcare workers plays a central role in improving safety culture. Over the past years, many authors have given strength to the concept that organisational culture is not unique but instead it consists of different ‘subcultures’. , Different authors delineated distinct ‘archetypes’, reflecting different assumptions regarding risk, ways in which accidents occur, and approaches to prevent them. , For Phipps and Ashcroft, these subcultures can coexist within a broader, overarching organisational culture. For example, different units within the same hospital could value patients' involvement in the therapeutic process in different ways, with significant implications in terms of risk management. Provonost and Sexton reiterated how safety culture should be considered a local phenomenon, finding that scores related to safety climate were more diverse between different units of the same hospital rather than at an inter‐hospital level. Overall, the culture of safety is the product of different forces operating on multiple levels.

Several authors posed a distinction between the concept of ‘safety culture’ and ‘safety climate’, although the two terms are often used interchangeably. , While safety culture refers to the organisation's norms and values, patient safety climate refers to workers' attitudes and perceptions of the level of priority given to safety, and is in fact defined as ‘the measure of the safety status perceived by members of an organisation in a specific place at a specific time’. , , On the basis of these assumptions, it can be stated that patient safety climate refers to the values and artefacts adopted by the organisational culture which are tangible and measurable, whereas the basic assumptions of culture are not as accessible.

The approach taken to measuring climate is generally to administer questionnaires, which are a key tool to assess workers' perceptions of safety and obtain reliable benchmarking data. Instruments measuring safety climate have spread extensively worldwide, one of the most successful being the Safety Attitude Questionnaire (SAQ) implemented by Sexton et al. The SAQ is one of the most widely used international security climate questionnaires as an informative and easy‐to‐use tool. , , , Another most popular tool is the Manchester Patient Safety Framework (MaPSaF), which was created to support healthcare organisations to reflect on their progress along the steps leading to the development of a ‘mature’ safety culture. It represents a valuable tool to facilitate reflections on safety culture, stimulate discussions about strengths and weaknesses, and motivate changes that are necessary to improve safety. , SAQ and MaPSaF have been indicated by the European Union Network for Patient Safety as two of the three most suitable and recommended tools (together with the Hospital Survey on Patient Safety Culture) to assess aspects of patient safety culture. ,

Numerous studies have shown a link between a good safety climate assessment and the adoption of safe behaviours by workers. , However, as they cannot tap into the deeper aspects of culture, these tools reflect an analytical approach to assessing safety culture, with all the interpretative limitations that this entails. An integrated evaluation involving the use of different tools and the adoption of diversified approaches could overcome this criticality. The authors of the questionnaires themselves emphasise that an integrated assessment using a combination of quantitative and qualitative tools is highly desirable for a better understanding of the underlying level of culture.

Despite evidences supporting the implementation of both SAQ and MaPSaF, up to our knowledge no studies have proposed their integrated use. The aim of this study is to present the first step of an improvement project conducted in three hospitals. The specific goals were to:

Collect data that could give evidence about the areas requiring priority interventions by the integrated use of Safety Attitude Questionnaire (SAQ) and Manchester Patient Safety Framework (MaPSaF);

Detect any subculture and the specific determinant(s) within the analysed context;

Evaluate pros and cons of the use of SAQ and MaPSaF as supporting tools for the entire process.

The rationale for the targeted combination of these two instruments lies not only in the diversity of the assessments carried out, but also in the purpose of their use. On the one hand, MaPSaF stimulates an awareness of problems through collective discussion; on the other hand, SAQ is an individual survey. A combined use can therefore create a ‘sense of urgency’ prior to the acquisition of specific information, and lead to the engagement of the energies needed to make changes. Providing objective information to conscious leaders can hopefully lead to more effective managerial action.

MATERIALS AND METHODS

Setting and population

This study was conducted in the Veneto Region, in north‐east Italy. The Italian National Health Service (INHS) is largely under the control of regional governments, which administer the territory through a subdivision into Local Health Authorities. , Each local authority plans and organises assistance and care for the population living in the area, supplying social and healthcare services and hospital care, by public and private providers. The present study was conducted in three primary hospitals all run by the same Local Health Authority (further indicated as hospital A, B and C). The three facilities accounted for 320, 222 and 80 beds, respectively; the total number of admissions in 2016 was 12,977, 11,877 and 772, respectively. The overall staff (N = 1,759) was invited to take part to the study.

Methods

The Italian version of the acute‐MaPSaF and SAQ were administered to the study population. The acute version of the MaPSaF was specifically developed to be used in the hospital setting (see Appendix). The framework includes 10 critical dimensions of patient safety: within each dimension, specific statements address five different levels of safety culture (pathological—reactive—bureaucratic—proactive—generative). Each level reflects the general attitude towards patient safety. Specifically, at the pathological level the investment in improving safety is very poor or absent; at the reactive, patient safety is taken into account only after an incident has occurred; bureaucratic organisations are very paper‐based and focus on accountability (e.g., giving evidence to regulatory agencies of their involvement in safety); finally, proactive and generative organisations put high efforts in improving safety and every member of the staff considers activities related to risk management as an integral part of everyday practice. The MaPSaF is generally used within a workshop conducted by an experienced facilitator who stimulates a discussion about patient safety culture and promotes self‐reflections through a comparison between the actual and desirable level of maturity.

We followed the World Health Organisation ‘Guidelines for instruments' translation and adaptation’ to develop the Italian version of MaPSaF. Specifically, the following steps were implemented:

A forward translation was performed by a health professional with experience in patient safety and safety culture (FM);

An experts panel (ST, MC, VN, and DV) reviewed the translation in order to check for inadequate expressions and recommend alternatives when appropriate;

An independent researcher translated the instrument back to English; discrepancies with the original version were discussed and solved by the experts panel;

The final version of the instrument was tested on a small sample of healthcare workers in order to verify the clarity and feasibility of the instrument (pre‐testing and cognitive interviewing).

The SAQ is a self‐assessment tool developed to measure employees' perceptions and attitudes toward safety (i.e., patient safety climate) in health organisations. The Italian cross‐culturally validated version of SAQ short form showed good psychometric properties. SAQ short form includes 36 items where respondents rate their agreement over a 5‐point Likert scale: ‘strongly disagree’ (1), ‘moderately disagree’ (2), ‘neutral’ (3), ‘moderately agree’ (4) and ‘strongly agree’ (5). The Italian version of SAQ confirmed the 6 safety dimensions identified from the original version, that is Teamwork climate, Safety climate, Job satisfaction (positive attitude about the work experience), Stress recognition, Perceptions of management (medical directorate of each hospital and medical direction of each department), and Working conditions. Five items were not related to any dimension and needed to be independently evaluated. The SAQ allows to measure safety climate both as a total score and per dimension/item. Score calculation took into account that three items (2, 11 and 36) are negatively worded (i.e., the higher the degree of agreement, the lower the perception of safety). Personal information and job‐related information were collected, including gender, facility, department, position and length of working experience.

Data collection

Professionals in head positions (i.e., medical directors, chief physicians, and nurse coordinators) from the three hospitals were invited to take part to a workshop at the beginning of the project. An expert in risk management (ST) made a presentation on safety culture and its measurements. Specifically, both SAQ and MaPSaF were extensively described in their aims and employment. Workshop attenders were then asked to individually rate the level of safety culture maturity of their hospital by choosing the level that best described each dimension on the MaPSaF evaluation sheet (see Appendix). A brief small‐group discussion followed to compare results from individual assessments. An experienced facilitator (FM) guided the plenary discussion, in order to raise self‐awareness about the need for changes and actively involve leaders in safety culture assessment with SAQ. Professionals were then provided with the necessary instructions and resources (e.g., information sheet and other survey materials) to organise SAQ distribution to the whole staff. SAQ questionnaires were distributed (on paper form) and 45 days were given for completion. At the end of the survey, professionals who had taken part to the introductory workshop were given a report providing with results and suggested actions, both for the hospitals and the services. Finally, they were asked to complete a questionnaire aiming to assess and compare MaPSaF framework and SAQ tool according to a set of aspects and to evaluate the advantages of their integrated use. This tool was developed by a group of experts (ST, FM, MC, DV) through a first step of identification of the important elements for questionnaires on safety culture and climate through literature research, and then with the development of a questionnaire. The aspects investigated the quality of data provided by the SAQ and MaPSaF questionnaires, the quality of information and opinions in the feedback regarding the SAQ/MaPSaF data, preferences on the type of instrument, opinions on the representativeness of the results, the purpose of the evaluation and the implementation of improvement actions. The assessment methods included scoring on a 1–10 scale, multiple‐choice and open answers.

Data analysis

The level of safety culture maturity in the hospitals was assessed by calculating the proportion of answers to the MaPSaF by level (A—pathological, B—reactive, C—bureaucratic, D—proactive, E − generative). A sub‐analysis for each dimension was also performed.

In SAQ data analysis, 100‐point scale scores were calculated according to the following formula :

Negatively worded items were reversed. Accordingly, the percentage of negative responses (PNRs; i.e., 0–2 on Likert scale or ≤25 on the 100‐point scale) and percentage of positive responses (PPRs; i.e., 4–5 on Likert scale or ≥75 on the 100‐point scale) were calculated both in total, per dimension and per item (excluding non‐responders). According to the literature, the percentage of sufficient responses (PSRs) was considered as ≥60 on the 100‐point scale and the percentage of target responses (useful in interpreting the results and defining priorities for action) (PTRs) as ≥80 on the 100‐point scale.

In order to assess the impact of different variables on safety culture, a set of bivariate statistical analyses was applied. In particular, differences were analysed both on overall and dimension scores on the 0–100 scale. The variables investigated were gender, facility, place of work, occupation and length of working experience. t‐test and one‐way ANOVA test were used respectively for two or more groups. After excluding collinearity, multivariate analysis (regression linear analysis) was performed (both on overall and dimension scores) including variables that resulted significant from univariate analysis. The level of statistical significance was set at 0.05. Stata 13 software (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp. LP) was used to perform the analyses.

RESULTS

Socio‐demographic data

The majority (67/95, 70.5%) of senior management figures returned the completed MaPSaF sheet. Among these, 47.8% were nurse coordinators and chief technicians, 41.8% were physicians and 10.4% did not specify their role.

A total of 1,083 SAQ questionnaires (response rate = 61.6%) were returned complete by the personnel. Responders were distributed as follows: 60.3% nurses, 14.36% physicians, 12.57% nurse aid personnel and 9.9% technicians (2.87% others).

MaPSaF assessment results

Figure 1 reports the level of maturity of the three facilities as a whole. Score distribution for each dimension showed the highest percentages of negative scores (levels A and B) for dimensions 6—Learning and continuous improvement (28.36%), 9—Staff education and training, and 10—Team working (both 29.85%). The highest percentages of positive scores (levels D and E) were detected for dimensions 2—Priority given to safety (23.89%), 3—System errors and individual responsibility (26.87%), and 4—Recording incidents and best practice (23.88%) (Figure 2).

FIGURE 1

Level of safety culture maturity in the hospitals assessed by calculating the proportion of answers to the MaPSaF by level (A—pathological, B—reactive, C—bureaucratic, D—proactive, E—generative). Abbreviation: MaPSaF, Manchester Patient Safety Framework

FIGURE 2

Sub‐analysis for each MaPSaF dimension (see Figure 1 legend for MaPSaF levels). Abbreviation: MaPSaF, Manchester Patient Safety Framework

SAQ assessment results

Table 1 reports overall scores distribution. Appendix displays the results per item and additional overall scores. The SAQ responders' group was characterised by a predominance of female and nurses. Professionals with over 21 years of experience accounted for 41.9% of responders, reflecting the ageing workforce of INHS. The percentage of missing responses varied from 0.74% (item 21—I am less effective at work when fatigued) to 19.11% (item 27—Problems are dealt with constructively by our hospital management), with a mean of 3.1 ± 4.2 missing responses per questionnaire. Items exploring Perception of management (items 24–28) showed the highest percentage of missing responses (range: 12.83%–19.11%). Four items obtained a PPRs ≥ 80%, that is I like my job (92.19%), I am less effective at work when fatigued (80%), I am more likely to make errors in tense or hostile situations (81.29%) and I experience good collaboration with nurses in this clinical area (82.66%). The four items with the worst scores (PNRs > 40%) were In this clinical area, it is difficult to discuss errors (43.55%), Hospital management supports my daily efforts (50.81%), The levels of staffing in this clinical area are sufficient to handle the number of patients (56.65%) and This hospital does a good job of training new personnel (41.61%).

TABLE 1

Results of SAQ assessment on a 100‐point scale. Data are presented according to the following thresholds: PNRs = score ≤ 25; PPRs = score ≥ 75; PSRs = score ≥ 60; PTRs = score ≥ 80. Results for the dimension Perception of management are presented as overall for hospitals and departments

Dimension	N	Mean (SD)	PNRs (%)	Neutral scores (%)	PPRs (%)
Teamwork climate	1,082	64.58 (22.16)	4.07	63.50	32.43
Safety climate	1,083	62.87 (21.29)	4.06	68.61	27.33
Job satisfaction	1,080	66.84 (24.41)	6.30	54.44	39.26
Stress recognition	1,082	75.17 (24.37)	4.43	43.62	51.95
Perception of management	1,068	53.32 (23.38)	11.42	73.12	15.46
Working conditions	1,057	57.55 (25.01)	7.56	71.61	20.83
Total	1,083	62.03 (16.14)	1.38	75.99	22.63

Items that showed the lowest percentage of missing responses (item 21; item 20—When my workload becomes excessive, my performance is impaired, 0.74 and 0.92%, respectively) belong to the Stress recognition dimension and are associated with a very high PPRs (80% and 77.65%, respectively). Low percentages of missing responses were also recorded for item 11 (In this clinical area, it is difficult to discuss errors, 1.2%) and the whole Job satisfaction dimension (range: 1.75%–3.51%). In addition, item 15 (I like my job) and item 18 (I am proud to work in this clinical area) showed high PPRs (92.19% and 64.76%, respectively).

Subcultures and determinants

Table 2 reports the average scores (total and for each dimension) for the individual socio‐demographic variables. The average score between facilities differed significantly (p = 0.003). This gap was detected also on each of the individual dimensions. However, for the dimensions Job Satisfaction, Stress Recognition, Perception of management (department) and Working conditions differences were not statistically significant. The departments of Medical Specialties and Medical Direction and Services showed a significantly higher average score than other departments (67.12 vs. 64.91, respectively). The only exception was the Stress recognition area, in which the Departments of Medicine (79.45) and Cardiovascular (78.52) showed a more positive assessment.

TABLE 2

Safety Attitude Questionnaire (SAQ) average scores (per dimension and total) for the individual socio‐demographic variables. Results are expressed as mean (standard deviation). H = hospital; D = department

	Teamwork climate	Safety climate	Job satisfaction	Stress recognition	Perception of management (H)	Perception of management (D)	Working conditions	Total
Hospital
A	65.66 (22.83)	63.77 (21.63)	67.27 (25.48)	75.27 (24.45)	46.33 (23.92)	57.16 (27.22)	56.50 (25.03)	62.14 (16.45)
B	62.08 (21.78)	61.34 (20.61)	65.67 (23.90)	74.40 (25.60)	46.14 (23.75)	57.27 (25.34)	57.80 (24.89)	60.80 (15.53)
C	71.50 (18.96)	66.90 (19.30)	70.64 (20.30)	77.93 (21.08)	58.00 (25.63)	62.19 (26.21)	63.13 (23.50)	67.53 (14.95)
	p < 0.05	p < 0.05	p = 0.228	p = 0.509	p < 0.05	p = 0.325	p = 0.103	p < 0.05
Department a
Medical direction and services	69.18 (20.76)	63.94 (20.99)	70.47 (23.91)	75.33 (23.63)	50.91 (24.83)	60.92 (26.90)	56.81 (25.72)	64.91 (15.69)
Surgery	58.45 (23.20)	59.54 (23.19)	58.08 (26.34)	75.05 (23.18)	40.75 (23.53)	53.04 (27.21)	54.19 (24.12)	56.69 (16.74)
Medical specialties	74.01 (19.05)	68.82 (21.46)	75.63 (21.44)	66.70 (30.70)	51.87 (21.63)	65.01 (24.00)	64.04 (25.27)	67.12 (15.03)
Maternal health and pediatrics	56.44 (20.94)	59.36 (19.03)	62.98 (23.12)	78.09 (20.77)	45.98 (22.38)	52.71 (22.38)	60.49 (23.22)	59.46 (13.83)
Medicine	64.03 (21.19)	63.11 (20.92)	64.47 (22.40)	79.45 (21.15)	48.44 (25.59)	54.86 (27.05)	53.57 (23.44)	61.02 (15.88)
Cardiovascular	58.33 (23.17)	60.59 (19.77)	65.81 (24.70)	78.52 (23.03)	43.94 (26.14)	53.93 (27.75)	56.89 (26.39)	59.63 (17.46)
	p < 0.05	p < 0.05	p < 0.05	p < 0.05	p < 0.05	p < 0.05	p < 0.05	p < 0.05
Position b
Nurse	60.57 (21.71)	59.40 (20.53)	64.19 (24.16)	76.19 (22.64)	44.38 (22.78)	52.70 (24.62)	56.08 (24.53)	59.13 (15.28)
Physician	76.78 (17.88)	71.40 (19.02)	73.59 (23.08)	78.18 (24.07)	49.17 (24.12)	72.70 (24.16)	60.09 (23.39)	68.90 (13.91)
Technician	62.80 (22.43)	61.93 (21.97)	65.51 (24.12)	73.75 (26.29)	50.97 (27.08)	55.05 (29.33)	55.61 (26.89)	61.28 (17.17)
Nurse aid personnel	70.17 (21.05)	69.22 (19.62)	70.51 (21.44)	76.15 (22.23)	53.81 (24.26)	61.95 (25.10)	61.66 (26.73)	67.10 (15.78)
	p < 0.05	p < 0.05	p < 0.05	p = 0.487	p < 0.05	p < 0.05	p = 0.078	p < 0.05
Gender
Female	63.86 (22.15)	62.36 (21.21)	66.60 (24.55)	75.75 (24.49)	47.55 (23.52)	55.75 (25.46)	63.86 (22.15)	58.01 (24.60)
Male	68.85 (21.41)	66.35 (20.72)	70.28 (23.45)	73.33 (23.84)	48.58 (25.64)	65.93 (27.61)	68.85 (21.41)	57.61 (25.36)
	p < 0.05	p < 0.0.05	p < 0.05	p = 0.197	p = 0.588	p < 0.05	p < 0.05	p = 0.833
Years of experience
0–4	66.01 (23.01)	64.75 (21.36)	70.86 (24.97)	75.65 (22.84)	47.52 (23.82)	55.89 (25.88)	61.08 (24.47)	63.10 (17.37)
5–10	63.51 (20.93)	60.95 (20.29)	66.85 (23.36)	77.65 (23.13)	46.96 (25.11)	57.20 (26.09)	56.36 (24.23)	61.54 (15.37)
11–20	65.31 (21.64)	63.03 (21.15)	65.03 (24.53)	75.39 (24.37)	47.93 (23.34)	57.82 (26.11)	56.48 (24.42)	61.85 (15.64)
>20	64.69 (22.72)	63.82 (21.72)	67.49 (24.35)	74.23 (25.04)	48.24 (24.67)	59.30 (26.45)	58.15 (25.50)	62.71 (16.28)
	p = 0.772	p = 0.398	p = 0.157	p = 0.477	p = 0.951	p = 0.619	p = 0.313	p = 0.750

Department of Cardiovascular Medicine (Cardiology Unit, Rehabilitation Cardiology, Sports Medicine); Department of Medicine (Gastroenterology, Geriatrics, Internal Medicine, Long‐term care, Physical Rehabilitation); Department of Medical Specialties (Emergency Medicine, Haemodialysis service, Nephrology, Neurology, Oncology, Pulmonology); Department of Surgery (Anaesthesia, Dermatology, General Surgery, Intensive Care, Ophthalmology, Orthopaedics, Otolaryngology, Urology, Week Surgery); Department of Maternal Health and Pediatrics (Gynaecology, Obstetrics, Pediatrics); Department of Medical Direction and Services (Casualties, Laboratory, Operating theatres, Outpatient services, Pathology, Pharmacy, Radiology, Sanitary transports, Sterilisation service, Transfusion service).

The group ‘Other’ was excluded from the analysis because of the small number (N = 29).

Working as a physician or a technician was associated with a more positive assessment (69.80 and 67.10, respectively) compared to nurses (59.13) and nurse aid personnel (61.28) and the difference was statistically significant (p < 0.001). This significance extended to all dimensions with the exception of Stress recognition (p = 0.487) and Working conditions (p = 0.078).

A statistically significant difference was also detected in the average total score for male and female staff (64.38 vs. 61.80; p = 0.036); this applied to all dimensions, with the exception of Stress recognition, Perception of management (hospital) and Working conditions. Distribution by years of experience showed no significant differences (p = 0.750) between the different categories, both for total score and individual dimensions.

Table 3 reports the results after application of the multivariate analysis (on the overall average score and for each dimension) for variables found to be significant in bivariate analysis (hospital, department, position, gender). The main differences were evident for the Teamwork dimension, for which the categories ‘Hospital C’, ‘Department of Medical Specialties’ and ‘Physician’ scored 15 point higher on average compared to the reference categories. In addition, Medical Specialties showed a significantly higher level of job satisfaction compared to other departments. Finally, physicians showed a significantly higher perception of hospital management compared to other professional categories.

TABLE 3

Regression coefficients and associated confidence intervals (95% CIs) on the mean total score and mean score by dimension for variables found to be significant in bivariate analysis. Significant differences are marked in bold. NS = not significant at the bivariate analysis. RG = reference group

	Teamwork climate	Safety climate	Job satisfaction	Stress recognition	Perception of management (H)	Perception of management (D)	Working conditions	Total
Hospital
A	6.0 (3.1; 8.9)	4.2 (1.3; 7.1)			0.7 (−2.7; 4.0)			2.7 (0.6; 4.9)
B	RG	RG	NS	NS	RG	NS	NS	RG
C	18.4 (11.4; 25.3)	9.7 (2.8; 16.6)			14.8 (6.7; 22.8)			11.7 (6.6; 16.8)
Department a
Medical direction and services	12.3 (8.2; 16.3)	5.5 (1.4; 9.6)	13.7 (9.2; 18.2)	0.3 (−4.0; 4.6)	9.0 (4.4; 13.6)	9.2 (4.0; 14.3)	2.6 (−1.9; 7.1)	9.0 (6.1; 12.0)
Surgery	RG	RG	RG	RG	RG	RG	RG	RG
Medical specialties	17.5 (13.0; 22.0)	9.4 (4.9; 13.9)	18.0 (12.8; 23.2)	−8.4 (−13.4; −3.4)	10.1 (5.0; 15.2)	13.2 (7.4; 18.9)	9.8 (4.7; 15.0)	11.3 (8.0; 14.6)
Maternal health and pediatrics	0.7 (−4.0; 5.3)	2.3 (−2.4; 7.0)	7.3 (1.9; 12.7)	3.0 (−2.2; 8.3)	6.7 (1.4; 12.1)	3.7 (−2.5; 9.8)	6.3 (0.9; 11.7)	5.2 (1.7; 8.6)
Medicine	−4.4 (−9.9; 1.1)	−3.0 (−8.6; 2.5)	5.2 (−0.5; 10.9)	4.4 (−0.9; 9.7)	1.3 (−5.1; 7.6)	0.2 (−6.2; 6.6)	−0.6 (−6.1; 4.8)	−1.7 (−5.8; 2.3)
Cardiovascular	0.2 (−4.9; 5.3)	2.1 (−3.0; 7.2)	8.6 (2.8; 14.4)	3.5 (−2.2; 9.1)	3.1 (−2.7; 9.0)	1.7 (−4.8; 8.1)	2.7 (−3.1; 8.5)	3.5 (−0.3; 7.2)
Position b
Nurse	RG	RG	RG		RG	RG		RG
Physician	15.7 (11.7; 19.7)	11.2 (7.1; 15.2)	9.6 (5.0; 14.1)		4.1 (−0.3; 8.6)	18.5 (13.4; 23.6)		9.3 (6.4; 12.3)
Technician	3.9 (−0.2; 8.1)	4.6 (0.4; 8.7)	2.2 (−2.5; 6.8)	NS	8.1 (3.2; 13.0)	3.5 (−1.8; 8.8)	NS	3.9 (0.9; 7.0)
Nurse aid personnel	6.9 (1.4; 12.4)	9.6 (4.0; 15.1)	5.3 (−0.2; 10.8)		4.8 (−1.5; 11.2)	8.8 (2.5; 15.1)		5.8 (1.7; 9.8)
Gender
Female	RG	RG	RG	NS	NS	RG	NS	RG
Male	2.8 (−0.6; 6.3)	3.5 (0.0; 7.0)	3.2 (−0.7; 7.1)			7.9 (3.5; 12.4)		2.3 (−0.2; 4.9)

Department of Cardiovascular Medicine (Cardiology Unit, Rehabilitation Cardiology, Sports Medicine); Department of Medicine (Gastroenterology, Geriatrics, Internal Medicine, Long‐term care, Physical Rehabilitation); Department of Medical Specialties (Emergency Medicine, Hemodialysis service, Nephrology, Neurology, Oncology, Pulmonology); Department of Surgery (Anesthesia, Dermatology, General Surgery, Intensive Care, Ophthalmology, Orthopedics, Otolaryngology, Urology, Week Surgery); Department of Maternal Health and Pediatrics (Gynecology, Obstetrics, Pediatrics); Department of Medical Direction and Services (Casualties, Laboratory, Operating theatres, Outpatient services, Pathology, Pharmacy, Radiology, Sanitary transports, Sterilization service, Transfusion service).

The group “Other” was excluded from the analysis because of the small number (N=29).

MaPSaF and SAQ comparison: Feedback on their integrated use

Among professionals completing both MaPSaF and SAQ (N = 67), the majority (67.1%) considered the two instruments complementary, as they provided with different kind of information; only 19% stated that an integrated use of the two instruments did not provide any additional information compared to the single tools. A small percentage (5.8%) did not find the results as representative of the organisation. Finally, 81% considered the final feedback as absolutely useful to stimulate and promote improvement strategies, whereas 19% described it as ‘partially useful’.

DISCUSSION

MaPSaF assessment regarding senior managers and SAQ showed a good response rate. According to the literature, a threshold of 60% is considered valuable in safety climate studies. , A key role in the achievement of this threshold could possibly be attributed to the engagement of the directors/coordinators through the MaPSaF workshops at the start of the project.

MaPSaF evaluation proved a bureaucratic level of patient safety culture in the organisation as a whole. According to different authors, bureaucratic organisations are usually characterised by low levels of cooperation, preoccupation with rules and lack of staff involvement in promoting changes. , Commitment to safety actually reflects a ‘formal’ adherence to requirements that are established mainly on a top‐down basis. In bureaucratic organisations, a lot of information concerning safety are usually collected but not effectively implemented to promote changes. In our study, some dimensions showed good results (Priority given to safety, System errors and individual responsibility, and Recording incidents and best practice) suggesting that initiatives concerning patient safety are mainly aimed at patient protection and not self‐protection, and that incident reporting is encouraged. However, a positive attitude towards safety alone does not lead to significant improvements. Indeed, the most critical items (Learning and continuous improvement, Staff education, and Training and team working) suggested that staff is not fully engaged, impeding the establishment of a virtuous improvement cycle. As stated by Braithwaite et al., powerful stakeholders have a pervasive, often hidden, influence over organisational initiatives. Indeed, insufficient stakeholders' involvement leads to superficial pseudo‐changes, where need for improvement is generally stated (Plan stage of the improvement cycle) but not followed by any effective action (Do stage), nor evaluation (Check stage).

Although the MaPSaF evaluation approach is predominantly analytical in nature, the framework provides some typical elements of an interpretative approach, which aim to describe the predominant characteristics of the organisational culture and provide a starting point for reflections on the objectives to be pursued. In this perspective, MaPSaF is useful at different levels:

induces the practitioner to reflect carefully on the level of culture corresponding to each dimension, in order to be able to make her/his own assessment;

provides a general framework in which the organisation takes on a dynamic role, allowing contextualised analysis of data collected with additional tools (including the SAQ);

identifies a reference ‘map’ for the objectives that can be set for each dimension (‘end points’), in order to introduce significant improvements and close the gap between the present and the desired culture.

As far as the results for SAQ are concerned, the percentage of responders was comparable in two out of three facilities while the third one was less represented (<100 responders), probably due to the smallest number of professionals. The overall total score showed a patient safety climate that was not completely satisfying, with more than 40% of the questionnaires denoting criticalities and opportunities for improvement. These data appear in line with similar studies in the literature. Carvalho et al. conducted a survey in three different public health facilities (573 participants) and reported an overall average score and participation rate which were comparable to the present investigation. Another study conducted in two hospitals of our Region reported data that are in line with the present paper.

Data emerging from SAQ assessment confirmed and completed the MaPSaF evaluation. Distribution of SAQ total scores showed a predominance of neutral scores. This result, combined with the high percentage of missing scores, raised the question whether SAQ assessment was perceived as a merely bureaucratic task rather than an opportunity to highlight relevant issues. Perception of management gained the lowest mean score; this trend was in line with data from other authors, , , , confirming that an effective collaborative relationship between frontline staff, middle management and top management still is a major issue. As stated by West and colleagues, every organisation should adopt a collective model of leadership, characterised by a ‘shared leadership where … the ebb and flow of power is situationally dependent on who has the expertise at each moment’. Our data suggested a lack of such a model of leadership, while evidencing the presence of a bureaucratic organisation where clinical governance still represents a theoretical vision rather than a systematic process.

The critical relationship between leadership and front‐line staff was also evidenced by the high percentage of missing responses concerning this dimension. In this area, items that achieved the highest PNRs (e.g., Hospital/Department management supports my daily efforts, Problems are dealt with constructively by our hospital management, and I get adequate, timely info about events that might affect my work from hospital management) suggested that managers were perceived as detached from frontline staff's safety concerns. In order to overcome this issue, a strategy which is widely recognised as relevant and useful is the Safety Walk Round (SWR). SWR enhances leadership commitment to patient safety and quality improvement by proactively supporting staff in facing daily safety issues and by creating a sense of belonging to the organisation, leading to the opportunity for strengthening trust. ,

The lack of a proper dialogue between staff and management was further underlined by the negative results related to the dimension Safety climate. Indeed, only half of the responders reported to receive an appropriate feedback, while the opportunity to discuss errors was not quite positively rated. Also, results from item 36 highlighted that communication breakdowns were common, suggesting that information exchange represented a challenging area. Evidence shows that feedback plays a critical role in every effective process. For example, closed‐loop communication techniques (e.g., read‐back/repeat‐back) are internationally recognised as effective to avoid misunderstandings which often result in adverse events. , Moreover, feedback is a key step of the ‘Plan‐Do‐Check‐Act’ cycle, allowing for a continuous loop of improvement. Lack of feedback is one of the main barriers identified in literature as contributing to prevent improvements from incident reporting systems. Furthermore, the absence of a concrete implementation of corrective actions resulting from incident reporting might lead to a bureaucratic use of this tool. A systematic review conducted by Sacks et al. showed that communication, safety climate and teamwork are the most critical domains targeted by improvement interventions. Accordingly, our data identified teamwork as another dimension that needs to be improved. Speak up when a problem with patient care occurred and Conflict management were perceived as particularly challenging. Disagreement in a team may play an important role to promote safety since it allows the integration of different perspectives, and conflicts may have detrimental effects when not adequately managed. , Speaking up is considered essential to patient safety since it allows communication openness and an adequate transfer of information, minimising the risk of adverse events. , A hierarchical organisation is often mentioned as a barrier to speaking up. On the other hand, when staffs are positive about levels of support this leads to improved patient satisfaction, health outcomes and ratings of quality of care.

Among SAQ dimensions, Stress recognition obtained the highest mean score, suggesting a high degree of awareness of workload and stress influence on work performance by the staff. Stress and burnout are recognised as important safety threats leading to suboptimal care. The high PPRs retrieved within this dimension suggest how professionals are aware that stress may negatively affect performance. However, literature reports how this subscale may be misinterpreted by responders and considered as a measure of stress level rather than stress recognition. It is worth to be noticed that the item with the worst score was Levels of staffing in this clinical area are sufficient to handle the number of patients. This finding is not unexpected, considering that in Italy shortage of healthcare professionals is a notable issue. Adequate staffing item often receives the lowest percentage of positive responses among all patient safety dimensions. In a study by Arrieta et al. only 27% of healthcare professionals considered their units to be adequately staffed and this percentage was analogous across all sub‐sectors (25–31%). Heavy workload is recognised as one of the main negative aspects affecting patient safety climate. , It was reported that a higher workload of the nursing team influenced the negative perception of patients' safety climate, particularly in the domains Teamwork climate, Safety climate, and Job satisfaction. Despite our results pointed out several safety domains that may benefit from improvement interventions, Job satisfaction reached a satisfying score with almost the whole staff stating to like their job, in line with a previous study by Relihan et al.

Our results show that safety climate is perceived in significantly different way by different departments and groups. These results were observed on total and individual dimensions scores, except for Stress recognition. This evidence confirms the presence of multiple subcultures: surgical departments showed a much lower total score compared to medical departments; also, physicians achieved a higher mean score than nurses. Differences in perception between medical and nursing staff are widely documented in the literature. Through the use of the Intensive Care Unit Management Attitudes Questionnaire (precursor of the SAQ for the intensive area), Thomas et al. found discrepancies in the perception of teamwork between medical and nursing staff. The leadership style of nursing coordinators was analysed by Katrinli et al., emphasising the need to foster empowerment and participation of nursing staff for greater job satisfaction. Also, Vlayen et al. documented a tendency of medical staff to overestimate the safety performance of their teams. Perceptions of a high workload and communication problems may play a role in the development of ‘subcultures’ in different professional groups.

On the contrary, no significant differences were found between male and female personnel for total mean score. These results were comparable to a study conducted by Gambashidze et al., which did not detect significant gender‐related influences on the perception of safety climate in the hospital context. However, an indirect influence was hypothesised from the low number of senior female management figures.

Overall, nurses had a different perception of safety culture in comparison with other professionals. For the domains Safety climate, Perceptions of management of the unit and of the hospital and Working conditions the differences were statistically significant when compared with other groups. However, the opposite was observed in the domain Stress recognition, where professionals other than nurses presented a higher mean score showing a better recognition of stressing factors influencing their work, in line with a previous study.

With regard to the years of working experience, the differences found between the different categories were not statistically significant. However, there was a tendency towards a more positive perception by less experienced operators, especially in the Job satisfaction area, whereas a more homogeneous distribution was assessed for Perception of the management and Teamwork climate areas. These results were in line with the study by Vlayen et al., where newly hired staff demonstrated an overall better perception of safety climate than more experienced colleagues. Also Bodur and Filiz highlighted a tendency towards more negative assessments with the increase in the years of experience. This might be the effect of the accumulation of errors through the professional path.

Multiple linear regression analyses confirmed significant differences for all the investigated variables, except for gender. In particular, working in a small hospital, in a medical department, and being a physician were associated with the most positive results. When analysing single dimensions, it was interesting to note how the main differences are in the Teamwork area. The more positive perceptions expressed by physicians seem to confirm the presence of a hierarchical model rather than a complementarity between different professional figures. Also, Job satisfaction detected an important gap between reports from surgical and medical departments, a finding that would require further investigation.

The analysis of SAQ data confirmed the priorities for action that were already identified in the previous analysis with MaPSaF. The two tools are able to provide different perspectives and visions on the same reality examined. The main criticalities found were assessed with an eye on the integrated use of the two tools, proposing a series of corrective actions, also highlighting areas for improvement and the ‘vision’ towards which the whole organisation should strive. For example, among the critical areas identified through MaPSaF was collecting information to meet safety standards rather than promoting effective safety initiatives, with safety initiatives imposed by management rather than actively pursued by the entire staff. This was reflected in the results from SAQ, which indicated a lack of adequate dialogue between management and staff, a problematic discussion of errors, and a perception of management's failure to take on safety suggestions. The integration of these findings led to proposals for improvement such as implementing tools to enhance collaboration between staff and management (e.g., SWA, provision of appropriate feedback), ensuring discussion of safety issues raised by staff (e.g., clinical audit) and promoting implementation of the improvement actions by ensuring monitoring and feedback to the healthcare worker.

At the end of the analyses, specific reports were produced for each unit and distributed at the final plenary meeting. Presenting these results exceeds the aims of this article. Nonetheless, we sought to highlight that in order to promote an improvement in the culture of safety it is essential to work on communication of information relating to clinical safety. The reports underlined the need to improve the dialogue on both communication flows (top‐down and bottom‐up). There was also the need to improve teamwork, with particular emphasis on the promotion of multi‐disciplinarity and management of conflict. These needs could be addressed through the implementation of strategies aimed at reinforcing collaboration and communication (e.g., TeamSTEPPS®) and the definition of specific training needs for each area. Also, developing effective decision‐making between clinicians and managers is a major outstanding challenge to performance management.

While simple in concept, performance management is usually difficult for public hospitals. Relationships between clinicians and managers retain much of their traditional character, with jealous guarding of the contested border between clinical autonomy and management prerogatives. , Differences in safety climate appear to be the result of the nature of work, interactions, working conditions and leadership patterns in each clinical care area. While there are some promising signs that clinical governance is providing with a new way of thinking about this problem, effective sharing of goals is a major challenge to performance management. As stated by West et al., each organisation should adopt a collective leadership model, in which there is still a formal hierarchy, but the role of leader is given to the person who has specific expertise on the issue at hand. In addition, a study by Wong and Laschinger has shown that leadership models characterised by transparency, promotion of self‐awareness and high ethical standards enhance the well‐being of nurses at work. Actions that could be taken to demonstrate leadership in patient safety could be to include medical staff in institutional patient safety programs.

Results of the feedback questionnaire given to respondents highlighted a favourable evaluation of an integrated use of the two instruments, particularly when referring to the achievement of complementary information. However, about one out of 3 responders expressed doubts about choosing the most appropriate actions in the light of the results obtained. It emerges the need to contextualise the criticalities detected through the involvement of the Risk Manager and the definition of the appropriate corrective actions. In addition, it would be useful to implement specific training programs to effectively guide the adoption of improvement initiatives and to adequately motivate staff.

The possible criticalities linked to the integrated implementation of the two tools lie mainly in the need for adequate time and resources to encourage staff involvement (e.g., participation in dedicated workshops and meetings, dissemination and collection of questionnaires), which are not always compatible with the working rhythms and organisational set‐up. Achieving a satisfactory adherence rate and cross‐sectoral participation depends on a strong commitment from management, which may not always be optimal.

This study presents some limitations. The response rate to the questionnaires, although sufficient, was not optimal especially in one of the three facilities. A further limitation was the administration of MaPSaF at the management/coordinators level only, reflecting a partial and less representative view compared to SAQ assessment. On the other hand, this choice was motivated by the impossibility of conducting workshops open to all staff and by the purpose of the initial assessment, which was to lay the foundations for awareness of the need for the assessment and to adequately motivate management for the subsequent SAQ. Nonetheless, the integrated use of the two instruments was overall rated as useful and the survey showed that the use of an integrated approach in the evaluation of a healthcare organisation's safety culture may result useful for an in‐depth analysis of the criticalities and the subsequent adoption of appropriate improvement strategies.

CONCLUSION

The survey showed that the use of an integrated approach in the evaluation of a healthcare organisation's safety culture may result useful for an in‐depth analysis of the criticalities and the subsequent adoption of appropriate improvement strategies. The strengths and weaknesses that emerged in the context provide a precise indication on the area most susceptible of improvement and the main factors which can influence the spread of an appropriate safety culture. The limits due to an approach using mainly instruments aimed at detecting safety climate should necessarily be overcome through the application of methodologies designed to capture the most interpretative aspects of the analysed culture. In this perspective, it is highlighted that the complementary nature of MaPSaF and SAQ can contribute to the definition of a more comprehensive picture of the culture of safety, with considerable benefits for all the professionals.

The survey also confirmed the presence of subcultures of safety and the difficulties in translating the organisation's needs into effective improvement strategies. Nonetheless, the presence of subcultures can also have a positive value: the analysis may lead to the adoption of targeted corrective measures, and examining ‘virtuous’ subgroups can help understanding key factors of a positive culture of safety, promoting a dynamic of mutual learning and integration.

CONFLICT OF INTEREST

Conflict of interest statement for all authors: The authors whose names are listed in the ‘Authors’ section certify that they have NO affiliations with or involvement in any organisation or entity with any financial interest, or non‐financial interest in the subject matter or materials discussed in this manuscript.

ETHICS STATEMENT

Ethical approval was waived for the present study because the dataset was suitably anonymised.

TABLE A1. Evaluation sheet (sample) of the Manchester Patient Safety Framework (MaPSaF). Specific statements (not reported) address five different levels of safety culture (A—pathological, B—reactive, C—bureaucratic, D—proactive, E—generative)

Dimension of patient safety culture	A	B	C	D	E
1. Commitment to overall continuous improvement
2. Priority given to safety
3. System errors and individual responsibility
4. Recording incidents and best practice
5. Evaluating incidents and best practice
6. Learning and effecting change
7. Communication about safety issues
8. Personnel management and safety issues
9. Staff education and training
10. Team working

TABLE A2. Safety Attitude Questionnaire (SAQ) assessment results per item. N = number of records. Missing = missing records. Mean = mean score per item on a 1–5 Likert scale. SD = standard deviation. PNRs = percentage of negative responses (1–2 on Likert scale). PPRs = percentage of positive responses (4–5 on Likert scale)

	Item	N	Missing (%)	Mean (SD)	PNRs (%)	Neutral scores (%)	PPRs (%)
Teamwork climate	1. Nurse input is well received in this clinical area	1,020	5.82	3.60 (1.19)	23.23	12.54	64.23
	2. In this clinical area, it is difficult to speak up if I perceive a problem with patient care	1,045	3.51	3.22 (1.38)	38.37	12.24	49.39
	3. Disagreements in this clinical area are resolved appropriately (i.e., not who is right, but what is best for the patient)	1,062	1.94	3.39 (1.35)	30.88	14.31	54.81
	4. I have the support I need from other personnel to care for patients	1,030	4.89	3.80 (1.22)	19.90	11.65	68.45
	5. It is easy for personnel here to ask questions when there is something that they do not understand	1,055	2.59	4.08 (1.18)	14.59	9.66	75.75
	6. The physicians and nurses here work together as a well‐coordinated team	1,024	5.45	3.39 (1.35)	29.78	13.37	56.85
Safety climate	7. I would feel safe being treated here as a patient	1,044	3.60	3.78 (1.24)	18.01	19.15	62.84
	8. Medical errors are handled appropriately in this clinical area	986	8.96	3.57 (1.20)	20.79	26.16	53.05
	9. I know the proper channels to direct questions regarding patient safety in this clinical area	1,050	3.05	3.68 (1.21)	18.95	19.52	61.53
	10. I receive appropriate feedback about my performance	1,054	2.68	3.39 (1.30)	28.08	19.35	52.57
	11. In this clinical area, it is difficult to discuss errors	1,070	1.20	2.98 (1.41)	43.55	16.07	40.38
	12. I am encouraged by my colleagues to report any patient safety concerns I may have	1,042	3.79	3.77 (1.24)	18.04	18.23	63.73
	13. The culture in this clinical area makes it easy to learn from the errors of others	1,042	3.79	3.45 (1.26)	23.32	21.49	55.19
Job satisfaction	15. I like my job	1,062	1.94	4.69 (0.75)	3.01	4.80	92.19
	16. Working here is like being part of a large family	1,045	3.51	3.38 (1.38)	29.09	16.26	54.65
	17. This is a good place to work	1,064	1.75	3.50 (1.31)	25.65	15.97	58.38
	18. I am proud to work in this clinical area	1,061	2.03	3.80 (1.20)	16.02	19.22	64.76
	19. Morale in this clinical area is high	1,060	2.12	3.01 (1.35)	39.33	18.20	42.47
Stress recognition	20. When my workload becomes excessive, my performance is impaired	1,073	0.92	4.07 (1.21)	13.69	8.66	77.65
	21. I am less effective at work when fatigued	1,075	0.74	4.15 (1.14)	10.79	9.21	80
	22. I am more likely to make errors in tense or hostile situations	1,063	1.85	4.23 (1.13)	10.34	8.37	81.29
	23. Fatigue impairs my performance during emergency situations (e.g. emergency resuscitation, seizure)	936	13.57	3.52 (1.45)	28.20	13.56	58.24
Perception of management (hospital)	24. Management supports my daily efforts	919	15.14	2.44 (1.24)	50.81	28.40	20.79
	25. Management does not knowingly compromise patient safety	895	17.36	3.46 (1.26)	20.33	32.51	47.16
	26. Management is doing a good job	921	14.96	2.94 (1.15)	32.46	38.00	29.54
	27. Problem personnel are dealt with constructively	876	19.11	2.73 (1.17)	37.89	36.75	25.36
	28. I get adequate, timely info about events that might affect my work	944	12.83	2.90 (1.22)	36.75	30.40	32.85
Perception of management (department)	24. Management supports my daily efforts	903	16.62	2.99 (1.40)	39.53	19.49	40.98
	25. Management does not knowingly compromise patient safety	879	18.84	3.76 (1.27)	16.60	24.68	58.72
	26. Management is doing a good job	908	16.16	3.38 (1.26)	24.66	26.87	48.47
	27. Problem personnel are dealt with constructively	893	17.54	3.16 (1.33)	32.02	24.07	43.91
	28. I get adequate, timely info about events that might affect my work	936	13.57	3.29 (1.35)	32.05	17.20	50.75
Working conditions	29. The levels of staffing in this clinical area are sufficient to handle the number of patients	1,029	4.99	2.54 (1.39)	56.65	13.60	29.75
	30. This hospital does a good job of training new personnel	1,019	5.91	2.85 (1.27)	41.61	26.39	32
	31. All the necessary information for diagnostic and therapeutic decisions is routinely available to me	972	10.25	3.35 (1.25)	28.29	21.39	50.32
	32. Trainees in my discipline are adequately supervised	924	14.68	3.79 (1.19)	15.15	23.70	61.15
Single items	14. My suggestions about safety would be acted upon if I expressed them to management	1,018	6.00	2.98 (1.29)	34.77	30.74	34.49
	33. I experience good collaboration with nurses in this clinical area	1,056	2.49	4.28 (1.01)	8.16	9.18	82.66
	34. I experience good collaboration with staff physicians in this clinical area	1,048	3.23	4.04 (1.08)	12.21	12.31	75.48
	35. I experience good collaboration with pharmacists in this clinical area	906	16.34	3.45 (1.15)	17.54	36.86	45.6
	36. Communication breakdowns that lead to delays in delivery of care are common	999	7.76	3.19 (1.32)	35.63	20.72	43.65

TABLE A3 Results of Safety Attitude Questionnaire (SAQ) assessment per dimension for PSRs = score ≥ 60, and PTRs = score ≥ 80. Results for the dimension Perception of management are presented as overall for hospitals and departments

Dimension	PSRs (%)	PTRs (%)
Teamwork climate	61.55	25.88
Safety climate	59.09	21.97
Job satisfaction	66.38	39.44
Stress recognition	77.81	52.12
Perception of management	39.41	13.10
Working conditions	44.46	21.00
Total	57.34	6.46