Development of the German version of the patient safety climate inventory to the Austrian context.

OBJECTIVES: In recent years, patient safety culture (PSC) in hospitals, including its development and measurement, has increasingly received attention in Europe. Even though several instruments have been developed for PSC measurement in European countries, there is, to date, no validated measure to assess PSC in Austria. The study at hand addresses this gap in the evidence base by psychometrically assessing the German 'Patient Safety Climate Inventory' (PaSKI) in terms of its potential suitability for the Austrian healthcare system. The goal is to theoretically develop and empirically verify a separate instrument for PSC measurement in Austria.
SETTING: Ten hospitals. PARTICIPANTS: Healthcare professionals (n=1202); doctors (n=142), nurse (n=645), other health workers (n=51), medical technology professions (n=170), management/administration (n=76), other (n=20), no response (n=98). PRIMARY AND SECONDARY OUTCOME MEASURES: The pretest was conducted with 101 health professionals. Psychometric evaluations, including exploratory factor analysis and confirmatory factor analysis, were performed with both an original version of the PaSKI and an adapted one. The original PaSKI and the newly adapted 'Austrian Patient Safety Climate Inventory' (A-PaSKI) were then compared.
RESULTS: The A-PaSKI's factor structure developed in our study differs from the original 14-factor structure (49 items) of the PaSKI. The new instrument consists of 10 factors (30 items), comprising seven departmental factors, two hospital factors, and one outcome factor. The new instrument A-PaSKI revealed satisfactory results on the model-level and internal consistency. The confirmatory factor analysis for the A-PaSKI (χ2 (360)=1408.245, p=0.0001) showed a good model fit, and the absolute and relative fit indices showed an excellent model adjustment. The construct validity was acceptable for nine and unacceptable for one factor.
CONCLUSIONS: This is the first validation study of a standardised safety culture measure in Austrian hospitals. The Austrian version of PaSKI demonstrated good psychometric properties, with acceptable to good internal consistency and construct validity for use in Austrian hospitals. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A scientific psychometric approach was designed and executed to test the Patient Safety Climate Inventory (PaSKI) model and create an optimal model.

Data were collected from a large pool of healthcare professionals (n=1202) across multiple hospitals in Austria.

The Austrian version of PaSKI demonstrated good psychometric properties for use by researchers and hospitals interested in assessing patient safety culture at the hospital, unit and professionals group level.

The construct validity was acceptable for the nine-factor and only unacceptable for one factor.

The response rate was low, but this has hardly any effect on the results as the sample includes a large number of hospitals, units and professional groups, which is completely sufficient for our psychometric analysis.

Introduction

Over the last few years, patient safety has received increased attention as the scale of safety problems in healthcare has become apparent.1 The estimated absolute number of adverse events (AEs) in the USA appears to be four times as high (400 000) as the number documented in the first published results of 1999 (98 000).2 Most of this increase is due to population growth and ageing. However, the AE mortality rate resulting from medical treatment decreased by 21% in the USA in the time period from 1990 to 2016.3 In European countries, it is estimated that one in every 10 patients is harmed while receiving hospital care. In Organisation for Economic Co-operation and Development countries, 15% of total hospital activity and expenditure are a direct result of AE.4 These studies demonstrate that AEs are still a significant public health challenge internationally. An AE is a circumstance or event that results in a patient’s injury due to a medical intervention rather than the underlying medical condition.5 AEs are also associated with high costs, complications and unnecessary readmissions to hospitals.6

Many authors have called for greater attention to be given to the development of a patient safety culture (PSC) as a foundational step to reducing errors and AEs.7 8 Safety culture is generally defined as ‘the product of individual and group values, attitudes, perceptions, competencies and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety management. Organisations with a positive safety culture are characterised by communications founded on mutual trust, by shared perceptions of the importance of safety and by confidence in the efficacy of preventive measures’.9 A safety culture promotes an individual’s and an organisation’s values, attitudes and perceptions regarding patient safety.10 11 A considerable number of studies indicate that a well-implemented PSC is linked to reduced infections, lower rates of AE, decreased readmission rates, direct positive relationships with financial performance and motivation to work safely.11–16

Before creating and improving PSC, it is crucial to measure the status quo to define a starting point.17 This process was adopted in the development of several corresponding measurement instruments.18 In the European Union alone, 19 different instruments are currently being used.19 The two most widely applied PSC instruments in hospital settings are the Safety Attitude Questionnaire20 and the Hospital Survey on Patient Safety Culture (HSOPSC) by the Agency for Healthcare Research and Quality (AHRQ).21 The HSOPSC is used in more than 45 countries and can thus be rated as the most widely used instrument worldwide.18 The HSOPSC questionnaire by AHRQ has no explicit theoretical basis.22 23 Rather, the HSOPSC was developed by conducting a literature review in the areas of safety management and accidents, organisational and safety climate and culture, medical error and error reporting, and patient safety. Subsequently, key dimensions of PSC were identified, and survey items were developed.22 For those reasons, the AHRQ first validated the self-developed instrument called HSOPSC in the USA, and then recommended its validation for other countries.17 22 24

Seven instruments measuring PSC are currently being used in the DACH countries (DACH is an acronym used to describe Germany (D), Austria (A) and Switzerland (CH)).25–31 The instrument ‘Vienna Safety Culture Questionnaire’ (WSF) was developed and validated in Austria.30 However, the authors did not include an exploratory factor analysis (EFA) in the related study, which was only conducted in the pretest. In addition to this, the WSF was only tested with nursing staff and staff in a psychiatric ward. Hence, this study’s validity has not been confirmed, and it can not be applied to all hospital staff. There is, therefore, no psychometrically tested instrument to measure PSC in Austria. Instead, PSC has been measured using the HSOPSC which has been tested and adapted to Germany or Switzerland32 even though these two validation studies revealed entirely different factor structures.25 31 However, there are a number of reasons why the German and Swiss versions cannot be immediately adapted to Austria. First, in the German language, there are several linguistic variations in the DACH countries, and the meaning of words may vary across cultures.33 Second, in the basic recommended translation strategies for HSOPSC published by the AHRQ,34 there is no report on a validated German language version of the HSOPSC for cross-cultural research in DACH countries. Third, the DACH countries’ health systems differ in several dimensions (eg, organisation, financing, provision of services, principal health reforms),35–37 thus directly or indirectly influencing PSC.

The objective of this research project was to develop an instrument for measuring PSC in Austria. In contrast to previous studies that used the HSOPSC as an instrument for psychometric analysis, we use the Patient Safety Climate Inventory (PaSKI) as a base. The PaSKI is the translated, extended, and psychometrically tested German version of the HSOPSC in Switzerland.25 It includes all HSOPSC factors and two additional factors with nine new items. The factors unit management support for patient safety (five items) and Unit handoffs and transitions (four items) were added to the PaSKI instrument to account for the importance of unit management in this type of hospital.25 We investigate if all 14 factors of the PaSKI (49 items) are applicable for measuring PSC in Austria. We use EFA and confirmatory factor analysis (CFA) to develop a sound factor structure for measuring PSC in Austria and subsequently review this factor structure. After the model-fit testing with CFA, the internal consistency and the construct level was calculated. With a validated instrument, hospitals will be able to obtain valid results on PSC. In this way, the hospitals will be able to sensitise their personnel in this area, measure and analyse their PSC’s status quo, identify strengths and weaknesses of their PSC, implement and evaluate targeted PSC measures, and draw comparisons both within and outside the organisation.38

Methods

Instrument and factors

The PaSKI is the theoretical basis for this study. The PaSKI consists of 14 factors, where each factor consists of 3–4 instrument items, totalling 49 instrument items. The instrument uses a five-point Likert response scale of agreement from ‘strongly disagree’ to ‘strongly agree’ or frequency from ‘never’ to ‘always’. It includes nine unit-level factors (supervisor, manager expectations and actions promoting safety, organisational learning—continuous improvement, teamwork within units, communication openness, feedback and communication about error, non-punitive response to error, staffing, unit management support for patient safety and unit handoffs and transition), three hospital-level factors (hospital management support for patient safety, teamwork across hospital units and hospital handoffs and transitions) and two outcome factors (overall perceptions of safety and Frequency of event reporting).25

Translation, survey design and pretest

The PaSKI had already been translated into German by the ETH Zurich (Swiss Federal Institute of Technology) and was made available to the author SD by Tanja Manser on request. Pfeiffer and Manser translated the HSOPSC version into German and then back-translated it with the help of various translators. Inconsistencies in the translations were resolved through discussions between the translators, the healthcare professionals and survey experts comparing original and translated versions.25

After rearranging the instrument in a new design, a pretest with 101 participants was conducted in a hospital. We used a survey design in line with the research recommendation.39 The data were collected online using LimeSurvey software.40 The invitation mode was a short email, the URL placement was at the end of the invitation, and the estimated time required for filling in the survey was 10–15 min. In the new design, only one department variable, that is, ‘What is your primary work area or unit in this hospital?’ was changed and adapted to the Austrian health system. In the pretest, we calculated EFA to check whether PaSKI is a good basis for the Austrian health system. In a second step, 25 health professionals from eleven hospitals were asked to assess their comprehension of the instrument (including all items and instructions). The aim of the interviews was to confirm whether all questionnaires are understandable. Semistructured interviews were conducted, lasting between 15 and 20 min. All recommendations for the item revision and item exclusion were discussed by the authors and a survey expert and included in the questionnaire.

Design

Survey of hospital professionals using the PASKI measure of PSC to assess and validate the measure for use in Austria.

Data sample

The authors contacted collegial management officials (medical director, nursing managers and administrative managers) from 80 public hospitals and invited them to participate in the survey. In the end, 10 hospitals accepted our invitation and took part in the survey.

Procedure

We provided the information on the study to all collegial management officials and quality managers. The quality managers explained the study’s purpose to all employees in a meeting and responded to questions. A link to the survey was then sent to all employees by email, with one reminder after 2 weeks. The actual data collection took 4 weeks in each hospital and was conducted between September and December 2017. In the last week, the quality managers held a meeting to remind the entire workforce once again to take part in the survey. The data collection was completely anonymised and confidential.

Data analysis

Only questionnaires with complete answers amounting to a minimum of 30% were accepted. The CFA can only be calculated with complete answers. The missing values were substituted for all other items utilising the maximum likelihood–expectation-maximisation algorithm (EM algorithm).41 The EM algorithm uses a two-step (an expectation step and a maximisation step) iterative procedure where missing observations are subsequently estimated. The EM algorithm provides estimates of the means and the covariance matrix, which can be used to receive consistent estimates of the parameters of interest.42 Further, all negative questions were decoded and labelled ‘r’. On the 5-point Likert scale, all items were scored from ‘strongly disagree’ (1) to ‘strongly agree’ (5) or, in case of frequency from ‘never’ (1) to ‘always’ (5). All data were evaluated with SPSS Statistics V.24.0 (Superior Performing Software System) and AMOS V.24.0 (Analysis of Moment Structures).

EFA: alternative model

An EFA was used to calculate the new factor structure based on empirical data from hospitals in Austria. The new factor structure constituted the calculation basis for the subsequent CFA, which only assumes that the factor structure was tested with empirical data. The EFA is the first prerequisite to test the model.43 44 To this end, the Kaiser-Meyer-Olkin-criterion (KMO) and the measure of sample adequacy (MSA) were calculated. Additionally, the EFA was computed using the maximum-likelihood analysis, which is also used by AMOS and thus a basis for CFA.43 44 All factors were strongly correlated, and we therefore calculated an oblique rotation for the factors to achieve improved interpretability.43 44 Items with a lower than 0.4 factor loading and high cross-loadings between first and second loading were excluded from achieving a convergent and discriminant validity.45

Confirmatory factor analysis

We conducted two CFAs, one to test whether the theoretically developed PaSKI from Switzerland was compliant with the empirical data from hospitals in Austria; the second one was to test whether the new developed alternative model, the so-called Austrian Patient Safety Climate Inventory (A-PaSKI), was compliant with the empirical data from hospitals in Austria. The models were tested on three levels—first on the model level, second for internal consistency, third on the construct level—and first assessed on the model level via global and local model fit.44 46 47 At the model level, to avoid all potential difficulties associated with χ2, other absolute fit indices were calculated, such as the root-mean-square-error of approximation (RMSEA) and Goodness-of-Fit-Index (GFI).44 47 Moreover, the comparative fit coefficient (CFI) was calculated as an incremental or comparative fit index. Also, the model fit was tested with a standardised root mean square residual (SRMR) by calculating the difference between the empirical variance-covariance of a variable and the theoretically calculated variance-covariance of this variable.47 The last coefficient calculated to assess model fit in this study was the Tucker-Lewis Index (TLI).48 The internal consistency was calculated using Cronbach‘s α for the original 14-factor PaSKI and for the new 10-factor A-PaSKI.46 In a third step, the models were tested on a construct level. CFA was used to determine the average extracted variance (AVE) and discriminant validity through the Fornell-Larcker criterion (FLR).49

Model comparison

Apart from the abovementioned individual significance tests and other coefficients, these 10-factor and 13-factor models were directly compared using so-called information criteria. These criteria take into account the adjustments (model fit) of the respective model and the model parameters and the sample size. In the study at hand, the Akaike information criterion (AIC)50 and the Bayesian information criterion (BIC) were used.51

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

The analysis was conducted in five steps: In the first step, the pretest results and feedback from health professionals were presented. In the second step, we explained how the response rate was calculated. Subsequently, sample demographics were presented. In the third step, the new questionnaire was presented. The new questionnaire was developed using EFA and checked using CFA. In the fourth step, the results of the new questionnaire A-PaSKI were compared with the original PaSKI questionnaire. Finally, the reliability and validity results of the questionnaire were presented. In the following, the results gained are discussed according to these five steps.

Survey design and pretest

The EFA pretest suggested that PaSKI had a good factor structure, and it was similar to the original PaSKI instrument. Therefore, the original PaSKI instrument structure and composition was retained.

Feedback from healthcare professionals

Changes were made following feedback from health professionals. Two items were removed as they were not understood or misunderstood (We have enough staff to handle the workload, u_A2; and We work in ‘crisis mode’ trying to do too much, too quickly, u_A14). Minor changes were made to five items (u_D1, u_D2, u_D3, u_A5r and u_A11). One item was added as it was considered to be of particular importance to Austrian hospitals (What is your primary work area or unit in this hospital?) Hence, the instrument consisted of 14 factors with 47 items.

Response rate and sample demographics

The collegial management officials at two hospitals did not allow us to ask or rephrase those two items: Hospital units do not cooperate well with each other (h_F2r) and Problems often occur during the exchange of information in the unit (u_F7r). Those items were only slightly improved in language. The management argued that these questions could not be asked as they suggested that the current workforce could be changed. However, we used these two questions in eight other hospitals. Therefore, those items were included in the analysis. The respective response rate was completely sufficient for the psychometric analysis. The survey was sent to 6587 potential participants, 1525 returned the questionnaire (response rate of 23.1%). All questionnaires with complete answers amounting to a minimum of 30% were accepted. All other questionnaires with missing values were replaced using the EM algorithm method. After our calculations following the EM algorithm, 1202 surveys remained for the psychometric analysis. The response rate in the ten participating hospitals ranged from 18.2% to 32.1%. Our sample was derived from 6 professional groups (doctors, nurse, other health workers, medical technology professions, management/administration and other) and 17 different hospital units (see table 1). However, to check on potential sample bias, we performed two nonresponse bias techniques (wave analysis and archival analysis)52 to determine significant differences between responders and non-responders. During wave analysis, we found significant differences in 7 from 30 variables. We found no significant differences between professional groups in our sample and professional groups in other Austria hospitals53 in the archival analysis.

Table 1

Professionals characteristics

Hospitals	Doctors	Nurse/registered nurse	Other health workers	Medical technology professions	Management/administration	Other	Total
	N (%)	N (%)	N (%)	N (%)	N (%)	N (%)	N (%)
Hospital A	1 (0.7)	11 (1.7)	0 (0.0)	2 (1.2)	4 (5.3)	0 (0.0)	18 (1.6)
Hospital B	0 (0.0)	15 (2.3)	2 (3.9)	9 (5.3)	8 (10.5)	0 (0.0)	34 (3.1)
Hospital C	14 (9.9)	81 (12.6)	8 (15.7)	32 (18.8)	2 (2.6)	0 (0.0)	137 (12.4)
Hospital D	3 (2.1)	39 (6.0)	0 (0.0)	21 (12.4)	4 (5.3)	1 (5.0)	68 (6.2)
Hospital E	9 (6.3)	26 (4.0)	0 (0.0)	12 (7.1)	9 (11.8)	1 (5.0)	57 (5.2)
Hospital F	13 (9.2)	30 (4.7)	2 (3.9)	5 (2.9)	3 (3.9)	1 (5.0)	54 (4.9)
Hospital G	9 (6.3)	69 (10.7)	13 (25.5)	20 (11.8)	7 (9.2)	3 (15.0)	121 (11.0)
Hospital H	18 (12.7)	32 (5.0)	6 (11.8)	5 (2.9)	5 (6.6)	1 (5.0)	67 (6.1)
Hospital I	2 (1.4)	24 (3.7)	2 (3.9)	10 (5.9)	4 (5.3)	5 (25.0)	47 (4.3)
Hospital J	73 (51.4)	318 (49.3)	18 (35.3)	54 (31.8)	30 (39.5)	8 (40.0)	501 (45.4)
Total	142 (100.0)	645 (100.0)	51 (100.0)	170 (100.0)	76 (100.0)	20 (100.0)	1104 (100.0)
Units in hospitals
Anaesthesiology and intensive care	28 (20.0)	95 (15.0)	0 (0.0)	1 (0.6)	0 (0.0)	1 (5.0)	125 (11.5)
Accident surgery	29 (20.7)	34 (5.4)	4 (7.8)	8 (4.7)	2 (2.9)	1 (5.0)	78 (7.2)
Roentgen	0 (0.0)	0 (0.0)	0 (0.0)	22 (13.0)	0 (0.0)	1 (5.0)	23 (2.1)
Laboratory	1 (0.7)	0 (0.0)	0 (0.0)	22 (13.0)	0 (0.0)	1 (5.0)	24 (2.2)
Surgery	9 (6.4)	57 (9.0)	3 (5.9)	2 (1.2)	3 (4.3)	1 (5.0)	75 (6.9)
Internalmedicine	10 (7.1)	74 (11.7)	6 (11.8)	2 (1.2)	11 (15.9)	0 (0.0)	103 (9.5)
Neurology	4 (2.9)	12 (1.9)	1 (2.0)	16 (9.5)	1 (1.4)	0 (0.0)	34 (3.1)
Orthopaedics	9 (6.4)	69 (10.9)	10 (19.6)	18 (10.7)	6 (8.7)	2 (10.0)	114 (10.5)
Psychiatry/mental health	7 (5.0)	30 (4.7)	2 (3.9)	11 (6.5)	0 (0.0)	0 (0.0)	50 (4.6)
Pneumology	5 (3.6)	11 (1.7)	0 (0.0)	2 (1.2)	3 (4.3)	1 (5.0)	22 (2.0)
Radio-oncology	3 (2.1)	5 (0.8)	1 (2.0)	9 (5.3)	0 (0.0)	1 (5.0)	19 (1.8)
Ophthalmology	2 (1.4)	8 (1.3)	1 (2.0)	4 (2.4)	3 (4.3)	0 (0.0)	18 (1.7)
Dermatology	1 (0.7)	14 (2.2)	0 (0.0)	0 (0.0)	1 (1.4)	2 (10.0)	18 (1.7)
Gynaecology	4 (2.9)	19 (3.0)	6 (11.8)	1 (0.6)	1 (1.4)	2 (10.0)	33 (3.0)
Ear–nose–throat diseases	3 (2.1)	11 (1.7)	1 (2.0)	1 (0.6)	0 (0.0)	0 (0.0)	16 (1.5)
Paediatricmedicine	11 (7.9)	47 (7.4)	0 (0.0)	2 (1.2)	0 (0.0)	0 (0.0)	60 (5.5)
Administration	0 (0.0)	0 (0.0)	1 (2.0)	5 (3.0)	19 (27.5)	2 (10.0)	27 (2.5)
No response	14 (10.0)	148 (23.3)	15 (29.4)	43 (25.4)	19 (27.5)	5 (25.0)	244 (22.5)
Total	140 (100.0)	634 (100.0)	51 (100.0)	169 (100.0)	69 (100.0)	20 (100.0)	1083 (100.0)

Exploratory factor analysis

The results of the adequacy of data demonstrated that the empirical data from hospitals in Austria is a suitable basis for applying the EFA.54 The value of the KMO coefficient amounted to 0.931, the values of the MSA coefficient for individual items were between 0.847 and 0.976. The Bartlett test was highly significant (χ2 (435)=18 077.817; p=0.0001). Through EFA, a new model—the A-PaSKI—was developed featuring a new 10-factor structure with 30 items: seven-unit factors (supervisor, manager expectations and actions promoting safety, teamwork within units, communication openness, feedback and communication about error, non-punitive response to error, unit management support for patient safety, unit handoffs and transitions), two hospital factors (hospital management support for patient safety, hospital handoffs and teamwork across hospital units) and one outcome factor (frequency of event reporting) (see table 2). A total of 17 items of the original 14-factor PaSKI model were excluded due to their low factor loading and high cross-loadings between first and second loading (see online supplemental appendix 1).

Table 2

Factors with loadings for the A-PaSKI alternative model

Item	Factor loading
	1	2	3	4	5	6	7	8	9	10
Factor 1: Hospital handoffs and teamwork across hospital units, Cronbach α=0.733
Problems often occur in the exchange of information across hospital units (h_F7r)	1.024	−0.055	−0.035	−0.019	0.017	0.003	0.000	−0.015	0.031	0.036
It is often unpleasant to work with staff from other hospital units (h_F6r)	0.504	0.087	0.043	0.055	0.000	−0.036	0.020	0.060	−0.056	−0.024
Factor 2: Unit management support for patient safety, Cronbach α=0.836
Unit management considers patient safety when programme changes are discussed (Si2)	0.046	0.709	−0.025	−0.004	−0.001	−0.040	−0.090	−0.028	−0.041	−0.034
Unit management has a clear picture of the risk associated with patient care (Si1)	0.015	0.689	−0.024	0.022	0.005	0.019	0.018	0.053	0.031	−0.005
Unit management provides a work climate that promotes patient safety (u_F1)	0.000	0.534	−0.038	0.102	−0.007	−0.058	−0.070	0.032	−0.189	0.019
The actions of unit management show that patient safety is a top priority (u_F8)	0.010	0.458	−0.037	0.022	0.062	−0.092	−0.030	−0.026	−0.196	0.025
Factor 3: Frequency of event reporting, Cronbach α=0.895
When a mistake is made, but has no potential to harm the patient, how often is this reported? (u_D2)	0.020	0.038	−0.934	−0.010	−0.037	0.008	0.041	−0.003	0.040	−0.001
When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported? (u_D1)	−0.014	0.019	−0.829	0.016	0.040	−0.019	−0.022	−0.047	−0.049	−0.014
When a mistake is made that could harm the patient, but does not, how often is this reported? (u_D3)	−0.009	−0.034	−0.762	−0.003	0.040	−0.034	0.010	0.083	−0.037	0.043
Factor 4: Teamwork within units, Cronbach α=0.765
In this unit, people treat each other with respect (u_A4)	−0.030	−0.031	−0.022	0.840	0.012	−0.008	0.017	0.025	−0.027	−0.031
When a lot of work needs to be done quickly, we work together as a team to get the work done (u_A3)	0.035	−0.017	0.019	0.743	0.027	−0.027	0.013	−0.012	0.033	0.067
People support one another in this unit (u_A1)	0.026	0.079	−0.004	0.587	0.007	0.009	−0.009	0.034	−0.009	0.006
Factor 5: Unit handoffs and transitions, Cronbach α=0.781
Important patient care information is often lost during shift changes (u_F5r – recoded)	−0.046	−0.001	−0.069	0.053	0.843	0.009	−0.031	0.023	0.059	−0.001
Things ‘fall between the cracks’ when transferring patients in the unit (u_F3r)	0.159	−0.010	−0.079	−0.031	0.653	0.045	−0.113	−0.012	−0.077	−0.122
Shift changes are problematic for patients in this unit (u_F11r – recoded)	0.013	0.039	0.071	0.060	0.570	−0.078	0.054	−0.010	−0.061	0.095
Factor 6: supervisor, manager expectations and actions promoting safety, Cronbach α=0.799
My supervisor/manager seriously considers staff suggestions for improving patient safety (u_B2)	0.012	0.037	−0.022	−0.013	−0.008	−0.866	−0.086	−0.019	0.029	−0.054
My supervisor/manager says a good word when he/she sees a job done according to established patient safety procedures (u_B1)	0.018	−0.047	−0.075	0.086	−0.045	−0.666	−0.058	0.028	−0.007	0.003
My supervisor/manager overlooks patient safety problems that happen over and over (u_B4r)	0.084	0.128	0.009	0.014	0.109	−0.521	0.087	0.015	−0.080	0.042
Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts (u_B3r)	−0.016	0.042	0.045	−0.035	0.139	−0.322	0.090	0.144	−0.124	0.143
Factor 7: feedback and communication about error,Cronbach α=0.762
We are given feedback about changes put into place based on event reports (u_C1)	0.002	0.060	0.018	−0.015	0.041	−0.100	−0.669	0.092	−0.073	0.051
We are informed about errors that happen in this unit (u_C3)	0.013	0.116	−0.072	0.007	0.114	−0.061	−0.531	0.074	−0.013	0.167
Factor 8: non-punitive response to error, Cronbach α=0.749
Staff feel like their mistakes are held against them (u_A8r)	0.006	−0.029	−0.007	0.119	−0.005	−0.042	−0.038	0.707	−0.028	−0.073
Staff worry that mistakes they make are kept in their personnel file (u_A16r)	0.057	0.030	−0.026	−0.066	−0.021	0.026	−0.032	0.627	−0.028	0.060
When an event is reported, it feels like the person is being written up, not the problem (u_A12r)	0.027	0.063	−0.042	0.080	0.051	−0.047	−0.032	0.560	−0.034	−0.035
Factor 9: hospital management support for patient safety, Cronbach α=0.870
The actions of hospital management show that patient safety is a top priority (h_F8)	−0.002	0.042	−0.031	0.015	−0.014	−0.013	−0.054	−0.050	−0.822	0.013
Hospital management seems interested in patient safety only after an adverse event happens (h_F9r)	0.044	−0.050	−0.026	−0.047	0.033	0.007	0.030	0.112	−0.750	0.005
Hospital management provides a work climate that promotes patient safety (h_F1)	0.019	0.117	−0.015	0.072	0.002	−0.010	−0.028	−0.009	−0.727	−0.023
Factor 10: communication openness, Cronbach α=0.664
Staff feel free to question the decisions or actions of those with more authority (u_C4)	0.029	−0.056	−0.055	0.096	−0.035	0.027	−0.132	−0.055	−0.022	0.599
Staff are afraid to ask questions when something does not seem right (u_C6r)	0.009	0.108	−0.017	−0.015	0.151	−0.109	0.141	0.246	0.035	0.423
In this unit, we discuss ways to prevent errors from happening again (u_C5)	0.032	0.153	−0.147	0.055	0.016	−0.138	−0.161	0.028	−0.071	0.393

A-PaSKI, Austrian Patient Safety Climate Inventory.

However, two items with a factor loading smaller than 0.4 were accepted, as they were very close to the 0.4 threshold and regarded as critical to the assessment of safety culture. These were ‘in this unit, we discuss ways to prevent errors from happening again’ (u_C5) (loading 0 0.393) and ‘Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts’ (u_B3r) (loading 0 0.322). During the EFA, three factors were excluded in their entirety (overall perceptions of safety, organisational learning—continuous improvement and staffing). Two items (u_A2 and u_A14) from staffing were excluded before the EFA and two more items during the EFA. The factors hospital handoffs and transitions and teamwork across hospital units were incorporated in a different factor in the EFA.

We carried out two CFA to compare the findings from the original PaSKI model with the alternative A-PaSKI model. The CFA for the original PaSKI model could not be calculated for Staffing, as its data was not sufficient. Neither was it possible to use AMOS. As a result, the CFA was calculated for the theoretically developed model without Staffing (13-factor model). This proved to be an adequate model (χ2 (867)=3589.731, p=0.0001)44 47 (see table 3). The χ2 test demonstrated the model scantly acceptable fit, as χ2/df was smaller than 5 (χ2/df=4.140). The two absolute fit indices (RMSEA=0.051, SRMR=0.052) demonstrated a good model fit, while the third fit index showed non-acceptable model fit (GFI=0.874). By contrast, the relative fit indices (CFI=0.899, TLI=0.885) revealed a non-acceptable model fit.

Table 3

Model fit indices or original and alternative A-PaSKI models

Model fit index	Criterion	Original (O)13-factor model	Alternative (A)10-factor model
χ2	/	3589.731	1408.245
df	/	867	360
p	Significant p values expected*	0.000	0.000
χ2/df	<5**	4.140	3.911
CFI	>0.90*	0.899	0.941
TLI	>0.90*	0.885	0.929
RMSEA	<0.07*	0.051	0.049
SRMR	<0.08*	0.052	0.041
GFI	>0.9***	0.874	0.927
AIC		3925.7	1618.2
ΔAIC†	>10****	2307.5
BIC		4781.1	2152.8
ΔBIC‡		2628.3

Threshold values references: *45, **79, ***80, ****55.

†ΔAIC=AICO – AICA.

‡ΔBIC=BICO – BICA.

AIC, Akaike information criterion; A-PaSKI, Austrian Patient Safety Climate Inventory; BIC, Bayesian information criterion; CFI, comparative fit coefficient; GFI, Goodness-of-Fit Index; RMSEA, root mean square error of approximation; SRMR, standardised root mean square residual; TLI, Tucker-Lewis Index.

The CFA for the 10-factor A-PaSKI (χ2 (360)=1408.245, p=0.0001) indicated a better model as opposed to the theoretical PaSKI. The χ2 test was highly significant, but in contrast to the theoretical model, the χ2 test presented a better model fit (χ2/df=3.911). All indices exhibited a better model fit. Moreover, the absolute fit indices showed a better model adjustment (RMSEA=0.049, SRMR=0.041, GFI=0.927)—as did the relative fit indices (CFI=0.941, TLI=0.929).

We carried out AIC and BIC to compare the 10-factor model findings with the 13-factor model. The lower AIC/BIC values of the new 10-factor model showed a better model fit than the theoretical 13-factor models. The difference of over 10 indicates that the 13-factor structure has no empirical support.55

Reliability

The results from the Cronbach’s α for the theoretical PaSKI revealed that only nine factors were above the acceptable value (α≥0.70). For five factors, the values were close to the limit. These results support the Cronbach’s α values from Switzerland, which were similar (see table 4).

Table 4

Internal consistency and construct validity of the original model PaSKI

Factors	# of items AUT	# of itemsCHE	# of itemsGER	Cronbach’s αAUT	Cronbach’s αCHE*	Cronbach’s αGER**
Unit level
Supervisor, manager expectations and actions promoting safety	4	4	3	0.79	0.78	0.75
Organisational learning—continuous improvement	3	3	/	0.67	0.68	/
Teamwork within units	4	4	3	0.65	0.73	0.78
Communication openness	3	3	/	0.64	0.64	/
Feedback and communication about error	3	3	/	0.78	0.79	/
Non-punitive response to error	3	3	3	0.74	0.71	0.73
Staffing	4	3	/	0.66	0.61	/
Unit management support for patient safety	5	4	/	0.87	/	/
Unit handoffs and transitions	4	4	/	0.64	/	/
Hospital level
Hospital management support for patient safety	3	3	3	0.70	0.83	0.83
Teamwork across hospital units	4	4	/	0.72	0.76	/
Hospital handoffs and transitions	2	2	/	0.88	0.71	/
Outcome measures
Overall perceptions of safety	4	4	/	0.86	0.75	/
Frequency of event reporting	3	3	3	0.78	0.88	0.87

Threshold value references: *25, **31

Bold values are data from this study.

AUT, Austria; CHE, Switzerland; GER, Germany; PaSKI, Patient Safety Climate Inventory.

The 10-factor structure’s reliability (Cronbach‘s α) for individual factors lay between 0.664 and 0.895, whereby the factor Communication openness is slightly below a good Cronbach’s α value of 0.7056 (see table 5).

Table 5

Internal consistency and construct validity of alternative model A-PaSKI

Factors	# of items	Cronbach’s α	AVE	FLR
Unit level
Supervisor, manager expectations and actions promoting safety	4	0.799	0.52	1.03
Teamwork within units	3	0.765	0.56	0.68
Communication openness	3	0.664	0.41	1.17
Feedback and communication about error	2	0.762	0.63	0.95
Non-punitive response to error	3	0.749	0.50	0.94
Unit management support for patient safety	4	0.836	0.57	1.11
Unit handoffs and transitions	3	0.781	0.55	0.75
Hospital level
Hospital management support for patient safety	3	0.870	0.69	1.00
Hospital handoffs and teamwork across hospital units	2	0.733	0.59	0.72
Outcome
Frequency of event reporting	3	0.895	0.74	0.66

A-PaSKI, Austrian Patient Safety Climate Inventory; AVE, average extracted variance; FLR, Fornell-Larcker criterion.

Construct validity

We applied AVE and FLR for PaSKI (13-factor model) and A-PaSKI (10-factor model). AVE revealed unacceptable results (AVE ≥0.5) regarding the theoretical 13-factor models construct validity for the following factors: organisational learning—continuous improvement, teamwork within units, communication openness, non-punitive response to error, unit handoffs and transitions, teamwork across hospital units and overall perceptions of safety. The AVE is acceptable for seven factors and unacceptable for the six remaining factors. The FLR results were even worse, as the results were positive (FLR ≤1) for three factors only: teamwork within units, non-punitive response to error, frequency of event reporting.

The alternative 10-factor A-PaSKI’s construct validity shows that the AVE is unacceptable for just one factor, communication openness (AVE ≥0.5), and acceptable for all other factors. The FLR revealed similar values, whereby communication openness and supervisor, manager expectations and actions promoting safety, and unit management support for patient safety had unacceptable values, whereas the other three had positive values (see table 5).

Discussion

This is the first PaSKI validation study in Austria to include all professional groups in a hospital (ie, doctors, nurses, administrative staff, etc). We were able to develop a new 10-factor model, namely the A-PaSKI, as a basis for measuring PSC in Austria. The new 10-factor model with 30 items demonstrated a particularly good model fit with our data.

The model fit (χ2 test) revealed positive results for both models. Moreover, the other individual model indices in the CFA highlighted the alternative model (A-PaSKI). The A-PaSKI also demonstrated better results on the internal consistency, as only factor communication openness in Cronbach’s α was slightly below the acceptable value. In the end, the construct level values confirmed the advantages of the A-PaSKI instead of the theoretically developed PaSKI. However, the construct validity in the new model was unacceptable for the factor communication openness. The only other study focusing on adapting the PaSKI instrument yielded similar results to ours—a good model fit of a different factor structure but low international consistency.25

The HSOPSC original 12-factor structure has so far only been demonstrated in four countries,57 with many other validations in different countries finding a different factor structure.31 58–64 Once again, this highlights the necessity to adapt the HSOPSC to the individual healthcare system and speaks against studies that apply the HSOPSC in different cultures without validation.65–69

Three fundamental factors were excluded (overall perceptions of safety, organisational learning—continuous improvement and staffing). Other international psychometric analyses found identical problems with the factors overall perceptions of safety26 31 59 60 70 and organisational learning—continuous improvement.26 31 70 71 Through EFA, we excluded a total of 17 items from the original Swiss PaSKI, seven of which were phrased in the negative, which may have been why the participants did not understand these items. Therefore, it is recommended to phrase items exclusively in the positive.72 Two items (u_A2 and u_A14) from the factor staffing were excluded in the pretest phase, and two more items (u_A5r and A7r) were excluded in the EFA. The fact that the factor staffing had only two variables during the EFA may be the underlying reason for its exclusion. Also, these variables were hardly loaded, which was also confirmed in the pretest results. This may be because the items were not adapted to the specific linguistic and cultural settings. Other studies, which include the factor staffing, were faced with similar problems.25 31 59–62 73 The factors hospital handoffs and transitions and teamwork across hospital units were included in the EFA in one factor. Germany’s psychometric study also merged those two factors.31 While Nieva and Sorra17 state that none of their factors measure the same construct, this is not the case for the Austrian version A-PaSKI. On the one hand, this could be due to the similarity of questions, and on the other hand, both factors dealt with collaboration between teams and units and were also correlated with others.22 25 57 64 71 74 75

Strengths and limitations

Our extensive and broad sample enabled us to cover a wide range of hospital in Austria. We believe that the new instrument can be applicable throughout Austria for the measurement of PSC. The survey also captured the views of a variety of healthcare professions. The study has several limitations. First, the new survey does not demonstrate good values in terms of construct validity for one factor that may limit the results' interpretation as a whole construct of PSC. Second, we did not examine the link between PSC and objective safety outcomes (eg, medical errors, complication rates or death rate), which will need to be addressed in future studies. Third, the response rate within hospitals was rather low. However, the purpose was to have a large enough sample to assess factor structure; we accept that a higher response rate would be needed for studies assessing and comparing safety cultures. Our sample of health professionals (n=1202) in ten different hospitals and 17 different hospital units was adequate for our psychometric analysis, because our survey included homogeneous populations, that is (persons having some strong group identity) in terms of their attitudes, opinions, perspectives, etc,76 and the sample is comprehensive across several hospitals, units and professional groups. We could confirm with archival analysis that our sample covers all professionals’ groups very well and is therefore well suited for developing the questionnaire. In summary, the wave analysis also confirmed that our sample is acceptable for the study.52

Implications of findings

This study confirms the importance of adapting the instrument to both the healthcare system and a specific country’s language. The validated instrument is the first step in creating a PSC in Austrian hospitals. Austrian hospitals should now use the developed instrument to launch specific activities aiming at improving their safety culture.

Future directions for research

Further research is needed to add new items or rephrase ones for the factors Staffing, Overall perceptions of safety, organisational learning—continuous improvement and hospital handoffs and transitions. The teamwork across hospital units items should be included in more detailed variables, for example, by deriving variables from theoretical team models. In this way, variables regarding goal orientation, task fulfilment, procedures, collaboration, taking on responsibility, etc, can be obtained. Future research should also generate additional variables for the two existing items of the factor hospital handoffs and transitions to become more stable. In addition to internal consistency, future research should also check for retest reliability. Test–retest reliability ensures that the measurements obtained are stable over time. Variables that directly depict patient transfers could be added, for instance, regarding the use of the transfer instrument I-PASS (illness severity, patient summary, action list, situation awareness and contingency plans, and synthesis by receiver).77 AHRQ has promoted the international translation guidelines for HSOPSC. The global outcomes highlight that these guidelines are insufficient, as the results of the original 12-factor model from the US revealed a worse model fit in some other countries. There is a need for more compact guidelines, that is, a standard approach to facilitate transferring an existing instrument to a different language or setting.78

Conclusion

The A-PaSKI demonstrates adequate factors, reliability and validity, indicating one problematic factor. Therefore, we recommend using the A-PaSKI in its full form. Caution should be taken when interpreting the data on the factor communication openness. Thus, A-PaSKI provides added value to an empirical development of this instrument and, subsequently, Austria’s PSC. The resulting survey was more culturally relevant and more suitable to capture the conditions encountered by health professionals in Austria