Hospital Survey on Patient Safety Culture (HSPSC): a systematic review of the psychometric properties of 62 international studies.

OBJECTIVE: To carry out a systematic review of the psychometric properties of international studies that have used the Hospital Survey on Patient Safety Culture (HSPSC).
DESIGN: Literature review and an analysis framework to review studies.
SETTING: Hospitals and other healthcare settings in North and South America, Europe, the Near East, the Middle East and the Far East. DATA SOURCES: A total of 62 studies and 67 datasets made up of journal papers, book chapters and PhD theses were included in the review. PRIMARY AND SECONDARY OUTCOME MEASURES: Psychometric properties (eg, internal consistency) and sample characteristics (eg, country of use, participant job roles and changes made to the original version of the HSPSC).
RESULTS: Just over half (52%) of the studies in our sample reported internal reliabilities lower than 0.7 for at least six HSPSC dimensions. The dimensions 'staffing', 'communication openness', 'non-punitive response to error', 'organisational learning' and 'overall perceptions of safety' resulted in low internal consistencies in a majority of studies. The outcomes from assessing construct validity were reported in 60% of the studies. Most studies took place in a hospital setting (84%); the majority of survey participants (62%) were drawn from nursing and technical staff. Forty-two per cent of the studies did not state what modifications, if any, were made to the original US version of the instrument.
CONCLUSIONS: While there is evidence of a growing worldwide trend in the use of the HSPSC, particularly within Europe and the Near/Middle East, our review underlines the need for caution in using the instrument. Future use of the HSPSC needs to be sensitive to the demands of care settings, the target population and other aspects of the national and local healthcare contexts. There is a need to develop guidelines covering procedures for using, adapting and translating the HSPSC, as well as reporting findings based on its use. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first systematic review of a patient safety culture (PSC) instrument that compares the psychometric properties of a PSC instrument across a range of international studies.

Our findings cast some doubt over the value of using the Hospital Survey on Patient Safety Culture (HSPSC) without prior consideration of the value of adapting it to fit national and local contexts.

Our findings relate to only one PSC instrument (the HSPSC) and do not cover other instruments (eg, Safety Attitudes Questionnaire and Patient Safety Climate in Healthcare Organisations (PSCHO)).

The diversity in study methodology and reporting of studies using the HSPSC means that firm conclusions about the reliability and validity of the instrument are difficult to draw.

Introduction

Over the course of the last few decades, the field of patient safety has expanded and evolved in a number of directions. Early work concentrated on understanding the extent and the types of human error, which were contributing to patient harm.1 2 More recently, the focus has shifted towards understanding the role played by human and organisational issues such as leadership, teamwork and communication in contributing to, as well as preventing, adverse incidents.3 4 One prominent organisational factor, safety culture, has in particular generated much interest and discussion.5–8 Use of the term ‘safety culture’ first came about as a result of the Chernobyl nuclear accident9 and has since been used as a way of understanding accidents in a wide variety of industries including aviation, oil and gas and most recently healthcare.10 11 Safety culture is typically taken to refer to ‘the attitudes, beliefs, perceptions and values that employees share in relation to safety in the workplace’.12 Safety culture as a construct is related to ‘safety climate’ and is typically associated with ‘the underlying assumptions and values that guide behaviour in organisations’, whereas safety climate focuses on ‘the direct perceptions of individuals’ of the underlying culture.13

Patient safety culture

Within healthcare, the design of survey instruments and other methods for measuring patient safety culture (PSC) has expanded considerably in the past few years.6 10 14 In Europe, for example, the European Network for Patient Safety Project15 identified 19 different survey instruments and other methods in use throughout the European Union (EU) member states until 2010. At that time, the most frequently used were the Hospital Survey on Patient Safety Culture (HSPSC,16 used in 12 EU member states), the Manchester Patient Safety Framework17 (used in 3 EU member states) and the Safety Attitudes Questionnaire18 (used in 4 EU member states). A similar review identified 26 studies within Europe, which had used the HSPSC instrument between January 2003 and February 2012.19 In the USA, the Middle East and the Far East, there is likewise evidence of a widening interest in measuring safety culture.20–24

Psychometric properties of PSC instruments

While increased efforts to measure PSC have been welcomed,4 6 others have suggested the need for a degree of caution. More than a decade ago, Flin et al argued in a series of articles25 26 that the psychometric properties of PSC instruments demonstrated poor levels of reliability and validity. More recently, Pronovost and Sexton27 warned that ‘the enthusiasm for measuring culture may be outpacing the science’. One explanation for low levels of reliability and validity may be the different interpretations of what constitutes a positive PSC among the diverse occupational groupings and subcultures involved in healthcare delivery.28–30 Previous work, for example, has shown that physicians report higher levels of PSC when there is good teamwork across units and a unit-level management that promotes PSC.31 Nursing staff, by contrast, are more likely to report higher levels of PSC safety when they feel that there is enough staff in the unit.31–33 In addition to these concerns, the psychometric properties of PSC instruments may be affected by other factors such as national culture.34 The extent to which healthcare workers are likely to report incidents and errors, for example, has been shown to be related to cultural norms such as fear of ‘losing face’ and the desire to avoid uncertainty and ambiguity,35 characteristics that are often associated with Eastern as compared with Western national cultures.36–38

Motivation for the review and aims

The use of a PSC instrument requires significant investment of time and other resources by healthcare providers. In addition, the outcomes from using the instrument often form the basis with which to identify areas for improvement and intervention within hospitals and other healthcare settings. In other cases, healthcare organisations sometimes use the findings to benchmark levels of PSC across units, departments and hospitals.18 It is therefore very important that these instruments demonstrate acceptable levels of reliability and validity when they are used.

Currently, there are no examples of systematic reviews of the psychometrics of PSC instruments. Hammer et al 19 published a review of the psychometrics of the HSPSC, but this was limited to Europe. The aim of this review was therefore to fill this gap in knowledge and to carry out a review of the international use of psychometric properties of one of the most well-known and widely used surveys on PSC, the HSPSC instrument.19 39 In addition, the review examined how the psychometric properties of the HSPSC varied across a range of different types of sample characteristics (eg, country of use, healthcare setting and participant job roles) in order to identify any trends or patterns in the data.

Methods

Literature search

A literature search for publications that had used the HSPSC was conducted using the Scopus, Web of Science, PubMed and PsycINFO electronic databases. The literature search was initially carried out by the lead author and then checked and updated by the second author. The outcomes from subsequent updates of the literature search were checked by the other authors. The search terms used were ‘HSPSC’ and the following acronyms and terms that have been used to describe the survey, that is, ‘HSPSC’, ‘HSOPSC’, ‘HSOPS’, ‘Survey on Patient Safety Culture’ and ‘SOPS’. All of these search terms were used in conjunction with the Boolean operator ‘OR’. As the original report on the psychometric properties of the HSPSC was published in 2004, the search was limited to studies published between 2004 and July 2018. In addition, citations and references to other publications describing use of the HSPSC were consulted, as well as a list of national contacts provided by the statistical services company (Westat–Agency for Healthcare Research and Quality (AHRQ)40), which provides support to the AHRQ and their work with the HSPSC. Online supplementary file 1 provides details of the full search strategy for the Scopus database.

Inclusion and exclusion criteria

We included all internationals studies that published psychometric properties on healthcare professionals’ perception of PSC measured with the HSPSC. To be eligible for inclusion in the review, studies had to be published as journal papers, book chapters or reports that are in the public domain (eg, PhD theses) and written in the English language. A wide variety of study designs were included in the review (eg, studies comparing datasets from two or more countries or regions, studies conducted in one country). The original report16 outlining the details of the HSPSC and the process used to compile the survey was also included in the review.

Framework for analysis

An analysis framework (figure 1) was developed in order to systematically compare (1) psychometric properties of the HSPSC, including internal reliability and the use of factor analysis to assess construct validity; and (2) characteristics of use, including country of use, sample characteristics (setting, participant job roles, sample size and response rate) and adaptations to the original US version of the HSPSC16 (language, procedure for translation and other changes to the original version of the survey). The framework was guided by previous research examining psychometric properties of PSC surveys.19 26 41

Figure 1

Analysis framework. HSPSC, Hospital Survey on Patient Safety Culture.

Patient and public involvement

No patients or members of the general public were involved in this study.

Findings

Search results

The initial search yielded a total of 8604 studies of potential relevance from the four databases. Sixty-two studies met the inclusion criteria and reported psychometric properties on the HSPSC for a total of 67 datasets (one study used the survey in two different countries,35 two studies used the survey in two different languages in the same country42 43 and two studies used the same survey over two different time periods44 45). The screening and filtering process of the database results is shown in figure 2.

Figure 2

Screening and filtering process for the database results.

Psychometric properties

Internal reliabilities

Table 1 shows the values of internal reliabilities using Cronbach’s α coefficient for 67 datasets from the 62 studies. An acceptable value of Cronbach’s α is typically taken as 0.70–0.9046–48; however, the first psychometric testing of the HSPSC16 considered levels of α≥0.60 as acceptable. Eleven studies did not report values of Cronbach’s α for one or more of the dimensions in the HSPSC. Of the remaining 769 values of Cronbach’s α given in table 1, there were 355 instances of HSPSC subscales (46.16% of all values) where the internal reliability reported was lower than a Cronbach’s α value of 0.70. The internal reliability for the dimension ‘staffing’ obtained an acceptable level (eg, Cronbach’s α score >0.70) for only 8 of the 62 datasets that reported this dimension. The dimension ‘frequency of events’ was above the level of acceptability for 61 of the 65 datasets that reported the internal reliability for this item. Thirty-five datasets reported Cronbach’s α values of <0.70 for 6 or more of the 12 dimensions in the HSPSC. Two datasets, from Lebanon21 and Turkey,49 reported values of α<0.70 for 10 dimensions, and another used in Oman50 reported values of α<0.70 for 11 dimensions.

Table 1

Internal consistencies of the 67 datasets in which the HSPSC was used, from the 62 articles

Authors	Country	Language	Hospital management support for patient safety	Supervisor/manager expectations/actions	Teamwork across hospital units	Teamwork within units	Communication openness	Hospital handoffs and transitions	Non-punitive response to error	Feedback and communication about error	Staffing	Organisational learning	Overall perceptions of safety	Frequency of event reporting
North and South America
Sorra and Nieva12 16	USA	English	0.83	0.75	0.80	0.83	0.72	0.80	0.79	0.78	0.63	0.76	0.74	0.84
Sorra and Dyer68	USA	English	0.79	0.79	0.79	0.83	0.73	0.81	0.78	0.78	0.62	0.71	0.74	0.85
Blegen et al 44	USA	English	0.70	0.65	0.68	0.77	0.64	0.76	0.72	0.82	0.44	0.64	0.53	0.83
			0.74	0.69	0.74	0.80	0.58	0.81	0.76	0.81	0.57	0.67	0.61	0.81
Castle69	USA	English	0.81	0.67	0.74	0.75	0.72	0.77	0.70	0.79	0.71	0.75	0.74	0.62
Castle and Sonon70	USA	English	0.66	0.62	0.74	0.80	0.71	0.75	0.72	0.70	0.81	0.77	0.78	0.71
Erler et al 71	USA	English	–	0.83	–	0.80	0.68	–	–	–	–	–	0.84	0.90
Marsteller et al 72	USA	English	0.85	0.71	0.81	0.78	0.68	0.82	0.68	0.73	0.80	0.76	0.75	0.86
Wagner et al 73	USA and Canada	English	0.84	0.81	0.84	0.79	0.74	0.83	0.79	0.78	0.70	0.69	0.76	0.80
Tomazoni et al 74	Brazil	Portuguese	0.60	0.74	0.60	0.61	0.64	0.64	0.47	0.72	0.46	0.74	0.43	0.88
Silva-Batalha and Melleiro75	Brazil	Portuguese	0.72	0.77	0.56	0.68	0.63	0.70	0.40	0.67	0.66	0.60	0.47	0.87
Andrade et al 76	Brazil	Portuguese	0.74	0.73	0.67	0.70	0.59	0.73	0.53	0.74	0.59	0.61	0.63	0.87
Europe
Vlayen et al 42	Belgium	Dutch	0.72	0.77	0.66	0.66	0.65	0.71	0.68	0.78	0.57	0.59	0.58	0.85
		French	0.77	0.75	0.68	0.82	0.72	0.72	0.64	0.76	0.52	0.59	0.63	0.87
Vlayen et al 43	Belgium	Dutch	0.72	0.77	0.69	0.65	0.66	0.70	0.70	0.76	0.55	0.50	0.54	0.85
		French	0.79	0.74	0.66	0.84	0.71	0.70	0.68	0.70	0.50	0.58	0.58	0.84
Stoyanova et al 77	Bulgaria	Bulgarian	0.86	0.81	0.65	0.63	0.77	0.92	0.75	0.63	0.33	0.61	0.34	0.86
Brborović et al 78	Croatia	Croatian	0.73	0.79	0.79	0.76	0.64	0.64	0.62	0.74	0.35	0.53	0.57	0.91
Boussat et al 79	France	French	0.66	0.81	0.56	0.71	0.63	0.60	0.60	0.66	0.60	0.62	0.65	0.84
Occelli et al 80	France	French	0.73	0.83	0.59	0.63	0.62	0.66	0.57	0.64	0.46	0.59	0.67	0.84
Kuosmanen et al 81	Finland	Finnish	0.79	0.78	0.72	0.81	0.52	0.71	0.72	0.67	0.62	0.60	0.66	0.67
Turunen et al 82	Finland	Finnish	0.83	0.75	–	–	–	–	0.79	0.78	–	–	0.74	0.84
Hammer et al 83	Germany	German	0.87	0.69	0.78	0.77	0.64	0.83	0.73	0.79	0.73	0.61	0.73	0.86
Gambashidze et al 29	Germany	German	0.83	0.75	0.79	0.78	–*	–†	0.73	0.83	0.79	–‡	–‡	0.87
Kapaki and Souliotis 84	Greece	Greek	0.79	0.84	0.88	0.80	0.77	–¶	0.71	–**	0.60	0.60	–††	0.82
Tereanu et al 85	Italy	Italian	0.82	0.75	0.70	0.85	0.79	0.77	0.71	0.76	–‡‡	0.74	–	0.93
Smits et al 86	The Netherlands	Dutch	0.68	0.70	0.72	0.66	0.72	0.65	0.69	0.78	0.58	–§§	0.64	0.79
Verbeek-Van Noord et al 87	The Netherlands	Dutch	0.72	0.78	0.78	0.60	0.67	0.66	0.55	0.75	0.47	–¶¶	0.67	0.81
Olsen88	Norway	Norwegian	0.79	0.77	0.65	0.77	0.68	0.65	0.64	0.70	0.65	0.51	0.76	0.82
Olsen and Aase 89	Norway	Norwegian	0.78	0.76	0.68	0.74	0.65	0.68	0.67	0.69	0.68	0.53	0.75	0.75
Haugen et al 90	Norway	Norwegian	0.80	0.85	0.73	0.75	0.67	0.68	0.68	0.73	0.59	0.64	0.78	0.82
Storm et al 45	Norway	Norwegian	0.74	0.79	0.62	0.83	0.46	0.69	0.73	0.63	0.63	0.67	0.68	0.70
	Norway	Norwegian	0.84	0.74	0.71	0.84	0.70	0.71	0.62	0.79	0.61	0.72	0.74	0.69
Eiras et al 52	Portugal	Portuguese	0.77	0.72	0.69	0.73	0.67	0.71	0.57	0.76	0.48	0.71	0.62	0.90
Tereanu et al 91	Romania	Romanian	0.73	0.78	0.74	0.75	0.83	–***	0.68	–†††	–‡‡‡	0.56	–	0.93
Sarac et al 92	UK	English	0.79	0.79	0.70	0.80	0.73	0.74	0.77	0.78	0.60	0.64	0.71	0.84
Waterson et al 93	UK	English	0.69	0.68	0.70	0.73	0.67	0.77	0.65	0.80	0.58	0.66	0.67	0.83
Robida 94	Slovenia	Slovene	0.82	0.74	0.74	0.74	0.74	0.66	0.61	0.72	0.65	0.36	0.65	0.88
Robida 95	Slovenia	Slovene	0.81	0.75	0.72	0.76	0.73	0.69	0.60	0.75	0.63	0.65	0.62	0.89
Hedsköld et al 96	Sweden	Swedish	0.81	0.78	0.72	0.76	0.68	0.74	0.74	0.76	0.67	0.66	0.71	0.87
Nordin et al 97	Sweden	Swedish	0.82	0.75	0.68	0.67	0.62	0.72	0.74	0.72	0.60	0.61	0.74	0.87
Danielsson et al 98	Sweden	Swedish	0.81	0.82	0.74	0.79	0.68	0.77	0.76	0.78	0.73	0.73	0.79	0.86
Pousette et al 99	Sweden	English	0.78	0.76	0.65	0.74	0.67	0.69	0.70	0.68	–§§§	0.62	–	–
Perneger et al 100	Switzerland	French	0.79	0.75	0.71	0.80	0.67	0.78	0.60	0.77	0.61	0.57	0.68	0.86
Pfeiffer and Manser63	Switzerland	German	0.83	0.78	0.76	0.73	0.64	0.71	0.71	0.79	0.61	0.68	0.75	0.88
Near and Middle East
Moghri et al 101	Iran	Farsi	0.79	0.80	0.77	0.68	0.68	0.76	0.77	0.74	0.64	0.70	0.57	0.78
Khater et al 23	Jordan	Arabic	0.60	0.54	0.61	0.73	0.46	0.71	0.61	0.70	0.41	0.52	0.50	0.78
Saleh et al 102	Jordan	English	0.77	0.67	0.66	0.82	0.64	0.71	0.63	0.73	0.77	0.60	0.65	0.73
Suliman et al 103	Jordan	Arabic	0.72	0.63	0.61	0.73	0.75	0.69	0.63	0.61	0.72	0.68	0.58	0.79
El-Jardali et al 21	Lebanon	Arabic	0.63	0.57	0.69	0.68	0.46	0.74	0.53	0.65	0.48	0.50	0.45	0.81
Al-Mandhari et al 50	Oman	English	0.59	0.53	0.63	0.53	0.59	0.63	0.60	0.47	0.41	0.54	0.51	0.78
Najjar et al 104	Palestine	Arabic	0.66	0.75	0.61	0.77	0.41	0.73	0.59	0.69	0.65	0.79	0.43	0.87
Hamdan and Saleem105	Palestine	Arabic	0.76	0.76	0.61	0.77	0.38	0.72	0.57	0.69	0.62	0.79	0.67	0.86
Aboshaiqah106	Saudi Arabia	English	0.59	0.61	0.62	0.72	0.53	0.70	0.61	0.76	0.34	0.64	0.15	0.87
El-Jardali et al 107	Saudi Arabia	Arabic	0.58	0.44	0.64	0.81	0.54	0.77	0.64	0.79	0.24	0.70	0.21	0.89
Alonazi108	Saudi Arabia	English	0.65	0.75	0.69	0.75	0.67	0.59	0.72	0.74	0.57	0.63	0.31	0.83
Alswat et al 109	Saudi Arabia	Arabic	0.52	0.40	0.63	0.76	0.53	0.78	0.69	0.73	0.21	0.61	0.23	0.90
Günes et al 49	Turkey	Turkish	0.65	0.68	0.65	0.74	0.54	0.63	0.55	0.69	0.54	0.56	0.54	0.91
Bodur and Filiz 110	Turkey	Turkish	0.59	0.67	0.73	0.84	0.67	0.70	0.13	0.82	0.02	0.56	0.43	0.93
Bodur and Filiz 111	Turkey	Turkish	–¶¶¶	0.72	0.80	0.83	–****	0.72	0.71	0.81	0.63	0.68	0.57	0.86
Far East
Nie et al 112	China	Chinese	0.67	0.51	0.63	0.72	0.47	–††††	0.75	0.64	0.63	0.74	0.64	–
Shu et al 113	China	Chinese	0.54	0.63	0.76	0.79	0.48	0.44	0.49	0.69	0.26	0.48	0.35	0.89
Wang and Tao 114	China	Chinese	0.61	0.64	0.62	0.67	0.79	0.68	0.48	0.61	0.31	0.46	0.87	0.80
Ito et al 115	Japan	Japanese	0.61	0.70	0.70	0.83	0.62	0.73	0.71	0.77	0.46	0.65	0.62	0.88
Fujita et al 35	Japan	Japanese	0.62	0.70	0.70	0.82	0.64	0.74	0.71	0.78	0.47	0.65	0.63	0.88
	Taiwan	Mandarin	0.59	0.65	0.72	0.77	0.54	0.72	0.56	0.46	0.58	0.63	0.26	0.83
Chen and Li 20	Taiwan	Chinese	0.70	0.73	0.69	0.78	0.51	0.76	0.70	0.36	0.51	0.68	0.52	0.53
Descriptive Statistics
	Minimum		0.52	0.40	0.56	0.53	0.38	0.44	0.13	0.36	0.02	0.36	0.15	0.53
	Maximum		0.87	0.85	0.88	0.85	0.83	0.92	0.79	0.83	0.81	0.79	0.87	0.93
	Mean		0.73	0.72	0.70	0.75	0.64	0.72	0.65	0.72	0.56	0.63	0.60	0.83
	SD		0.09	0.09	0.07	0.07	0.10	0.07	0.11	0.09	0.15	0.09	0.16	0.07
	<0.7		21	20	32	13	46	19	37	19	54	47	43	4
	Percentage <0.7		32.31	29.85	49.23	19.70	71.88	31.15	56.06	29.69	87.10	75.81	69.35	6.15
	<0.6		7	6	3	1	17	2	14	3	31	18	24	1
	Percentage <0.6		10.77	8.96	4.62	1.52	26.56	3.28	21.21	4.69	50.00	29.03	38.71	1.54

Bold values are lower than a value of 0.7 Cronbach alpha.

*The dimensions ‘communication openness’ and ‘feedback and communication about error’ were combined into one factor.

†The dimensions ‘hospital handoffs and transitions’ and ‘teamwork across hospital units’ were combined into one factor.

‡The dimension ‘organisational learning’ was omitted.

§The dimensions ‘staffing’ and ‘overall perceptions of safety’ were combined into one factor.

¶The dimensions ‘hospital handoffs and transitions’ and teamwork across hospital units’ were combined into one factor.

**The dimensions ‘communication openness’ and ‘feedback and communication about error’ were combined into one factor.

††The dimension ‘overall perceptions of safety’ was omitted.

‡‡The dimensions ‘staffing’ and ‘overall perceptions of patient safety’ were omitted.

§§The dimensions ‘organisational learning’ and ‘feedback and communication about error’ were combined into one factor.

¶¶This dataset used the tool translated by Smits et al 86 where the dimensions ‘organisational learning’ and ‘feedback and communication about error’ were combined into one factor.

***As the dimensions ‘handoffs and transitions’ had only two items, the internal reliability was not determined for this dimension.

†††The dimensions ‘feedback and communication about error’ and ‘communication openness’ were combined into one factor named ‘communication’.

‡‡‡The dimensions ‘staffing’ and ‘overall perceptions of patient safety’ were omitted.

§§§The dimensions ‘staffing’, ‘overall perceptions of safety’ and ‘frequency of events reporting’ were omitted.

¶¶¶The dimension ‘hospital management support for patient safety’ was grouped with the dimension ‘teamwork across hospital units’.

****The dimension ‘communication openness’ was grouped with the dimension ‘feedback and communication about error’.

††††The dimensions ‘frequency of events reporting’ and ‘handoffs and transitions’ were omitted.

HSPSC, Hospital Survey on Patient Safety Culture.

Five dimensions demonstrated consistently weak internal reliability (ie, mean α<0.70) across the 67 HSPSC datasets. The dimension staffing was the weakest (mean α=0.56). This was followed by ‘overall perceptions of safety’ (43 datasets, where α<0.7, mean α=0.60), ‘organisational learning’ (47 datasets, where α<0.7, mean α=0.63), ‘communication openness’ (46 datasets where α<0.70, mean α=0.64) and ‘non-punitive response to error’ (37 datasets, where α<0.7, mean α=0.65).

Construct validity using factor analysis

Of the 62 articles, 37 (59.68%) tested construct validity using factor analysis. For a total of 16 articles, both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) had been conducted. For another 3 datasets, EFA had been carried out, and in 18, CFA had been conducted. EFA provides information about the optimal number of factors, whereas CFA determines how well the original 12 factorial structures developed by Sorra and Nieva16 or alternatives fit the dataset. The appropriateness of the CFA model was assessed by measures of global and local fit.46 47 51 The report by Sorra and Nieva16 is one of the 16 articles where both EFA and CFA had been conducted, resulting in a factorial structure of 12 factors. In 11 datasets, the original 12 HSPSC dimensions were confirmed. In six datasets, two factors were combined to create one. One dataset52 based on the CFA resulted in a good overall fit. Online supplementary file 2 provides further details of the use of factor analysis to assess the construct validity for our sample.

Sample characteristics

Country of use

The study sample consisted of a total of 30 different countries. Eleven studies reported the use of the survey in North America (n=8) and South America (n=3), 30 in Europe, 15 in the Near East and the Middle East, and 7 in the Far East. The growth of the use of the HSPSC from 2004 to 2018 across the world is shown in figure 3.

Figure 3

Growth of use of the HSPSC from 2004 to 2018 across world regions of the studies included in our sample. HSPSC, Hospital Survey on Patient Safety Culture.

Setting and participant job roles

The setting within which the survey was used was predominantly hospitals, with 47 of the 62 published studies (75.81%) using the survey in acute general, public, private, psychiatric, community, and secondary and tertiary hospitals. Of the remaining 15 studies, 3 used the survey in long-term care settings. Four studies sampled specific job roles (eg, nursing staff and medical directors of hospitals). The remaining eight studies used the survey within different specialist hospital units or areas (eg, emergency, critical care transport and neonatal intensive care). The job roles and clinical backgrounds of participants across the studies (table 2) indicated that the majority of participants worked as nursing staff or healthcare assistants (mean percentage=50.77%±13.05% across the studies), followed by medical and technical staff (mean percentage=11.67%±7.73%), physicians (mean percentage=9.56%±6.44%), and managers and administrative staff (mean percentage=7.47%±6.37%).

Table 2

Weighted mean percentage of the different participant roles and the weighted SDs across all 67 datasets from the 62 studies

Participant role	Mean (%)	SD (%)
Nurse/health assistant	50.77	13.05
Medical/technical staff	11.67	7.73
Physicians	9.56	6.44
Manager/administrative staff	7.47	6.37
Others	6.73	4.64
Patient care assistant/hospital aide/care partner	6.58	9.89
No answer/missing	3.44	6.09
Pharmacist	1.16	1.33

Sample size, response rate and adaptations to the original version of the HSPSC

Sample sizes varied across the studies. In total, data were reported from 408 563 respondents from 1119 healthcare organisations.16 Site sample size ranged from 331 hospitals to studies carried out within a single site. Most studies (85.48%) did not identify the number of hospital units in which respondents worked. Sample sizes ranged from 51 to 111 478 individuals, with a median of 1026. Response rates ranged from 17% to 98% (mean=38.94%±29.54%). Of the 62 studies, the HSPSC instrument was applied in 20 different languages. Vlayen et al 42 43 used a French and a Dutch translation. One study35 used the HSPSC in Japan and Taiwan, translating the survey into Japanese and Mandarin. From the 67 datasets, 16 obtained results through the use of an English version of the survey. Another 27 datasets required the HSPSC to be translated into a language other than English using several different translation procedures, and 24 datasets used the survey translated in previous studies. Two datasets used the original AHRQ translation guidelines. Fourteen datasets reported using forward–backward translation. Twelve datasets reviewed and verified the translation using an independent expert group. Five datasets made no modifications to the original version of the instrument, and 28 datasets (41.79%) did not state whether modifications were made. Online supplementary files 3 and 4 provide details of the sample sizes, response rates and adaptations to the HSPSC in the 67 datasets.

Discussion

Implications of the review for users of the HSPSC

A clear implication of our review is that researchers and healthcare practitioners should adopt a degree of caution when using the original, unmodified version of the HSPSC16 to measure PSC. Nearly 50% of items from the 12 HSPSC dimensions in our sample demonstrate weak levels of reliability. In some cases, the internal reliabilities of specific dimensions (eg, staffing and overall perceptions of safety) were very weak. By contrast, the internal reliabilities for other dimensions (eg, ‘teamwork across/within units’) were stronger. While most studies (59.68%) that used CFA found a fit with the original 12-factor model,16 inconsistencies in carrying out assessments of construct validity (eg, testing EFA models using CFA and carrying out cross validation) make it very difficult to draw firm conclusions about the validity of the HSPSC. This difficulty is compounded by the fact that only 43% of the datasets in our sample report the outcomes from using factor analysis to assess construct validity. It has been 15 years since the original report describing the HSPSC instrument was released by AHRQ, and one recommendation from our review is that many of the instrument’s dimensions and items need to be revised. Further development and revision of the HSPSC might result in improved psychometric properties and increase its suitability for use across a range of international settings and contexts.

The need for guidance covering adaptations and other changes to the HSPSC

Just under a quarter of the studies in our sample made changes to the original survey items and/or dimensions. Over a third of the sample did not state what modifications, if any, were made to the original US version of the survey. It is well known that changing items and dimensions, alongside other variations in procedure (eg, procedures used to translate the HSPSC and reverse scoring questions), are likely to impact on the psychometric properties of the instrument.53 54 Alterations to the HSPSC also make it difficult to obtain valid measurements and to carry out cross-national comparisons. We would recommend scientific support and guidance for users of the HSPSC covering necessary adaptations in survey items to meet language requirements of different national, regional and healthcare contexts.55 56 Some guidelines already exist but are limited in coverage (eg, focusing solely on translation57) and could be expanded to consider, among other things, advice regarding changes to items and dimensions and the reporting of findings. In addition, the development of guidelines covering the reporting of data from PSC surveys might make it easier for healthcare managers and decision makers to benchmark data within their organisations in order to promote organisational learning processes.

Educational support, resources and the development of standards

We also found a lot of variation across the studies with regard to the reporting of psychometric data, particularly the outcomes of using factor analysis. While recognising that this is something of a ‘grey area’ (eg, agreement with regard to CFA fit indices) and that the analysis and interpretation of psychometric properties require some skills and knowledge, there is room for further improvement in the future, particularly in terms of the provision of educational resources and the development of standards and guidance covering criteria and levels of acceptability for psychometric data. One way forward may be to work towards a set of networks for sharing, comparing and benchmarking the results from using the HSPSC and other PSC instruments and tools. Some of these networks already exist (eg, the web resources provided for the HSPSC by AHRQ) but may be expanded in the same way that other types of collaborative networks and ‘observatories’ work in other domains of safety (eg, road safety58) to include protocols and standards for data collection, analysis and reporting,55 as well as other educational and training resources.

Implications of the review for researchers

The field of PSC is relatively new in comparison to the range of other scientific disciplines that have examined safety and organisational culture over the last 30 or so years (eg, safety science, human factors and organisational psychology59 60). Likewise, the use of safety culture instruments is well established in a variety of industrial sectors outside of healthcare (eg, nuclear energy, aviation and construction). There is much that could be learnt both in terms of theory and methodology from this large body of knowledge and experience. We would point to three specific areas for the future. First, there is a need to understand the influence of a range of other influences on PSC, including the role played by national culture and professional subculture. Wagner et al,61 for example, found clear differences between the Netherlands, the USA and Taiwan in terms of responses to some HSPSC dimensions (eg, communication openness and non-punitive response to error). The large variability in terms of the reliability and validity of the HSPSC across our sample might be due to the influence of professional groups and other subcultures within healthcare.30 62 Many HSPSC items may be very differently interpreted and may elicit very different opinions among different healthcare groups (eg, nursing staff, physicians and managers31 63).

Second, there is a need to expand the range of theoretical constructs that influence and shape PSC. Research on, for example, the relationship between ‘speaking up’ and ‘voice’, levels of management control and PSC is starting to appear64 65; however, there is potential for expansion. Similarly, research on the extent of agreement on levels of PSC among healthcare units and groups (‘climate strength’56) is starting to appear. A range of other constructs influencing PSC, which are well established in other disciplines (eg, organisational psychology59), including leadership styles and organisational justice, might also form part of an agenda for future research. Finally, we know very little about the levels of change in PSC over time, how these levels may decline, improve or stagnate, and what triggers these changes.

Thirdly, we would point to the value of using mixed methods to measure PSC as a means of addressing and taking into account the important role played by diverse social, cultural and subcultural contexts within different healthcare settings. Combining survey data with data drawn from interviews or focus groups, for example, is currently under-represented in the literature on PSC,26 despite being relatively common in studies of PSC in other safety-critical industries (eg, aviation, where interviews and focus groups are often used as a means to check and validate other data, including those from survey instruments66).

Limitations of the review

Our review is the first to systematically examine the psychometric properties of a PSC instrument across a broad range of international contexts; however, it focuses on only one of several instruments currently in use (eg, refs 18 67). We would point to the need for similar systematic reviews of the psychometric properties of these instruments to be carried out in the future. Finally, the diverse ways in which studies in our sample reported details of the procedure, which was used to administer the instrument, as well as how they reported findings from its use, made it difficult to derive firm conclusions about the reliability and validity of the HSPSC.