Linguistic Validation and Cultural Adaptation of Bulgarian Version of Hospital Survey on Patient Safety Culture (HSOPSC).

BACKGROUND: Patient safety (PS) is one of the essential elements of health care quality and a priority of healthcare systems in most countries. Thus the creation of validated instruments and the implementation of systems that measure patient safety are considered to be of great importance worldwide. AIM: The present paper aims to illustrate the process of linguistic validation, cross-cultural verification and adaptation of the Bulgarian version of the Hospital Survey on Patient Safety Culture (B-HSOPSC) and its test-retest reliability.
METHODS: The study design is cross-sectional. The HSOPSC questionnaire consists of 42 questions, grouped in 12 different subscales that measure patient safety culture. Internal con-sistency was assessed using Cronbach's alpha. The Wilcoxon signed-rank test and the split-half method were used; the Spear-man-Brown coefficient was calculated.
RESULTS: The overall Cronbach's alpha for B-HSOPSC is 0.918. Subscales 7 Staffing and 12 Overall perceptions of safety had the lowest coefficients. The high reliability of the instrument was confirmed by the Split-half method (0.97) and ICC-coefficient (0.95). The lowest values of Spearmen-Broun coefficients were found in items A13 and A14.
CONCLUSION: The study offers an analysis of the results of the linguistic validation of the B-HSOPSC and its test-retest reliability. The psychometric characteristics of the questions revealed good validity and reliability, except two questions. In the future, the instrument will be administered to the target population in the main study so that the psychometric properties of the instrument can be verified.

Introduction

Patient safety (PS) is a key determinant of healthcare quality in medical facilities and is considered a priority in most countries [1] [2] [3]. The World Health Organization views PS as a global issue and has established a World Alliance for Patient Safety to promote international cooperation and facilitate the process of improving PS worldwide [4].

Studies from all over the world have documented that between 4% and 16% of all hospitalised patients are victims of medical errors, which are preventable to a great extent [5] [6]. Medical services are complex, specific, and not always predictable. Even if doctors and other care providers have proven medical expertise, and even if all applicable rules and procedures have been adhered to, undesirable event or complications could occur [7] [8].

To ensure that PS standards are followed and to establish breaches of these standards, the medical staff should be encouraged to share information, regarding PS. A step forward in enhancing PS is the ISO 9001 certification of hospitals [9].

It has been proven by the World Alliance for Patient Safety that the instruments to measure improvements in medical practice are constantly evolving and advancing [10]. The Institute of Medicine (IOM) declares that patient safety is “indistinguishable from the delivery of quality healthcare” and therefore encourages the use of patient safety reporting systems (PSRS). The latter is very useful in evaluating the causal factors of harm to patients by medical care [11] [12]. Another issue of great importance is the elaboration of validated instruments and systems to measure PS in medical practice worldwide [1]. Experts from all over the globe are focused on creating instruments, able to measure the level of PS safety and to register adverse events/incidents or errors in medical practice [3]. The objective of collecting and analysing data is to ensure continuous learning and improvement quality of medical care. Unfortunately, on a global scale, the spectrum of validated instruments for measuring PS in hospital settings is limited. Moreover, presently, in Bulgaria, no such tools have been introduced and functioning [3].

In our country, in the specialised literature on PS, a single instrument has been described; however, collected data applying it are not comparable to those, from other countries [13].

An important contribution to the evaluation of hospital patient safety culture and quality assurance is the HSOPSC questionnaire, elaborated by the Agency for Healthcare Research and Quality (AHRQ). The questionnaire is based on selected reliable psychometric characteristics and has been validated in more than 20 countries [1] [2]. It contains 42 questions, grouped in 12 different sub-scales and allows reporting and registration of undesirable events and errors. It also provides information about some social and demographic factors as employment place and work position of the respondents. From 2007 to date, international databases offer annually, survey data from more than 600 hospitals in the USA [14].

The present paper aims to illustrate the process of linguistic validation and cultural adaptation of the Hospital Survey on Patient Safety Culture (HSOPSC) to Bulgarian healthcare settings and its test-retest reliability.

Participants and Methods

The study design is cross-sectional. The translation, linguistic validation, and cultural adaptation of the B-HSOPSC questionnaire were carried out within the frame of the Medical University - Plovdiv’s project №11/2016, named: “Introduction a web-based platform for registration and evaluation of hospital patient safety culture and conduction of a representative study for the country.” Ethical approval was obtained from the University Research Ethics Committee (№ 05/19.10.2017). Each stage of the recommended protocol is presented in Figure 1 and is described in details below. The English source version of the HSOPSC questionnaire was translated into Bulgarian (forward translation) by two independent translators (experienced healthcare professionals), with Bulgarian as a native language and excellent users of English as a second language.

Figure 1

Stages of cross-cultural adaptation of B-HSOPSC questionnaire

The translators provided written reports on the decision-making process, the linguistic difficulties, and the encountered problems. The two Bulgarian translations were compared, and a synthesised version was created at a consensus meeting. Additionally, the Bulgarian text was compared with another two translated versions in Slavic languages (the language of both, the Macedonian and the Croatian version is very similar to the target language-Bulgarian). At the next stage, the synthesised version was back-translated into English by two professional translators, who had no access to the original English version of the questionnaire.

Cognitive interviews were carried out using concurrent think-aloud and probing techniques [15], attempting to collect information about potential problems in the B-HSOPSC questionnaire. At the 2-hour cognitive debriefing, the B-HSOPSC questionnaire was administered to an expert committee with 15 members, representing the studied population. The expert group included professionals of various medical specialities, a psychologist, language professionals and translators, and an occupational health specialist [16]. The respondents participating in the cognitive debriefing had equal representation of both genders. Moreover, they were representative of the major characteristics of the target group and were experts in their professional field. All experts, after completion of the questionnaire, were interviewed by the local project manager. Interviews addressed each item in the B-HSOPSC questionnaire and checked if participants had indicated any difficulty understanding the questions or if they would phrase it differently. The expert committee had to assess whether the questionnaire words and phrases describe the same ideas or subjects in both, the original and the adapted version of the questionnaire. This assessment ensured that all items were properly translated and were relevant in the new setting. During the debriefing process, any discrepancies or uncertainties about the meaning of items were addressed. Participants were encouraged to propose alternative ways of rendering the meaning of the original. Based on the suggestions and interpretations (evaluated for conceptual equivalence) and equivalence (in construct operationalisation), as well as on the identified discrepancies between the original text and its translation, the Bulgarian version of HSOPSC questionnaire, B-HSOPSC, was created.

A general protocol was developed for the translation of instruments into Bulgarian; It included: description of each step in the translation process, a field test-retest study and psychometric characteristics analysis. Pre-test measurement was performed with a convenience sample of 150 respondents from the five university hospitals in Plovdiv. After consenting to participate, the sample group completed the questionnaire at work and returned it to the authors. The B-HSOPSC was resent to the same employees for completion four weeks later. Thus, the reliability of the initial answers was tested.

After analysing the test-retest results, the final version of B-HSOPSC questionnaire was created.

The data from all questionnaires were entered into the appropriate statistical software program. As items were worded in both positive and negative directions, negatively worded items (A5, A7, A8, A10, A14, A16, A17, B3, B4, C6, F2, F3, F5, F6, F7, F9 and F11) first were reverse coded (Table 1). To compare item scale scores obtained during the test and re-test, the Wilcoxon signed-rank test was applied. To evaluate intra-rater reliability, the split-half method was used, and Spearman-Brown coefficient was calculated (rsb). An average inter-item correlation of at least 0.50 was regarded as good [17]. The intra-class correlation coefficient (ICC), using the test-retest method, was also used to estimate the inter-rater reliability to check consistency and reproducibility. Internal consistency was assessed using Cronbach’s alpha. An alpha value of 0.60 was considered as the lowest acceptable value [17].

Table 1

Results from the test-retest reliability of the panel questionnaire among hospital employees (N = 146)

Questions	Wilcoxon test	Spearman-Brown coefficient (rsb)	Cronbach’s Α
			I measu-rement	II measu-rement
Safety culture dimensions (unit level)
1. Supervisor/manager expectations and actions promoting safety			0.779	0.805
B1 My supervisor/manager says a good word when he/she sees a job done according to established patient safety procedures	1.136	0.803
B2 My supervisor/manager seriously considers staff suggestions for improving patient safety	0.302	0.921
B3 Whenever pressure builds up, my supervisor/manager wants us to work faster, even if it means taking shortcuts	0.577	0.959
B4 My supervisor/manager overlooks patient safety problems that happen over and over	2.496	0.715
2. Organisational learning—continuous improvement			0.606	0.607
A6 We are actively doing things to improve patient safety	0.367	0.803
A9 Mistakes have led to positive changes here	1.387	0.910
A13 After we make changes to improve patient safety, we evaluate their effectiveness	0.052	0.228
3. Teamwork within hospital units			0.662	0.628
A1 People support one another in this unit	0.632	0.936
A3 When a lot of work needs to be done quickly, we work together as a team to get the work done	0.879	0.928
A4 In this unit, people treat each other with respect.	0.977	0.743
A11 When one area in this unit gets busy, others help out	1.027	0.757
4. Communication openness			0.776	0.770
C2 Staff will freely speak up if they see something that may negatively affect patient care	0.351	0.919
C4 Staff feel free to question the decisions or actions of those with more authority	1.615	0.906
C6 Staff are afraid to ask questions when something does not seem right	1.977	0.738
5. Feedback and communication about error			0.611	0.633
C1 We are given feedback about changes put into place based on event reports	0.877	0.773
C3 We are informed about errors that happen in this unit	1.100	0.840
C5 In this unit, we discuss ways to prevent errors from happening again	0.185	0.834
6. Non-punitive response to error			0.697	0.748
A8 Staff feel like their mistakes are held against them	1.052	0,770
A12 When an event is reported, it feels like the person is being written up, not the problem	1.136	0,910
A16 Staff worry that mistakes they make are kept in their personnel file	0.243	0,903
7. Staffing			0.304	0.331
A2 We have enough staff to handle the workload.	1.387	0.936
A5 Staff in this unit work longer hours than is best for patient care	3.231	0.707
A7 We use more agency/temporary staff than is best for patient care	0.416	0.769
A14 We work in ‘crisis mode’, trying to do too much, too quickly	1.464	0.318
8. Hospital management support for patient safety			0.855	0.860
F1 Hospital management provides a work climate that promotes patient safety	0.577	0.979
F8 The actions of hospital management show that patient safety is a top priority	0.000	0.952
F9 Hospital management seems interested in patient safety only after an adverse event happens	0.707	0.952
Safety culture dimensions (hospital-wide)
9. Teamwork across hospital units			0.694	0.653
F4 There is good cooperation among hospital units that need to work together	0.302	0.865
F10 Hospital units work well together to provide the best care for patients	0.905	0.909
F2 Hospital units do not coordinate well with each other	0.992	0.835
F6 It is often unpleasant to work with staff from other hospital units	0.511	0.832
10. Handoffs and transitions			0.854	0.918
F3 Things “fall between the cracks” when transferring patients from one unit to another	0.284	0.879
F5 Important patient care information is often lost during shift changes	2.516	0.860
F7 Problems often occur in the exchange of information across hospital units	0.570	0.804
F11 Shift changes are problematic for patients in this hospital	0.577	0.909
Outcome measures
11. The frequency of event reporting			0.823	0.864
D1 When a mistake is made but is caught and corrected before affecting the patient, how often is this reported?	0.894	0.885
D2 When a mistake is made but has no potential to harm the patient, how often is this reported?	0.814	0.891
D3 When a mistake is made that could harm the patient, but does not, how often is this reported?	1.363	0.839
12. Overall perceptions of safety			0.331	0.339
A15 Patient safety is never sacrificed to get more work done	0.486	0.913
A18 Our procedures and systems are good at preventing errors from happening	1.291	0.872
A10 It is just by chance that more serious mistakes don’t happen around here	0.720	0.868
A17 We have patient safety problems in this unit	1.115	0.780
Patient safety grade			-
E1, Please give your work area/unit in this hospital an overall grade on patient safety	0.577	0.974		-
Number of events reported	-
G1 In the past 12 months, how many event reports have you filled out and submitted?	1.732	0.963		-

Data were processed, using the IBM SPSS Statistics 22 software. The level of statistical significance was set at P < 0.05.

Results

Using standardised procedure, consistency in the content and face validity between the original HSOPSC instrument and the B-HSOPSC were ensured by cross-cultural adaptation [16]. To maintain the instrument content validity at a conceptual level across different cultures, the applicability of original HSOPSC items was checked at a cognitive debriefing interview. The group of interview participants had the following characteristics: the average age of 44.29 years, (SD 8.43); gender ratio M: F (n) was 3:12; employed (n)-15 and the average duration of current job position - 9 years (SD 5.43). The cognitive debriefing interview resulted in a revision of some items in the B-HSOPSC. The phrase in item А 15 never sacrificed was replaced by never ignored. The possible answers to the questions related to work position were adapted to the national healthcare setting. Questions about the hospital characteristics (teaching status, ownership, and geographic region) were added, as well as questions, investigating the attitude and willingness of respondents to report undesirable events if there is an anonymous communication system that guarantees their privacy. To protect respondents’ privacy, other demographic characteristics related to the respondents were not included.

One hundred and fifty questionnaires were distributed, and all of them were returned. Questionnaires with missing data were excluded. Thus, data from 146 questionnaires were analysed.

The high reliability of the instrument was confirmed by the split-half method (0.97) and the ICC-coefficient (0.95). The Spearman-Brown coefficient for most items was satisfactory (r > 0.70), except questions A13 and A14.

The overall Cronbach’s alpha for the B-HSOPSC questionnaire is 0.918. The internal consistency measured on the structure of 10 sub-scales showed that the Cronbach’s alpha was above 0.70 for six of the sub-scales (sub-scales 1, 4, 6, 8, 10, 11), and ranged from 0.60 to 0.69 for the other four (sub-scales 2, 3, 5, 9). Sub-scales 7 Staffing and 12 Overall perceptions of safety had the lowest coefficients (Table 1).

The number of items and their content remained the same as in the original to a great extent, due to the consensus of the expert committee.

However, after the test-retest analysis, minor changes and adjustments were made to item 13 and item 14. In the course of the cognitive interview, some medical professionals argued that there was lack of conceptual validity for these two items in the Bulgarian setting.

In item A 14 the term crisis model was substituted by the phrase working under conditions of insufficient resources, and the wording of item A 13 After we execute changes to improve patient safety, we evaluate their effectiveness became After we execute changes to improve patient safety, we evaluate whether they lead to positive results. At that stage, no changes to sub-scale 7 and sub-scale 12 were made. Following statistical analysis of accumulated data from the future national representative study through a web-based platform and evaluation by the expert committee, some of the questions might be excluded. Eventually, the final version of B-HSOPSC may be amended.

Discussion

Other researchers, under similar circumstances, have reached unsatisfactory values of Cronbach’s alpha in certain sub-scales [18] [19] [20] [21] [22] [23] [24] [25]. Most often, low values are reported in sub-scale 7 Staffing (Table 2) [18] [20] [21] [24].

Table 2

Values of Cronbach’s α across original dimensions in the questionnaire and comparison to US, English, Dutch, Croatian, Portuguese, French and Slovenian data

Dimensions	№ of items	Cronbach’s α
		US	UK	Dutch	Croatian	Portuguese	French	Slovenian	Bulgarian
Supervisor/manager’s expectations and actions regarding safety	4	0.75	0.68	0.70	0.79	0.72	0.83	0.74	0.81
Organisational learning—continuous improvement	3	0.76	0.66	0.57	0.53	0.71	0.59	0.36	0.61
Team work within hospital units	4	0.83	0.73	0.66	0.76	0.73	0.63	0.74	0.63
Communication openness	3	0.72	0.67	0.72	0.64	0.67	0.62	0.74	0.77
Feedback and communication regarding errors	3	0.78	0.80	0.75	0.74	0.76	0.64	0.72	0.63
Non-punitive response to errors	3	0.79	0.65	0.69	0.62	0.57	0.57	0.61	0.75
Staffing	4	0.63	0.58	0.49	0.35	0.48	0.46	0.65	0.33
Hospital management’s support for patient safety	3	0.83	0.69	0.68	0.73	0.77	0.73	0.82	0.86
Teamwork across hospital units	4	0.80	0.70	0.59	0.79	0.69	0.59	0.74	0.65
Handoffs and transitions	4	0.80	0.77	0.68	0.64	0.71	0.66	0.66	0.92
Frequency of event reporting	3	0.84	0.83	0.79	0.91	0.90	0.84	0.88	0.86
Overgroups perceptions of safety	4	0.74	0.67	0.62	0.57	0.62	0.67	0.65	0.34

In Croatia, researchers report very low values of Cronbach’s alpha for sub-scale 7 Staffing (0.35), which is similar to our findings [18]. Croatian authors speculate that some items might have been interpreted differently due to some specific national characteristics. They refer particularly to question A7, dealing with issues regarding employment of locum staff through the medical agencies to cope with work overload. As this practice is unusual in most European health systems, question A7 had to be revised and adapted to the specifics of the respective national healthcare system [18]. Eiras at al., show that, once question A7 is excluded, the internal consistency in sub-scale 7 Staffing increases from 0.48 to 0.57 [19]. The practice of recruiting locum agency staff is not popular in Bulgaria, regardless of the fact that it is legally allowed.

Most of the items were found to be valid, yet the staffing sub-scale had rather low reliability, as revealed by research in the USA [26]. Other researchers examining the psychometric properties of the HSOPSC found that the items had acceptable psychometric properties except for the staffing subgroup and questions regarding supervisor/ manager’s expectations and actions promoting PS [27].

Our study has some limitations. The convenience sampling method was used for the healthcare professionals’ selection in the test-retest study. In fact, the linguistic validation and the cross-cultural adaptation of the B-HSOPSC questionnaire made very few factual amendments to the original. We assume that the result is not skewed regarding overall patient safety culture evaluation.

The present paper examines the development of a translation protocol and the use of cognitive debriefing as part of the cultural adaptation process. We assessed the semantic, idiomatic, experiential, and conceptual equivalence between the source and the target questionnaire. During the field testing, the psychometric characteristics of the questions exhibited good validity and reliability, except for a couple of two items. In the future, the instrument will be administered to the target population in the main study so that the psychometric properties of the instrument can be verified. Therefore, the comparison between our studies results regarding health care professionals’ evaluation of patient safety and the results of other similar studies possible.