Variation in Perception of Safety Culture in Out-of-hours Family Medicine Service in Croatia.

INTRODUCTION: The Safety Attitudes Questionnaire (SAQ) is among the most frequently cited tools for measuring safety culture in healthcare settings. Its ambulatory version was used in this study. The aim was to assess safety culture in out-of-hours (OOH) family medicine service and its variation across job positions, regions, and respondents' demographic characteristic.
METHODS: A cross-sectional observational study was carried out targeting 358 health professionals working in the 29 largest Croatian healthcare centres providing out-of-hours family medicine service. The response rate was 51.7% (185 questionnaires). The questionnaire comprised 62 Likert items with 5 responses (fully disagree to fully agree). Scores of negatively worded items were reversed before analysis. Scores on the total scale and subscales were calculated as additive scores. The study included demographic data on gender, age, working experience, and job position. Repeated measurement analysis of variance was used to assess variation of Safety Attitudes Questionnaire - Ambulatory Version (SAQ-AV) sub-scales.
RESULTS: Nurses assessed safety culture higher than did physicians and residents. Teamwork climate had higher scores than Ambulatory process of care and Organizational climate. Stress recognition and Perceptions of workload had the lowest overall scores. Variation across gender, age, working experience, and region was not statistically significant.
CONCLUSIONS: SAQ-AV can be used to identify areas for improvement in patient safety at OOH GPs. There is a need to improve staffing and support for OOH GP residents. Further research is needed in order to gain better understanding of factors influencing observed variations among job positions.

Introduction

Even though the very first sentence of the Hippocratic Oath – First, do no harm – primes physicians to focus on the importance of patient safety, the trend of occurrence of the term “patient safety” in research publications has only seen steady growth in the last 20 years, since the report “To Err Is Human” was published (1, 2). Measuring patient safety culture has proved to be a challenge, and different approaches have been used (3, 4). The Safety Attitudes Questionnaire (SAQ) has been among the most frequently cited tools for measuring safety culture in various healthcare settings (3, 5). The SAQ was originally developed for a hospital setting, but soon after its development, it was adapted for use in the ambulatory setting as the SAQ-AV (6). Recently, the SAQ-AV has been translated and validated in several European countries (7, 8, 9, 10, 11, 12, 13).

Research on variability of safety attitudes regarding job positions and healthcare settings is still scarce. Gehring et al. found differences in safety culture between job positions and types of office organizations in primary healthcare (14). Bondevik et al. analysed differences in SAQ subscales regarding job position, gender and age, and found differences in safety climate and job satisfaction (8). Kwon at al found difference in SAQ subscales between nurses and doctors in operating room settings (15). Klemenc-Ketiš et al. found differences in patient safety climate scores and quality of collaboration scores between physicians, nurse practitioners, and practice nurses (16).

To the best of our knowledge, there has been no previous research on variation in safety attitudes in primary healthcare in Croatia. Out-of-hours (OOH) primary healthcare services in Croatia are provided by healthcare centres and emergency medical services. In healthcare centres, OOH services are provided by family physicians, who rotate between practice in normal working hours and OOH service. Their OOH services are available on weekends and holidays, excluding night shifts.

In this study we present results of a patient safety attitudes survey in OOH family medicine service in Croatia. The study was a part of the international study Patient Safety Culture in European Out-of-hours Services (SAFE-EUR-OOH) led by the Norwegian coordinating group of the European research network for out-of-hours primary healthcare (EurOOHnet). We aim to identify patterns of variation in the Croatian version of the SAQ-AV sub-dimensions in relation to geographical region and job positions, as well as among SAQ-AV subscales.

Methods

Type of study and settings

In July and August 2015, we carried out a cross-sectional observational study in the 29 largest Croatian healthcare centres providing out-of-hours (OOH) family medicine service (FM). These centres account for 85% of family medicine service staff in Croatia. Regional analysis was based on the level two statistical regions (NUTS2) in the European Union (17).

Participants

Family physicians, nurses, and residents with e mail addresses, working at the 29 healthcare centres, were contacted by e-mail and asked to fill out an anonymous and voluntary online survey. In this way, 385 people were invited to participate in the survey. All nurses were practice nurses.

Data collection

Out of 385 invited participants, 185 filled in the online survey (response rate 51.7%). After descriptive analysis, three outliers and participants with missing demographic data were excluded, resulting in a sample size of 140.

Questionnaire

We used the Croatian version of the Safety Attitudes Questionnaire – Ambulatory Version (SAQ-AV). Psychometric properties of this questionnaire were analysed by Mesaric et al. (11). The original questionnaire comprised 62 items where study participants rated their agreement on a 5 point ordinal scale (from fully disagree to fully agree). Scores of negatively worded items were reversed before analysis. Only items confirmed in the previous analysis were used (37 items) (11). Scores on the total scale and five subscales, namely Organizational climate (14 items), Teamwork climate (8 items), Stress recognition (6 items), Ambulatory process of care (5 items), and Perceptions of workload (4 items), were calculated as additive scores with a range between 0 and 100. The total score was calculated using only items from the subscales Organizational climate, Teamwork climate, and Ambulatory process of care (27 items), because Stress recognition and Perception of workload were reliable scales, but were not sub-dimensions of patient safety culture attitude (11). Higher score values indicate more positive perception of patient safety culture. The study also included demographic data on gender, age, working experience, and job position.

Statistical analysis

Statistical analysis comprised descriptive analysis and repeated measurement analysis of variance, followed by Tukey’s Honest Significant Difference method for post-hoc pair-wise comparisons (18). In order to assess variation in SAQ scores over SAQ sub-scales and among job positions we performed repeated measurement analysis of variance with SAQ sub-scale (Organizational climate, Teamwork climate, and Ambulatory process of care) as a within-subject factor and job position as a cross-subject factor. All statistical tests were conducted at the level of statistical significance α=0.05. Analyses were done in R and RStudio (19, 20).

Results

Table 1 summarizes descriptive statistics for demographic variables. The sample comprised 114 (81.4%) female, and 26 (18.6%) male participants. There were 85 (60.7%) physicians, 28 (20.0%) residents, and 27 (19.3%) nurses. Almost a third of all participants (n=45, 32.1%) were between 31 and 40 years old. There were 41 (29.3%) junior staff with up to five years of working experience. Healthcare centres from all four NUTS2 regions of Croatia were well represented, and the largest group of participants (n=54, 38.6%) came from the healthcare centres in the City of Zagreb.

Table 1

Demographic characteristics of the Croatian out-of-hours family medicine service employees participating in the study (N=140).

Variable	n (%)
Position
Nurse	27 (19.3)
Resident	28 (20.0)
Physician	85 (60.7)
Gender
Male	26 (18.6)
Female	114 (81.4)
Age (years)
<=30	32 (22.9)
31-40	45 (32.1)
41-50	25 (17.9)
51-60	33 (23.6)
>=61	5 (3.6)
Working experience
<=5	41 (29.3)
6-10	29 (20.7)
11-20	24 (17.1)
21-30	30 (21.4)
31-40	15 (10.7)
NUTS2 region
City of Zagreb	54 (38.6)
Adriatic Croatia	29 (20.7)
Pannonian Croatia	28 (20.0)
North Croatia	29 (20.7)

Descriptive statistics for the five subscales and the total SAQ scale, broken down by NUTS2 regions of Croatia, are presented in Table 2. The lowest mean scores were those for Stress recognition, and the highest for the Teamwork climate. There were no statistically significant differences among the NUTS2 regions in any of the sub-scales. There were also no significant differences in any of the SAQ sub-scales regarding gender, age and working experience.

Table 2

Descriptive statistics for SAQ sub-scales and the total score by NUTS2 regions of Croatia.

SAQ sub-scales	City of Zagreb ⴳ(SD)	Adriatic Croatia ⴳ(SD)	Pannonian Croatia ⴳ(SD)	North Croatia ⴳ(SD)	Croatia ⴳ(SD)	p
Organizational climate	60.8 (22.3)	68.8 (17.9)	64.5 (19.9)	57.9 (23.2)	62.6 (21.3)	0.6490
Teamwork climate	77.9 (13.7)	81.2 (14.5)	82.8 (12.6)	73.6 (13.9)	78.7 (13.9)	0.3377
Stress recognition	42.9 (23.1)	44.0 (19.8)	45.1 (25.6)	41.8 (22.1)	43.3 (22.6)	1.0000
Ambulatory process of care	67.4 (21.1)	71.7 (18.6)	70.9 (19.1)	65.3 (22.0)	68.6 (20.3)	1.0000
Perceptions of workload	62.0 (21.6)	65.5 (24.4)	56.2 (26.5)	49.8 (25.0)	59.1 (24.3)	0.3377
Total score	67.1 (17.3)	73.1 (15.4)	71.1 (14.7)	64.0 (17.0)	68.5 (16.5)	0.5548

Legend: SAQ – Safety Attitudes Questionnaire; – mean; SD – standard deviation; NUTS2 – EU nomenclature of territorial units for statistics, level 2; p – p-value for ANOVA by NUTS2 region using Holms’ adjustment for multiple testing

Comparison in SAQ sub-scales between staff on differentjob positions is provided in Table 3.

Table 3

Descriptive statistics for SAQ sub-scales, SAQ total score, Stress recognition, and Perceptions of workload by job position.

Scale	Job position			p
	Nurse ⴳ(SD)	Resident ⴳ(SD)	Physician ⴳ(SD)
Organizational climate	72.7 (20.9)	53.7 (18.6)	62.4 (21.1)	0.0180
Teamwork climate	83.6 (15.7)	74.6 (12.2)	78.5 (13.6)	0.1626
Ambulatory process of care	77.0 (21.4)	63.4 (22.4)	67.6 (18.6)	0.1347
SAQ total	76.7 (17.5)	61.7 (13.2)	68.1 (16.2)	0.0159
Stress recognition	45.8 (27.3)	45.8 (21.3)	41.7 (21.5)	1.0000
Perceptions of workload	58.1 (27.5)	62.3 (20.0)	58.3 (24.7)	1.0000

Legend: SAQ – Safety Attitudes Questionnaire; ⴳ – mean; SD – standard deviation; p – p-value for ANOVA by NUTS2 region using Holms’ adjustment for multiple testing

Results of repeated measurements analysis of variance with job position as a cross subject factor, and SAQ subscale as a within subject factor are reported in Table 4.

Table 4

Repeated measurements analysis of variance for SAQ scores with SAQ sub-scale as a within-subject factor and job position as a cross-subject factor.

Source of variation	df 1	df 2	F	p
Intercept	1	274	3003.09	<0.0001
Job position	2	137	5.91	0.0035
SAQ sub-scale	2	274	56.59	<0.0001
Interaction	4	274	1.18	0.3201

Legend: df 1 – numerator degrees of freedom; df 2 – nominator degrees of freedom

Interaction between job position and SAQ sub-scale was not significant. Post hoc, Tukey’s honestly significant difference test revealed no significant difference between residents and physicians, while both residents and physicians differed significantly from nurses, with nurses’ average scores being the highest. All pairwise comparisons among SAQ sub-scales were statistically significant. Average Teamwork climate scores were the highest and average Organization climate score the lowest. Figure 1 shows estimated fixed effects on SAQ sub-scales for different sub-scales and job positions.

Figure 1

Fixed effects for SAQ sub-scales and job position.

Discussion

Psychometric analysis of the Croatian version of SAQ-AV revealed a three factor structure with subscales for Organizational climate, Teamwork climate, and Ambulatory process of care. (11). These scales are, however, not directly comparable to other studies using SAQ questionnaire, because sets of items loading on these subscales are not the same. Analysis of variation in SAQ subscales over job position, gender, age, working experience, and NUTS2 region revealed significant variation only among job positions. Nurses consistently had more positive perceptions of patient safety culture than physicians, followed by residents. Similar findings were reported by Bondevik et al. (8) and Smits et al. (21). Klemenc-Ketiš et al. also report more positive perceptions by nurses; however, differences among professional groups were not significant (16). Smits et al. (ibid.) attribute this finding to the fact that nurses spend more time at the FM office, while physicians and residents often rotate. The same argument applies in the case of Croatia.

Residents’ perception of Organizational climate was the lowest, indicating need to improve organizational support to resident GPs. To the best of the authors’ knowledge, there were no studies on residents’ perceptions of workload and organizational support in OOH FM in Croatia. However, a recent Swiss study reported that residents in general practice “… indicated not having enough time for a private life”, they also had higher risk of burnout, and lower mental wellbeing in comparison to physicians (22). Comparing SAQ sub-scales, all job position groups assessed teamwork climate with the highest average scores, and Organization climate with the lowest average scores. All pair-wise comparisons between SAQ sub-scales were statistically significant. Teamwork climate was also perceived more positively in other similar studies (8, 16, 21, 23, 24). Stress recognition and Perceptions of workload were not recognized as dimensions of patient safety (11), and in our analysis they had the lowest overall scores. This may be a consequence of a chronic lack of staff and high turnover in OOH FM service.

It is hard to provide evidence based explanation for these findings, due to the scarcity of research on OOH family medicine services in Croatia. Regional comparison showed that variation among Croatian regions in patient safety culture is relatively small. However, we have included in this study only the largest healthcare centres, and results might be different if smaller healthcare centres were included. On the other hand, the SAQ questionnaire was developed for larger organizations, and might not be valid for such small healthcare centres.

The relatively low response rate is a limitation of the study. We have no information about the non-respondents, and that could be a source of bias. We used the Croatian version of the SAQ-AV questionnaire, which has a somewhat different factor structure from the original, thus the comparability of scores is somewhat limited. The study included all large healthcare centres and presents, to the best of our knowledge, the first study of patient safety attitudes in OOH FM service in Croatia.

Future research could explore to what extent perceptions of patient safety culture varies across work sites in Croatian OOH FM service (25). Such a result would indicate how large the potential for improvement is.

Conclusions

The study shows that the SAQ-AV can be used to identify areas for improvement in patient safety at OOH FM service. Low results for Stress recognition and Perceptions of workload point to the need to improve staffing in OOH FM service. Residents’ low assessment of Organizational climate show the need to improve support for OOH FM residents. Further research is needed in order to gain a better understanding of factors influencing observed variations among job positions.