Assessing Patients' Perceptions of Safety Culture in the Hospital Setting: Development and Initial Evaluation of the Patients' Perceptions of Safety Culture Scale.

OBJECTIVES: Both patient satisfaction and hospital safety culture have been recognized as key characteristics of healthcare quality and patient safety. Thus, both characteristics are measured widely to support quality and safety improvement efforts. However, because safety culture surveys focus exclusively on the perspective of hospital staff, the complimentary information to be gained from patients' perceptions of safety culture has received little research attention so far. We aimed to develop a measure explicitly focusing on patients' perceptions of safety culture in the hospital setting and perform an initial evaluation of its measurement properties.
METHODS: We employed a multistep development approach including (a) literature review of survey instruments for patient experience and safety culture and (b) item categorization and selection. We evaluated the measurement properties of the final item set focusing on factor structure, internal consistency, item difficulty, and discrimination. Data were collected from June to December 2015 via an online patient survey conducted routinely by a health insurer.
RESULTS: Overall, 112,814 insured persons participated in the online survey (response rate = 19.7%). The final 11-item set formed a single scale that was named Patients' Perceptions of Safety Culture scale. Its measurement properties were deemed satisfactory based on this initial evaluation.
CONCLUSIONS: The Patients' Perceptions of Safety Culture scale contributes to both a more comprehensive view of patients' experience of healthcare and a more balanced approach to safety culture measurement in healthcare. It contributes to an increased recognition of patients' views on safety-relevant aspects of their care that provide important inputs to patient safety improvement.

Patients' experience of healthcare has been widely recognized as a key characteristic of the quality of care.[1-3] Surveying patients concerning their perceptions and experience of different aspects of care during their hospital stay is an established mechanism for monitoring and improving healthcare quality. Moreover, the publication of patient experience data offers patients an additional source of information they might want to consider when choosing a hospital.[4]

In many countries, parallel mechanisms have been established for incorporating patients' experiences into quality improvement.[5] For example, patients are surveyed by hospitals that use this information to identify areas of improvement at the service or unit level and for quality reporting.[6] Furthermore, healthcare insurance companies routinely conduct patient surveys shortly after hospital discharge to provide input into health insurers' discussion with hospitals concerning improvements in the provision of care, especially during care transitions between healthcare sectors.[7]

These surveys usually ask about patients' experience of various aspects of their hospital stay and the transparency of care processes. Recently, the call for patient involvement in all aspects of care has been extended to patient safety.[8-10] Thus, patient safety issues should complement the current spectrum of dimensions of the patients' experience.[11,12] One particular aspect to consider is patients' perceptions of the safety culture in the hospital setting.

Generally, safety culture is regarded as a facet of organizational culture that refers to how safety is viewed and treated in an organization, which depends on shared values and norms of members of this organization.[13] These values and norms affect the attitudes, perceptions, and behavior of all members of an organization.[14] Thus, safety culture is considered the foundation of patient safety by directing the attention of staff to safety issues and encouraging safe work practices.[15]

For the last three decades, there has been a debate in the literature on the distinction between safety culture and safety climate[13,16,17] that has methodological implications. Summarizing this debate, safety culture is seen as the source for patterns of behavior that can be observed, described, and changed,[18] whereas safety climate has been defined as the shared perceptions of employees about safety-relevant aspects of their work environment.[19,20] Thus, it has been pointed out that staff surveys may capture safety climate rather than safety culture.[16] In this article, we use the expression “perceptions of safety culture” to reflect that these perceptions can be from an inside or outside perspective of the organization.

So far, safety culture in healthcare has been defined exclusively focusing on healthcare staff and their shared beliefs concerning patient safety that form the basis of behavioral patterns.[21] Although a large number of survey instruments to measure healthcare staff perceptions of safety culture have been developed and increasingly used around the world,[16,22,23] patients' experiences and perceptions of safety culture have received little attention so far. During a hospital stay, patients and relatives experience safety culture through observation and interaction, even if some aspects of safety culture may not be directly accessible to them. Thus, when aiming to provide a comprehensive assessment of safety culture, the perspective of patients and relatives may offer relevant information complementing the perspectives of staff and hospital management.[24]

Specific Aims

This study aimed to contribute to this goal by developing a measure explicitly focusing on patients' perceptions of safety culture in the hospital setting. Specifically, we aimed to develop an item set, pilot it by including it in the postdischarge patient survey of a large health insurer, and perform an initial evaluation of its measurement properties. The development of such a measure contributes to both a more comprehensive view of the patients' experience and a more balanced approach to safety culture measurement and improvement in healthcare.

METHODS

In developing a measure for patients' perceptions of safety culture in the hospital setting, we chose a multistep procedure.

Identification of Survey Instruments

In a first step, we searched the electronic databases PubMed and Google Scholar for survey instruments designed to capture patients' experiences during a hospital stay published between January 2000 and April 2015 in German or English. We applied the following search terms using the Boolean operators AND and OR to combine selected search terms: patient satisfaction, patient involvement, patient participation, patient safety, health*care, and health*care survey. In addition, we extended our search to existing staff surveys of safety culture designed to capture hospital staff perceptions, because safety culture is usually assessed from the perspective of healthcare workers. Thus, we ran a second search applying the same limits concerning the publication period and language with the following search terms, again using the Boolean operators AND and OR: safety climate, safety culture, quality indicators, patient safety, health*care, and health*care surveys. For both searches, we applied the inclusion criterion of application in a hospital setting. We also searched the references of identified articles. Exclusion criteria were disease-specific or hospital-specific questionnaires.

Item Categorization

In a second step, all items of the instruments identified in step one were submitted to a systematic coding procedure (Fig. 1, flowchart).

FIGURE 1

Flowchart of procedure for item categorization and selection (for abbreviations see Table 1).

TABLE 1

Survey Instruments Selected for Detailed Review

All items of the identified patient experience surveys were first reviewed by two independent reviewers (C.M. and M.K.) for whether or not they referred to patients' experience of safety culture during the hospital stay. Any disagreements were resolved by discussion within the core research team (C.M., M.K., and T.M.). The selected items were then coded according to seven core dimensions of safety culture based on Itoh et al.[25]: communication, leadership, management commitment to safety, nonpunitive approach to error, safety systems, teamwork, and work pressure.

Furthermore, items from staff surveys of safety culture were first reviewed for whether or not the safety culture aspect addressed by the item might be suitable for patient assessment. Then, we categorized candidate items into the same seven core dimensions of safety culture[25] as described previously. The initial coding was then reviewed by a multidisciplinary group of researchers (A.H., C.M., M.K., and T.M.) and discussed until consensus was reached.

Item Selection

In a third step, we integrated the items extracted from both sources and removed duplicates. Staff survey items were reformulated to reflect the patient perspective on safety culture. Remaining items were prioritized on the basis of a consensus discussion in the entire project team to ensure that central aspects of safety culture that can be assessed from a patient perspective were covered. To ensure seamless integration into a patient experience survey conducted routinely by a large health insurer, the rating scale for the item set was aligned with the format used in other parts of the survey (i.e., “strongly agree,” “somewhat agree,” “somewhat disagree,” “strongly disagree,” and “not applicable”).

Data Collection and Sample

For data collection purposes, the final item set was incorporated into the Technician Health Insurance Hospital Survey[26] conducted every other year among all insured persons who themselves or their children have been admitted to a hospital for at least one night, thereby excluding outpatients (about 800,000 eligible persons per year). The survey is routinely administered approximately six weeks after discharge with the survey being sent out in monthly waves. Eligible persons received a letter with login details to a personalized online questionnaire. The validation of this data collection process showed that all insured groups represented in the former paper-pencil version were also represented in the online sample. In this study, we included data collected monthly from June to December 2015.

Statistical Analyses

In evaluating the item set, we conducted the following analytical steps. Descriptive item analyses included calculation of mean, standard deviation, variance, skewness, and kurtosis. We also calculated item difficulty and discrimination index.[27,28] After testing for sampling adequacy (Kaiser-Meyer-Olkin >0.5 and Bartlett test of sphericity), a principal component analysis using varimax rotation was carried out. For factor extraction, we employed Kaiser criterion (Eigenvalues >1) and Scree test (i.e., extraction of factors above an inflection point on a graph of plotted Eigenvalues).[27] Internal consistency was assessed using Cronbach α. A coefficient of 0.70 or greater was interpreted as acceptable internal consistency.[29] Response categories ranged from 1 (strongly disagree) to 4 (strongly agree) with an additional category in case the item was “not applicable” to the respective hospital stay. All items marked as “not applicable” were treated as missing values. Missing values were not replaced during the analyses because we did not feel that imputation was warranted because some items did not apply to all patients (e.g., having had contact with different services within the hospital). All statistical analyses were carried out using SPSS Version 22 (IBM Corp, Armonk, NY).[30]

RESULTS

Following the research procedure outlined previously, we selected six surveys of patients' experiences of their care[26,31-36] that covered a range of themes and the two staff surveys of safety culture[37,38] that are used most frequently in hospitals around the world[39] for detailed review (Table 1). These instruments consisted of a total of 428 items; 324 from surveys of patients' experiences of their care and 104 from staff surveys of safety culture.

Safety Culture Aspects Covered by the Selected Instruments

Although the staff surveys were focused on safety culture, the surveys of patients' experiences of their care covered a much wider range of themes. Figure 1 clearly shows the different focus of the surveys of patients' experiences of their care and staff surveys of safety culture. Of the 428 items in patient and staff surveys, 100 items were coded as referring to “experience of safety culture in the hospital setting.” When coding these items according to the seven core dimensions of safety culture,[25] 35% covered content of more than one dimension. Across both data sources, the most frequent category describing the experience of safety culture concerned communication (n = 53) followed by safety systems (n = 37).

Sample questions from the surveys of patients' experiences of their care and the staff surveys of safety culture that could be assessed from the patients' perspective if reformulated are also provided in the flowchart (Fig. 1).

Composition of the Final Item Set

The final selection of suitable items aimed at allowing for capturing the patient experience of safety culture for a broad spectrum of patients with different treatment paths. For example, a safety culture dimension that was excluded concerned the nonpunitive approach to error for which we were unable to identify suitable items based on our review (Fig. 1), likely because it is rather difficult to assess from the patient perspective. The following key themes emerged from the consensus process: feeling safe, information flow, teamwork, communication with patients, communication in the care team, staffing, and others (Table 2). After eliminating redundancies and reformulating some safety culture items from the staff to the patient perspective, a total of 11 items prototypical for these themes were selected. Of this item set for assessing patients' perceptions of safety culture in the hospital setting, four items had already been part of the TKHS and seven items were subsequently added (see Table 2 and appendix 1, http://links.lww.com/JPS/A132 for item set in German).

TABLE 2

Final Item Set Used to Assess Patients' Perceptions of Safety Culture in the Hospital Setting

Pilot Testing and Initial Evaluation of Measurement Properties

Sample

Overall, 112,814 insured persons participated in the online survey from June to December 2015 (response rate = 19.7%). Fifty-one percent of the respondents were male. The mean (standard deviation) age for the overall sample was 52.2 (19.4) years. Most participants (94.3%) referred to their own hospital stay and 5.7% referred to their child's hospital stay while answering the questionnaire.

The items concerning patients' perceptions of safety culture in the hospital setting were well accepted by the respondents. This was reflected by a rather low proportion of missing values (2.6%–4.8%), which was similar to the other survey items. Overall, ratings were rather positive with 35.5% to 55.6% of responses “strongly agree” (see Table 3).

TABLE 3

Distribution of Patients' Perceptions of Safety Culture in the Hospital Setting (N = 112,814)

Measurement Qualities

As can be seen in Appendix 2, http://links.lww.com/JPS/A133, skewness and kurtosis values show that all patient safety items are not normally distributed. All distributions were peaked and positively skewed. As a rule of thumb, the discrimination index should be greater than 0.3.[27] This applied to all items (range = 0.66–0.82) and is an indicator of good discrimination ability. Item difficulty ranges between 17.8 and 25.0.

Factor Structure

Firstly, we established that the data set meets the requirements for exploratory factor analysis. Even after exclusion of missing data, our sample size of 55,792 was more than sufficient to perform factor analyses.[28] Furthermore, Bartlett test of sphericity was statistically significant (P > 0.001) and the Kaiser-Meyer-Olkin was 0.959. Secondly, principal component analysis revealed a one-factor solution explaining 65.2% of the variance. All factor loadings were positive and ranged between 0.71 and 0.88. The one-factor solution showed good internal consistency (Cronbach’s α = 0.95).[40] To ensure that there was no unnecessary duplication of item content, we ran interitem correlation analyses. Interitem correlations were 0.6 on average and only one was 0.81 indicating no unusually high overlap.

DISCUSSION

Summary

Previously, safety culture surveys focused on the perspectives of hospital staff and management. In line with approaches to explicitly include the patient perspective in health services research and quality improvement, this study contributes by extending safety culture assessment to the perspectives of patients in the hospital setting. We named the item set Patients' Perceptions of Safety Culture scale (PaPSC scale).

With 11 items, the PaPSC scale is rather short given the complexity of the underlying construct of safety culture. Considering the different dimensions of safety culture known from staff surveys that guided the development process,[25] it was rather surprising that the items of the proposed PaPSC scale belong to a single factor. However, the initial evaluation revealed good measurement properties. In addition, the items represent several facets of safety culture that patients may experience in a hospital and are suitable for different treatment paths. This balances the need for comprehensive safety culture assessment with the practical limitations of potentially overburdening patients with extensive surveys.

A main strength of this study is the large sample size. Most patients participating in the survey were able to provide assessments of safety culture. However, not all items of the PaPSC scale are equally applicable to all patients. This is reflected by an increased percentage of “not applicable” responses for two items. Firstly, the item “The different services (ward, x-ray, physiotherapy, etc.) are well coordinated.” could only be assessed if multiple hospital units were involved in a patient's care. Secondly, the item “Staff freely spoke up whenever they had the impression that something was amiss.” could only be assessed if the patient either witnessed such a conversation or felt that something had gone wrong.

Future research should explore the reasons behind the use of “not applicable” in-depth by performing cognitive interviews with patients. For example, the relatively high proportion of patients who felt unable to respond to the question “I had the impression that patient safety was always a top priority” (10.1%) might reflect that these patients had not thought about patient safety issues much before or during their hospital stay or that they do not understand the cues they might use to make such an assessment. This may point to an important area for patient education.

Interestingly, we also observed a relatively high rate of “not applicable” for the items “staff freely spoke up whenever they had the impression that something was amiss.” This may imply that nothing was amiss or that patients were not able to observe such behaviors in their presence. Conversely, the fact that more than half of the patients felt comfortable making an assessment on this item may imply that they witnessed a situation where it was relevant that staff spoke up and that this behavior does not only occur behind closed doors. Because speaking up is a growing research field in patient safety[41] and the patient's role in speaking up has been highlighted previously,[9] patients' perceptions of this important aspect of safety culture should be a focus of future studies.

Measuring patients' perceptions of safety culture in the hospital setting has several benefits. Firstly, our understanding of patients' experience of healthcare can be broadened and patients can be sensitized for safety culture. Previously, safety culture had rarely been included in patient surveys, which mainly focused on patient satisfaction.[26,31-36] In contrast, the PaPSC scale explicitly addresses safety culture in the hospital setting. By integrating the PaPSC scale, patient experience surveys can be extended to issues of safety culture that are increasingly acknowledged as impacting on patient safety and as a foundation for sustainable patient safety improvement.[15]

Secondly, the patient perspective provides complementary information on safety culture[11] that has rarely been considered when aiming to assess and develop safety culture. Thus, the PaPSC scale contributes to closing the gap concerning new ways of measuring safety culture that go beyond the staff view. In the future, this work could be extended to other groups involved in patient care such as informal caregivers and family members.[42] In addition, the PaPSC scale could be an additional component in evaluating patient safety measures and potentially for internal and external benchmarking.[43] In summary, using the PaPSC scale can contribute to strengthening the patients' voice in patient safety and to improving our understanding of safety culture by providing a novel perspective complementing staff views.

Limitations

The item selection process was based on a critical review of previous survey instruments used to assess either patient's experiences of their care or staff perceptions of safety culture. This review is unlikely to be complete because many patient surveys have been developed and adapted by individual trusts or hospitals. Furthermore, although we believe to have covered a range of patient experience questionnaires relevant to the research goals, our approach may have restricted the thematic scope of the resulting PaPSC scale. Firstly, we did not specifically include search terms such as “patient experience” or “patient perceptions.” However, these search terms seem to have a considerable overlap with our search terms because the resulting studies and survey instruments covered these topics and also made use of this terminology. In addition, we did not involve patients in the item development process. However, several of the instruments we based our item set on had been developed with input from patients so that their perspectives were considered indirectly.[32,33,36]

Because we considered two previously independent lines of research that converged on key themes of safety culture, we are quite confident that the PaPSC scale captures what was intended. Nevertheless, future research aiming at an in-depth understanding of patients' experience of safety in healthcare might benefit from additional qualitative research into patients' perceptions of safety culture in various healthcare settings. For example, we used the safety culture dimensions by Itoh et al.[25] to categorize survey items. Although these dimensions were well suited for the items identified by our literature search, they had originally been developed for staffs' perceptions of safety culture. Specifically, aspects of safety that are unique to the patient experience such as the feeling of being safe should be explored more thoroughly in the future.

From a methodological point of view, it should be noted that we used an online survey for data collection, which might have limited the representativeness of our sample. However, contradictory to what a potential media bias implies (i.e., reaching mainly younger people), we found that the response rate in the age group younger than 46 years was significantly lower than in the group between the ages of 46 and 75 years. This is similar to response rates for the previous paper-pencil version of the survey. Thus, a possible media bias can be considered as rather low. In addition, it has to be noted that inclusion criteria for the survey were set to focus on inpatients and that we were unable to perform a nonresponder analysis because of the anonymous survey. Future applications of the PaPSC scale need to make sure that it can also be used for other patient populations.

Because we present results of an initial evaluation of measurement properties, our finings need to be interpreted with caution and more research is required to conduct a full psychometric evaluation. For example, the internal consistency (Cronbach α = 0.95) may be inflated because of the rather high number of items included in the scale.

Finally, the validity of the PaPSC scale will have to be established in future studies testing it against outcome data.

CONCLUSIONS

This study contributes to an increased recognition of patients' views on safety-relevant aspects of their care by broadening the thematic focus of patient surveys to include patients' perceptions of safety culture in the hospital setting. At the same time, it acknowledges safety issues as relevant to the patient experience during a hospital stay. Given its thematic scope and good measurement properties, the PaPSC scale can be recommended for integration into routine collection of patient experience data in healthcare. Systematically including patients' perceptions of safety culture when designing, implementing, and evaluating interventions offers novel insights and contributions to patient safety improvement while strengthening active patient participation.