Development of a patient safety culture scale for maternal and child health institutions in China: a cross-sectional validation study.

OBJECTIVE: This study aimed to develop a patient safety culture (PSC) scale for maternal and child healthcare (MCH) institutions in China.
METHODS: A theoretical framework of PSC for MCH institutions was proposed through in-depth interviews with MCH workers and patients and Delphi expert consultations. The reliability and validity of the PSC scale were tested in a cross-sectional survey of 1256 MCH workers from 14 MCH institutions in Zhejiang province of China. The study sample was randomly split into half for exploratory and confirmatory factor analyses, respectively. Test-retest reliability was assessed through a repeated survey of 63 voluntary participants 2 weeks apart.
RESULTS: The exploratory factor analysis extracted 10 components: patient engagement in patient safety (six items), managerial response to patient safety risks (four items), perceived management support (five items), staff empowerment (four items), staffing and workloads (four items), reporting of adverse events (three items), defensive medical practice (three items), work commitment (three items), training (two items) and transfer and handoff (three items). A good model fit was found in the confirmatory factor analysis: χ2/df=1.822, standardised root mean residual=0.048, root mean square error of approximation=0.038, comparative fit index=0.921, Tucker-Lewis index=0.907. The PSC scale had a Cronbach's α coefficient of 0.89 (0.59-0.90 for dimensional scales) and a test-retest reliability of 0.81 (0.63-0.87 for dimensional reliability), respectively. The intracluster correlation coefficients confirmed a hierarchical nature of the data: individual health workers nested within MCH institutions.
CONCLUSION: The PSC scale for MCH institutions has acceptable reliability and validity. Further studies are needed to establish benchmarking in a national representative sample through a multilevel modelling approach. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Through a multi-stage research progress of quantitative and qualitative methods, a patient safety culture (PSC) scale for maternal and child healthcare (MCH) institutions was developed and verified on the basis of views and responses from different stakeholders, including front-line staff, managers, patients and experts in relevant fields.

A hierarchical nature of PSC within MCH institutions in China has been preliminarily explored by intracluster correlation coefficients and design effects.

Like most PSC tools, the PSC scale for MCH institutions collects data based on subjective reporting, in the absence of observable indicators.

The results are context specific and caution should be taken when generalising the results.

Introduction

The burden of unsafe healthcare is overwhelming,1–3 resulting in increasing attention all over the world.4–7 One of the most salient challenges for building a safer healthcare system is to develop a just culture,8 9 in which common values, beliefs, attitudes, norms and behavioural characteristics on patient safety are shared by members of an organisation.10 11 This is often called patient safety culture (PSC) or patient safety climate. A good PSC leads to good attitudes and behaviours of health workers, better patient care outcomes and more resilient organisations.12–18

It is important to measure PSC before any targeted intervention strategies are developed.11 19 PSC can be observed through a range of organisational behaviours, such as leadership, communication, teamwork, error reporting, non-punitive approach to errors, continuous learning and evidence-based practice and so on.20 21 There have been a number of PSC measurement tools widely used around the world, such as the Hospital Survey on Patient Safety Culture (HSPSC),22 the Patient Safety Climate in Healthcare Organizations (PSCHO),23 the Safety Attitudes Questionnaire,24 and the Manchester Patient Safety Assessment Framework.25 Several PSC tools have also been made available in China.26–31 However, there is a consensus that PSC measurements need to adapt to the contexts of services, institutions, units and even individuals.32–36

Maternal and child healthcare (MCH) is one of the most important concerns in the sustainable development goals.37 MCH services refer to care for children during growth and development period and care for women during pregnancy, childbirth and postpartum period. Such services involve not only medical interventions on illness conditions, but also health promotion activities and preventive care. Integration of these services is essential to ensure optimal care outcomes, which often involves multiple providers and cross-organisational collaborations. Unfortunately, MCH consumers (women and children) are usually disadvantaged in access to healthcare due to their relatively low socioeconomic status.38 39 China has 3078 MCH institutions, with the majority (99.7%) owned by the government.40 The comprehensive network of MCH institutions covers all geographic catchments at the provincial, municipal and county levels.41 Besides providing MCH services, MCH institutions are considered part of the public health management system in China, taking a leading role in MCH care associated matters such as MCH statistics, workforce development and service supervisions. This network is believed to have played a critical role in China’s remarkable achievements on reducing under-five mortality rate and maternal mortality ratio42 and China has been evaluated by WHO as one of 10 low-income and middle-income ‘fast-track’ countries for MCH.43 As China has increasingly participated in global health governance, China’s model and experiences in MCH has been introduced into many low-income and middle-income countries.44–46

There is a paucity in the literature measuring PSC in MCH care settings, except for some studies in MCH clinical services.47 48 The existing PSC tools, which mainly focus on hospital services, may not be applicable to MCH settings due to the special nature of MCH services, especially for those public health services.49 Thus, this study aimed to develop a PSC scale tailored to the specific context of MCH institutions in China. This scale can be used to measure PSC in MCH care settings and helpful to develop other PSC tools for public health services.

Methods

Theoretical framework development

A theoretical framework was developed for conceptualising PSC through in-depth interviews (November 2014–April 2015) with 79 MCH workers (20 managers, 59 care providers) and 39 patients from six MCH institutions (three in Hebei province and three in Beijing). A PSC framework emerged as a result of both inductive (based on the existing PSC theories) and deductive (open coding arising from data50) coding, containing 12 components and 69 items. Findings of the in-depth interviews and informed consent statement were published elsewhere.51

The content validity of the framework was evaluated through Delphi expert consultations. Fifteen experts (in the areas of MCH, patient safety, health services management, nursing management and evidence-based medicine) were invited to participate in three-round mail/email consultations (from September to November 2015). This resulted in a consensus of a 12-component framework, comprising 67 items: 30 items were modified to avoid ambiguity; three were moved from their prior assigned components to new ones; six items were rephrased into three; one item was deemed irrelevant and removed; two new items were added.

Item reduction

The theoretical framework was transformed into a questionnaire (the PSC survey for maternal and child health institutions, PSC scale for MCH institutions (PSCS-MCHI) (V.1.0)), with each item being rated on a 5-point Likert scale: 1=strongly disagree, 2=disagree, 3=neither, 4=agree, 5=strongly agree. Two additional items were added in line with the HSPSC developed by the Agency for Healthcare Research and Quality,22 52 measuring ‘overall grade of patient safety’ (5=excellent, 4=very good, 3=acceptable, 2=poor, 1=failing) and ‘number of events reported in the last 12 months’ (1=no events, 2=1 to 2 events, 3=3 to 5 events, 4=6 to 10 events, 5=11 to 20 events, 6=21 events or more), respectively.

In January 2016, we invited 61 health workers from three MCH institutions in Beijing to participate in six focus groups (7–12 participants with different backgrounds for each group, two groups per institution). They completed the questionnaire first and then offered recommendations for improving the questionnaire. On average, the participants spent 12 min to complete the questionnaire (including the sociodemographic questions). As a result of the focus groups, wording of several questions was modified, along with a more detailed instruction for completion (PSCS-MCHI (V.2.0)).

In January and February 2016, a pilot survey was conducted on 429 MCH workers (including 46 repeated survey 2 weeks apart, according to approximately 10% sampling design in each institution and the principle of voluntary participation) from the same three MCH institutions in Beijing. Both classical test theory and item response theory53 were applied in data analyses: 21 items were filtered out, resulting in a 46-item scale (PSCS-MCHI (version 3.0)).

Validation study

The validation study was conducted in Zhejiang province from October 2016 to January 2017. Zhejiang has 88 MCH institutions.40 A call for expression of interest resulted in 14 MCH institutions participating in the study, including all levels of MCH institutions: one provincial hospital, three municipal hospital, one county-level hospital and nine county-level centre (online supplementary appendix A shows the characteristics of the participating institutions).

All MCH workers from the participating institutions were eligible for this study, including managers, medical doctors, nurses, midwives, public health workers, allied health workers, pharmacists and support workers (eg, IT technicians, administrative staff, security workers and cleaners). They were invited to self-complete the survey anonymously on a voluntary based. A total of 1329 questionnaires were dispatched (along with an informed consent statement, return of the questionnaire was deemed consent, which was completely voluntary and anonymous), representing 22% of the eligible participants. The participants were requested to return their completed questionnaires within 2 weeks to a deposit box placed in their institutions. All returned questionnaires with at least 80% of completed items (37 out of 46 items) were deemed valid and included in data analyses. This resulted in a final sample size of 1256 (91% of returned questionnaires). Of the participants, 70 volunteered to repeat the survey 10–14 days after completion of the initial survey: 63 (90%) returned a valid questionnaire in the repeated survey. This study’s ethical approval was obtained from Peking University Third Hospital Medical Science Research Ethics Committee (IRB00006761-M2014040).

Patients and public involvement

Patients and the public were not directly involved in the design and conception of this study.

Statistical analysis

The 46 items measuring PSC contained 15 negatively worded items. They were all coded in a consistent way, with one indicating the worst PSC and five indicating the best PSC. Item scores in the same component were summed up to generate a composite score. As most exiting PSC tools, the data of scores in this study are not weighted yet. The questionnaire also collected socio-demographic information of the respondents (eg, age, sex, educational attainment, years of working in health industry, years of working with current employer, position, average weekly workload and direct contacts with patients). The distribution of respondents with different characteristics was compared between the two samples split for factor analyses (see details below) and against the total sample.

The sample was randomly split into half for exploratory factor analysis (EFA, n=628) and confirmatory factor analysis (CFA, n=628), respectively. The EFA extracted a structure of the scale, which was then confirmed in the CFA. We used principal axis factoring with varimax rotation in EFA. The number of extracted factors was determined by eigenvalues (λ>1). An item with a lower than 0.4 factor loading score on any extracted factors or with serious cross-loadings (similar loadings on two or more factors) was removed.28 54 We performed CFA with robust maximum likelihood because the distribution of data was not normal.55 The fitness of model was examined using a series of indices: chi-square/df (χ2/df <3), standardised root mean residual (SRMR <0.05), root mean square error of approximation (root mean square error of approximation (RMSEA) <0.05), comparative fit index (CFI >0.90), and Tucker-Lewis index (TLI >0.90).56–58 In addition, the factor loading scores have to exceed 0.40.

The total sample was then used for further examinations of the reliability of the confirmed scale, and the hierarchical nature of the PSC data. The subscales (components) of the PSC scale were supposed to be correlated but with a <0.80 correlation coefficient to be considered unique and free from multicollinearity.52 We used Spearman rank-order correlation tests because the distribution of data was not normal (online supplementary appendix 1). We calculated Cronbach’s alpha (α) coefficients to assess the internal consistency of the PSC scale and its subscales. The test–retest reliability was assessed using Spearman rank-order correlation coefficients (for non-normal distributed data). A Cronbach’s α or a Spearman correlation coefficient over 0.70 was considered an indication of good reliability.11

We assumed that the PSC data had a hierarchical structure: individual health workers nested within MCH institutions. To test the hypothesis, we calculated intracluster correlation coefficients (ICCs) to assess both within-level homogeneity and between-level heterogeneity.59 The ICCs for each subscale were estimated as a ratio of between-level variance (σ2 w) to total variance(σ2 b) using an empty model (which is identical to a one-way random effect analysis of variance): ICC=σ2 b/(σ2 w +σ2 b).60 A high ICC value (>0.05) indicates that multilevel modelling should be considered for explaining between-level heterogeneity.28 52 But because ICC values are sensitive to the within-level sample size, we calculated design effects, which equal to 1+(average within level sample size−1)*ICC. The multilevel nature of data should be assumed when the design effect is higher than 2.52 61 In this study, we tested three sets of two-level models: individualposition (individuals clustered by position); individual-institution (individuals clustered by institution); individual-(position*institution) (individuals clustered by position and institution). In order to reach the least within-level sample size (≥10),60 only 36 groups with 10 or more individuals in each group (up to 985 individuals in total) were included in the individual-(position*institution) analysis.

We used Epidata V.3.1 software for data entry, Mplus V.5.1 software for CFA, and SPSS V.20.0 software for all other analyses.

Results

Characteristics of respondents

Most respondents (87%) were female; about half were younger than 35 years old; 72% obtained a university degree. The largest group of respondents was medical doctors (29%), followed by nurses and midwives (28%), allied health (13%) and public health workers (12%). No significant differences were found in the characteristics of respondents between the samples for EFA and CFA (table 1).

Table 1

Characteristics of respondents (%)

Characteristics	Total(N=1256)	Sample for EFA (N=628)	Sample for CFA(N=628)	X2	P value
Age (years)
≤24	9.39	9.71	9.08	3.787	0.285
25–34	39.65	40.29	39.01
35–44	28.98	26.59	31.37
≥45	21.89	23.25	20.54
Missing	0.08	0.16	0.00
Sex
Male	12.74	12.42	13.06	0.157	0.692
Female	83.52	84.24	82.80
Missing	3.74	3.34	4.14
Education
Up to secondary education	7.56	8.60	6.53	2.344	0.504
Associate degree	20.14	20.06	20.22
Bachelor degree	61.23	59.87	62.58
Master degree or higher	10.75	11.15	10.35
Missing	0.32	0.32	0.32
Years of working in health industry
<1	6.69	7.17	6.21	4.906	0.428
1–5	22.45	21.82	23.09
6–10	21.89	23.73	20.06
11–15	12.98	11.46	14.49
16–20	13.06	12.74	13.38
≥21	22.61	22.77	22.45
Missing	0.32	0.32	0.32
Years of working with current employer
<1	9.71	10.67	8.76	8.246	0.143
1–5	27.63	26.59	28.66
6–10	21.97	23.57	20.38
11–15	12.50	10.35	14.65
16–20	10.35	10.03	10.67
≥21	17.44	18.15	16.72
Missing	0.40	0.64	0.16
Average weekly workload (hours)
≤8	63.61	64.65	62.58	2.420	0.298
9–10	28.82	27.23	30.41
≥11	7.17	7.96	6.37
Missing	0.40	0.16	0.64
Direct contacts with patients
Yes	85.51	85.99	85.03	0.228	0.633
No	9.08	9.55	8.60
Missing	5.41	4.46	6.37
Institution
Provincial MCH hospital	23.57	23.25	23.89	0.336	0.953
Municipal MCH hospital	41.64	41.40	41.88
County MCH hospital	7.88	8.28	7.48
County MCH centre	26.91	27.07	26.75
Position
Manager	7.56	7.48	7.64	1.714	0.974
Medical doctor	29.14	29.14	29.14
Nurse	24.12	23.41	24.84
Allied health	12.98	13.22	12.74
Midwife	3.98	3.82	4.14
Pharmacist	4.62	5.10	4.14
Public health worker	11.86	12.42	11.31
Others	5.65	5.25	6.05
Missing	0.08	0.16	0.00

Percentages may not add to 100 due to rounding.

CFA, confirmatory factor analysis; EFA, exploratory factor analysis; MCH, maternal and child healthcare.

Construct validity of the PSC scale

The EFA resulted in a scale with 10 components and a total of 37 items (table 2). Nine items were removed: four items (Q92, Q56, Q43 and Q46) were removed due to low factor loadings (≤0.40); three (Q21, Q81 and Q114) were removed due to serious cross-loadings; one (Q53) was removed because it was the sole item for an additional factor (factor 11). In addition, another item (Q72) was removed because it loaded high on a factor (Factor 1) deviated from its prior assigned component.

Table 2

Item factor loadings: results from EFA (principal-axis factoring with varimax rotation) and CFA (robust maximum likelihood)

Item	EFA (N=628)											CFA (N=628)
	1	2	3	4	5	6	7	8	9	10	11
1. Patient engagement
Q125. I respect patient rights and willingness	0.852	0.113	0.078	0.104	0.002	0.017	0.052	0.150	−0.003	0.077	−0.001	0.793
Q124. We emphasise health education of patients	0.805	0.132	0.036	0.195	−0.033	0.109	0.116	0.174	0.079	0.074	0.022	0.844
Q123. We often take advices from patients to improve services	0.786	0.120	0.026	0.163	−0.014	0.050	0.103	0.119	0.147	0.108	0.015	0.813
Q121. I inform patients as fully as I can (eg, alternative plans and risks)	0.772	0.168	0.072	−0.051	−0.021	0.075	0.024	0.099	0.127	0.062	0.067	0.681
Q122. I respond to any questions asked by patients	0.767	0.092	0.092	0.051	0.073	0.145	0.017	0.099	0.080	0.003	0.009	0.599
Q126. Patients are encouraged to involve in patient safety initiative	0.701	0.132	0.042	0.271	−0.041	0.151	0.031	0.050	0.051	0.059	0.002	0.708
(D)Q72. When a potential risk emerges, we will remedy it timely to avoid worsening situation	0.445	0.281	0.115	−0.018	−0.073	0.304	0.089	0.269	0.173	0.230	0.003	–
2. Managerial response to patient safety risks
Q54. Timely feedback is given to reported adverse events	0.194	0.701	0.179	0.175	0.064	0.197	0.059	0.117	0.087	0.125	−0.085	0.740
Q71. Apart from incidents, minor errors or near misses are treated seriously as well	0.201	0.694	0.108	0.154	−0.020	0.027	0.022	0.098	−0.034	0.101	−0.043	0.603
Q52. Frontline workers are encouraged to report adverse events	0.233	0.662	0.198	0.255	−0.048	0.207	0.028	0.122	0.098	0.114	−0.073	0.773
Q25. Risk management and responding mechanism has been established	0.205	0.600	0.030	0.259	0.003	0.192	0.136	0.243	0.145	0.132	0.008	0.766
(D)Q21. In the organisation, regulations and procedures are improving continuously	0.259	0.461	0.047	0.430	−0.019	0.103	0.176	0.250	0.294	0.071	0.095	–
3. Perceived management support
Q103. In the institution, public health departments and their staff are subject to ignorance or prejudice (R)	0.026	0.155	0.746	−0.050	0.166	0.002	0.056	0.101	0.109	−0.028	0.028	0.524
Q35. In resource allocation within the institution, priority is given to clinical rather than public health activities (R)	0.061	0.250	0.699	0.086	0.129	0.033	0.147	−0.081	0.064	0.002	0.111	0.459
Q11. Management support is not demonstrated for accomplishing all public health tasks (R)	0.097	0.249	0.605	0.041	−0.024	0.043	0.278	0.061	−0.057	0.135	0.132	0.561
Q63. Individual workers involved in adverse events are prejudiced against by colleagues (R)	0.149	−0.128	0.596	0.050	0.106	0.216	0.046	0.154	0.121	−0.039	−0.126	0.470
Q44. Communication is unsatisfactory between supervisors and subordinates (R)	0.183	0.163	0.434	0.271	0.091	−0.011	0.327	0.036	0.017	0.226	−0.338	0.551
4. Staff empowerment
Q14. Managers are committed to creating a good working atmosphere	0.242	0.254	0.109	0.723	0.021	0.102	0.002	−0.006	0.052	0.237	−0.012	0.690
Q22. In the institution, fair and feasible motivation mechanisms exist	0.040	0.189	0.030	0.702	0.118	0.149	0.002	0.084	0.179	0.043	−0.125	0.617
Q26. Frontline workers are able to get involved in decision-making	0.134	0.073	−0.045	0.649	0.154	0.165	−0.039	0.108	0.225	−0.006	0.090	0.617
Q16. Managers make efforts to improve facilities and environments	0.276	0.298	0.204	0.544	−0.004	0.031	−0.068	0.060	−0.011	0.226	−0.025	0.585
(D)Q81. Managers emphasise continuous learning and training of health workers	0.200	0.373	0.090	0.444	−0.034	0.040	0.080	0.306	0.400	−0.086	0.032	–
5. Staffing and workloads
Q33. Staffing is far from sufficient to meet workload requirements (R)	−0.024	−0.042	0.047	0.017	0.788	−0.081	0.098	−0.031	−0.031	0.052	−0.029	0.765
Q34. Overloading prevents us from providing the best services we can (R)	−0.007	−0.025	0.148	0.043	0.775	−0.027	0.107	−0.034	0.106	0.063	−0.037	0.780
Q32. I often feel too busy in work (R)	−0.013	−0.037	0.203	0.132	0.593	−0.042	0.088	−0.017	−0.197	0.070	−0.088	0.504
Q36. Frontline workers have enough time to prepare from servicing one patient or procedure to another	−0.022	0.086	−0.156	0.011	0.565	0.004	−0.279	−0.033	0.363	0.158	0.129	0.425
(D)Q92. I feel burned out from my work (R)	−0.002	0.221	0.287	0.145	0.404	0.060	0.264	0.251	0.001	−0.050	0.002	–
6. Reporting of adverse events
Q67. If an incident happens and may harm the patient, I will report it	0.166	0.139	0.047	0.217	−0.065	0.775	0.113	0.099	0.062	0.092	0.113	0.789
Q61. If an incident happens and may not harm the patients, I will report it as well	0.071	0.128	0.080	0.087	0.042	0.771	−0.055	0.075	0.094	0.019	−0.084	0.589
Q66. If an incident involves my colleagues, I will report it as well	0.292	0.149	0.097	0.112	−0.176	0.697	0.093	0.131	−0.015	0.113	0.098	0.846
(D)Q56. We emphasise analysing and learning from adverse events	0.310	0.268	0.057	0.075	−0.104	0.325	0.250	0.300	0.240	0.233	−0.058	–
7. Defensive medical practice
Q112. To avoid dispute, I may yield to patient demands rather than adhere to principles and guidelines (R)	0.067	0.075	0.065	−0.054	0.012	−0.019	0.737	0.010	0.130	0.027	0.007	0.461
Q111. To mitigate high risks, we may refuse some patients with conditions we may be able to handle (R)	0.138	−0.027	0.136	−0.034	0.154	0.111	0.710	0.036	0.202	−0.084	0.056	0.542
Q12. Managers cannot give priorities to patient safety for concerns of profits or reputations (R)	0.044	0.250	0.299	−0.041	0.118	0.067	0.527	0.069	−0.113	0.149	0.090	0.572
(D)Q114. I admit that unnecessary treatments and interventions exist in the institution (R)	0.018	−0.113	0.436	0.240	0.057	0.073	0.457	0.136	−0.129	0.057	0.194	–
(D)Q43. In the institution, cross-unit teamwork is not satisfactory (R)	0.017	0.075	0.325	0.255	0.119	−0.070	0.366	0.085	−0.030	0.229	−0.365	–
8. Work commitment
Q93. I can serve patients with compassion and empathy	0.216	0.203	0.035	0.095	−0.013	0.119	0.052	0.743	0.030	0.045	−0.096	0.634
Q94. I have patience and good attitudes in my work	0.243	0.145	0.062	0.152	0.055	0.184	0.115	0.695	−0.016	0.122	0.015	0.736
Q84. I need to learn continuously	0.333	0.233	0.231	0.021	−0.103	0.002	−0.055	0.534	0.012	0.127	0.173	0.655
9. Training
Q86. In the institution, we are provided with enough training to improve competency	0.157	0.078	0.060	0.276	0.064	0.084	0.141	0.043	0.728	0.087	−0.057	0.792
Q85. New employees are provided with enough training to adapt to regulations and procedures quickly	0.202	0.104	0.105	0.168	−0.009	0.072	0.098	−0.028	0.721	0.189	−0.024	0.804
10. Transfer and handoff
Q41. Cross-institutional referrals are efficient to ensure safe care for women and children	0.118	0.211	−0.042	0.113	0.221	0.125	0.009	0.018	0.110	0.727	−0.072	0.606
Q42. Supervision and coordination of care within the same area across MCH institutions is satisfactory	0.137	0.187	0.041	0.176	0.157	0.063	0.074	0.173	0.125	0.720	0.052	0.734
Q45. Handoff is handled seriously and carefully	0.292	−0.035	0.331	0.041	−0.220	0.137	0.015	0.311	0.298	0.433	0.118	0.534
(D)Q46. Teamwork is satisfactory in my unit	0.281	−0.081	0.309	0.101	−0.131	0.099	0.036	0.388	0.378	0.394	0.000	–
(D)Q53. In the institution, adverse events are mostly attributable to individual reasons (R)	0.110	−0.072	0.150	0.018	−0.049	0.038	0.162	0.021	−0.035	0.038	0.782	–

(R) indicates reverse score for a negatively worded item.

(D) indicates an item that was eliminated as a result of EFA.

CFA, confirmatory factor analysis; EFA, exploratory factor analysis; MCH, maternal and child healthcare.

The 10 components were named as: (1) patient engagement (six items); (2) managerial response to patient safety risks (four items); (3) perceived management support (five items); (4) staff empowerment (four items); (5) staffing and workloads (four items); (6) reporting of adverse events (three items); (7) defensive medical practice (three items); (8) work commitment (three items); (9) training (two items); (10) transfer and handoff (three items). The CFA confirmed that the 10-component structure of PSC scale (37 items) had a good model fit: χ2=1040.263, df=571, p<0.001, χ2/df=1.822, SRMR=0.048, RMSEA=0.038 (90% CI 0.034 to 0.041), CFI=0.921, TLI=0.907. All of the 37 items had a factor loading exceeding 0.4 (table 2).

PSC scores and correlations

The mean item scores ranged from 2.18 to 4.16, with a ceiling effect (percentage of respondents with the maximum score) ranging from 0.48% to 23.81% and a floor effect (percentage of respondents with the minimum score) ranging from 0% to 20.78%. The respondents had a mean subscale score ranging from 2.37 (‘staffing and workloads’) to 4.02 (‘work commitment’). Ceiling effects and floor effects on subscale scores were found in 0.16%–5.10% and 0%–2.23% of respondents, respectively (details can be found in online supplementary appendix B).

The subscales had weak to medium positive correlations (p<0.001), except for the component of ‘staffing and workloads’ (table 3). ‘Staffing and workloads’ was found to be negatively correlated with ‘reporting of adverse events’ and ‘work commitment’ (p<0.05). It had no significant correlations (p>0.05) with ‘patient engagement’, ‘managerial response to patient safety risks’ and ‘transfer and handoff’.

Table 3

Spearman correlations, internal consistency and test–retest reliability of patient safety culture subscales

	Subscale										PSG	NER
	1	2	3	4	5	6	7	8	9	10
Spearman rank-order correlation (n=1256)
1. Patient engagement	1
2. Managerial response to patient safety risks	0.518**	1
3. Perceived management support	0.256**	0.353**	1
4. Staff empowerment	0.454**	0.591**	0.272**	1
5. Staffing and workloads	−0.042	−0.014	0.080*	0.183**	1
6. Reporting of adverse events	0.428**	0.446**	0.239**	0.359**	−0.056*	1
7. Defensive medical practice	0.232**	0.268**	0.416**	0.130**	0.085*	0.182**	1
8. Work commitment	0.450**	0.436**	0.252**	0.293**	−0.111**	0.294**	0.158**	1
9. Training	0.376**	0.382**	0.224**	0.430**	0.140**	0.303**	0.237**	0.212**	1
10. Transfer and handoff	0.415**	0.448**	0.336**	0.390**	0.033	0.331**	0.229**	0.321**	0.401**	1
Patient safety grade (PSG)	0.232**	0.242**	0.191**	0.277**	0.145**	0.189**	0.244**	0.135**	0.274**	0.220**	1
Number of events reported (NER)	−0.009	0.020	0.044	0.005	−0.052	0.066*	−0.029	0.059*	−0.007	−0.003	−0.090*	1
Mean (SD)	3.93 (0.47)	3.77 (0.50)	3.40 (0.62)	3.52 (0.61)	2.37 (0.65)	3.70 (0.55)	3.45 (0.67)	4.02 (0.44)	3.54 (0.75)	3.88 (0.49)	3.86 (0.70)	1.24 (0.50)
Cronbach’s α	0.900	0.828	0.726	0.760	0.690	0.774	0.591	0.730	0.789	0.657	–	–
Test–retest reliability (n=63)	0.835	0.868	0.816	0.831	0.874	0.713	0.839	0.634	0.805	0.697	0.698	0.954

Spearman rank-order correlation: **p<0.001; *p<0.05.

Overall, the respondents gave a positive rating to patient safety: 16.64% ‘excellent’, 54.58% ‘very good’ and 27.45% ‘acceptable’, compared with 1.16% ‘poor’ and 0.17% ‘failing’. The majority (78.59%) did not report any events over the past 12 months, while 18.47% reported 1–2 events, 2.85% reported 3–5 events and 0.08% reported 6–10 events. Patient safety grade was correlated with all of the 10 PSC subscales (p<0.001). Whereas, the number of events reported was only correlated with ‘reporting of adverse events’ and ‘work commitment’ (p<0.05).

Reliability of the PSC scale

The PSC scale had a Cronbach’s α coefficient of 0.89, with seven dimensional α exceeding 0.70. Three subscales had lower than 0.7 α coefficients: 0.591 for ‘defensive medical practice’ (increased to 0.645 if Q12 was removed); 0.690 for ‘staffing and workloads’ (increased to 0.715 if Q36 was removed); 0.657 for ‘transfer and handoff’ (increased to 0.731 if Q45 was removed). Further analyses revealed negligible impacts on the validity of the PSC scale by removing Q12, Q36 and Q45 (online supplementary appendix C–E). The test–retest reliability of the PSC reached 0.81, with eight subscales exceeding 0.7 and two exceeding 0.6 (table 3).

ICCs and design effects

Low ICC values were observed, possibly as a result of large within-cluster sample size. In this study, we had an average of 157 individuals per position and 90 individuals per institution. Only two subscales in the individual-position analyses, three subscales in the individual-institution analyses and seven subscales in the individual−‘position×institution’ analyses generated an ICC value greater than 0.05 (table 4).

Table 4

ICCs and design effects in analyses with individuals clustered by position, institution and position†institution (N=1256)

Subscale	Individual clustered by position					Individual clustered by institution					Individual clustered by (position*institution)
	F value	σ2 w	σ2 b	ICC	Design effect	F value	σ2 w	σ2 b	ICC	Design effect	F value	σ2 w	σ2 b	ICC	Design effect
Patient engagement	1.752	0.217	0.001	0.005	1.716	2.489*	0.215	0.004	0.018	2.626‡	2.074**	0.223	0.010	0.043	2.116‡
Managerial response to patient safety risks	2.593*	0.248	0.003	0.012	2.865‡	2.909**	0.245	0.007	0.028	3.472‡	2.499**	0.240	0.016	0.063‡	2.625‡
Perceived management support	3.089*	0.377	0.007	0.018	3.844‡	8.776**	0.353	0.049	0.122‡	11.848‡	3.818**	0.340	0.052	0.133‡	4.449‡
Staff empowerment	4.104**	0.362	0.010	0.027	5.194‡	3.014**	0.361	0.012	0.032	3.863‡	2.746**	0.341	0.030	0.081‡	3.102‡
Staffing and workloads	8.537**	0.412	0.029	0.066‡	11.259‡	1.837*	0.426	0.003	0.007	1.622	3.512**	0.389	0.047	0.108‡	3.803‡
Reporting of adverse events	4.726**	0.301	0.006	0.020	4.049‡	1.323	0.306	0.002	0.006	1.578	2.165**	0.295	0.013	0.042	2.097‡
Defensive medical practice	2.316*	0.444	0.004	0.009	2.393‡	8.759**	0.414	0.040	0.088‡	8.841‡	4.017**	0.400	0.052	0.115	3.991
Work commitment	2.268*	0.193	0.002	0.010	2.600‡	1.739*	0.193	0.002	0.010	1.913	2.229**	0.184	0.010	0.052	2.340
Training	9.389**	0.534	0.029	0.052‡	9.036‡	8.609**	0.518	0.060	0.104‡	10.239‡	4.545**	0.488	0.076	0.135	4.504
Transfer and handoff	1.000	0.242	0.001	0.004	1.642	2.218*	0.240	0.004	0.016	2.459‡	1.816*	0.241	0.008	0.032	1.835
Patient safety grade	3.494*	0.471	0.013	0.027	5.190‡	3.401**	0.472	0.128	0.213‡	19.987‡	2.793	0.445	0.039	0.081‡	3.095‡
Number of events reported	3.619*	0.243	0.003	0.012	2.902‡	3.525**	0.240	0.007	0.018	3.522‡	2.889	0.244	0.019	0.072‡	2.878‡

Individuals clustered by position: 1256 individuals (level 1) nested within eight positions (level 2).

Individuals clustered by institution: 1256 individuals (level 1) nested within 14 institutions (level 2).

σ2 w: within-level variance; σ2 b: between-level variance.

*F value: **p<0.001; *p<0.05.

†Individuals clustered by (position†institution): 985 individuals (level 1) nested within 36 (position†institution) groups (level 2), excluding 62 groups with less than 10 individual compositions.

‡A value exceeding the cut-off threshold (ICC>0.05, design effect>2.00).

ICC, intracluster correlation coefficient.

The design effects (adjusted by the number of individuals within a cluster) revealed eight subscales in the individual-position analyses, seven subscales in the individual-institution analyses and nine subscales in the individual-‘position*institution’ analyses exceeding the threshold of 2.00 (table 4). The design effects of ‘patient safety grade’ and ‘number of events reported’ also exceeded 2.00 in all of the three sets of ICC analyses (further details on ICCs and design effects can be found in online supplementary appendix F).

Discussion

Components of PSCS-MCHI

This study developed a PSCS-MCHI, comprising 10 subscales measured by 37 items. The PSCS-MCHI is tailored to the special context of MCH institutions in China. Its compositions share some common themes captured in the existing tools measuring PSC. For example, ‘managerial response to patient safety risks’, ‘perceived management support’, ‘staffing and workloads’, ‘reporting of adverse events’, ‘training’ and ‘transfer and handoff’ have been featured in the HSPSC and the PSCHO.22 23 However, there are also several themes that are particularly highlighted in the PSCS-MCHI, including ‘defensive medical practice’, ‘work commitment’, ‘patient engagement’ and ‘staff empowerment’. Questions in relation to these themes may appear in other PSC tools, but most are not treated as a separate subscale.

The underlined reasons behind the emergence of these new subscales have to be traced back from the special contexts of MCH services in China. China has experienced unprecedented economic growth over the past few decades, accompanied by considerable advancement in medical technologies. This has stimulated very high consumer expectations on quality of healthcare. Such high expectations are further compounded by the one-child family planning policy adopted in China since late 1970s (although the government has recently relaxed this policy by allowing two children for each family). However, medical care outcomes often fall short of expectations. The contradiction between reality and expectations often leads to consumer distrust in medical practice, which has been further exacerbated by soaring financial burdens of medical care.62 Consequently, defensive medical practice, in particular over-provision of service, has become a growing concern from the consumer perspective. Consumers demand increasing power in clinical decision making. Meanwhile, health workers also want consumers to be more accountable to their own healthcare. This is particularly important for MCH care, which requires consumers to comply with a plan involving continuing and coordinated services across a long period of time. Although patients have enjoyed a high level of freedom in choosing care providers in China, clinical autonomy of health workers has been subject to increasing restrictions due to policy and managerial interventions. It is important to note that health workers, including medical practitioners, are usually salaried full-time employees of health organisations in China. Empirical evidence shows that a lack of participation of health workers in management decisions may jeopardise patient care outcomes.63 Understandably, ‘staff empowerment’ and ‘work commitment’ are valued by health workers as important elements of PSC.

Psychometric properties of PSCS-MCHI

The PSCS-MCHI (37 items) has acceptable reliability and validity. The scale was developed through a rigorous process, involving in-depth interviews, expert consultations, focus groups, and quantitative psychometric testing (figure 1). The 10 component structure of the PSCS-MCHI extracted by the EFA is confirmed by the CFA. The scale has demonstrated acceptable internal consistency and test–retest reliability. The hierarchical nature of the PSC data informed by the ICCs and design effects indicates that PSC is not just a simple sum up of individual characteristics. PSC is deeply rooted and shaped by the characteristics of collective bodies in an organisation, beyond the level of individuals. This finding aligns with the definition of culture and is consistent with studies conducted elsewhere.28 52 Interestingly, our results show that variations in ‘patient engagement’ and ‘transfer and handoff’ are more likely to be shaped by institutional differences rather than position differences. By contrast, variations in ‘staffing and workloads’, ‘reporting of adverse events’ and ‘work commitment’ are more likely to be shaped by position differences rather than institutional differences. The other components of PSC, including ‘managerial response to patient safety risks’, ‘perceived management support’, ‘staff empowerment’, ‘defensive medical practice’ and ‘training’ are sensitive to both position and institutional differences.

Figure 1

Development of patient safety culture survey for maternal and child health institutions (PSCS-MCHI).

We found that ‘staffing and workloads’ is negatively correlated with ‘reporting of adverse events’ and ‘work commitment’, and it has no significant correlations with ‘patient engagement’, ‘managerial response to patient safety risks’, and ‘transfer and handoff’. Interpretations of these results need to be cautious. It has been widely agreed that a shortage of workforce and excessive workloads are detrimental to patient safety.64 65 However, in a system where patients enjoy freedom to choose care providers, crowdedness (high workloads) can be a reflection of patient preference and perceived high quality of care.51 66 In China, there is no ‘safe workload’ policy. Managers often encourage health workers to take on more workloads to demonstrate loyalty and commitment to the organisations.62 The hierarchical nature of the PSC data may also confound the inter-subscale correlation analyses based on data measured at the individual level. MCH institutions with higher patient loads are more likely to employ health workers with higher qualifications,67 68 increasing their prospect of nurturing a better PSC.62 69

PSC in MCH institutions

Although there is a lack of benchmarking criteria, this study revealed particular low scores in ‘staffing and workloads’, ‘perceived management support’ and ‘defensive medical practice’. Indeed, understaffing and overloading is a common complaint in China’s healthcare industry.26 27 62 Meanwhile, defensive medical practice is also prevalent and has attracted increasing criticism, damaging medical professionalism.70–72 It is important to notice that the shortage of perceived management support in the MCH setting often points to a lack of acknowledgement on the contributions of preventive care, poor communications, and prejudice to individuals who commit errors. Surprisingly, the respondents of this study gave relatively high ratings on patient safety and reported few events despite these low scores. Further studies are needed to better understand the situation of patient safety in MCH institutions.

While workforce development is important, more attention should be diverted to health promotion and preventive care in MCH settings. The lack of trust between patients and providers imposes a serious challenge to the development of PSC73 74 and adopt of a non-punitive approach to medical errors.8 75 76 Coordinated efforts of the society is needed to cope with this problem, involving all stakeholders including health workers, patients, health managers, policy makers and medical students.77–82

Suggestions drawn from the findings

There are several suggestions that can be drawn from the findings of this study. First, Staff empowerment is extracted as a component of PSC in this study and high involvement management has been proved to improve employees’ well-being.83 It is necessary to make a priority of promoting frontline workers’ engagement in decision-making processes, through creating a flat organisational structure, improving the decision-making capabilities of frontline workers, and ensuring them by policies and regulations.36–40 Second, considering that health service outcomes are produced by providers and customers together, patients should be encouraged to participate in all processes of healthcare decision making, which can be helpful to improve the relationship between providers and customers and thereby to reduce providers’ defensive medical practice. Third, overloading is an outstanding problem in patient safety, especially in China’s medical institutions. However, work commitment has been overemphasised to encourage health workers to bear overloaded work, which may harm their own health.84 It’s time to establish a ‘safe workload’ policy in China and put both patients and health workers in a safer place.

Limitations

This study has several limitations. First, the study sample was not randomly selected and it was just undertaken in Zhejiang province. A national representative study will be needed to establish benchmarking criteria, which is essential for appropriate data interpretation. This limitation also results in that the findings in this study are context specific and caution should be taken when generalising these findings. Second, this paper focuses on introducing a tailored tool for MCH institutions (PSCS-MCHI), including its components and psychometric properties, as well as indicating that PSC in MCH institutions exists potential multilevel heterogeneities in individual level, positional level and institutional level. Further analyses and studies should be aimed to explore multilevel variables how to influence PSC of the organisation, such as gender, education, work experience and other personality traits in individual level, different groups in positional level, grade-ranking, scale and other organisational traits in institutional level. Third, although both patients and health workers were involved in developing the theoretical framework of the PSCS-MCHI, this questionnaire’s objects are defined as health workers, since PSC is a kind of common value shared by members of an organisation.10 11 However, considering that patient engagement is an important part of PSC, it should be necessary to make the PSCS-MCHI verified by patients or develop a specific subscale for patients in further studies.35

Conclusion

The PSCS-MCHI is a valid tool for assessing PSC in MCH settings. The scale comprises ten subscales, which have demonstrated a hierarchical structure of data. PSC in MCH settings is shaped by the characteristics of individuals, job positions and institutions. Multilevel modelling is advised for studies using the PSCS-MCHI.