Development of the Chinese version of the Hospital Autonomy Questionnaire: a cross-sectional study in Guangdong Province.

OBJECTIVE: We aimed to develop a questionnaire for quantitative evaluation of the autonomy of public hospitals in China.
METHOD: An extensive literature review was conducted to select possible items for inclusion in the questionnaire, which was then reviewed by 5 experts. After a two-round Delphi method, we distributed the questionnaire to 404 secondary and tertiary hospitals in Guangdong Province, China, and 379 completed questionnaires were collected. The final questionnaire was then developed on the basis of the results of exploratory and confirmatory factor analysis.
RESULTS: Analysis suggested that all internal consistency reliabilities exceeded the minimum reliability standard of 0.70 for the α coefficient. The overall scale coefficient was 0.87, and 6 subscale coefficients were 0.92 (strategic management), 0.81 (budget and expenditure), 0.85 (financing), 0.75 (financing, medical management), 0.86 (human resources) and 0.86 (accountability). Correlation coefficients between and among items and their hypothesised subscales were higher than those with other subscales. The value of average variance extracted (AVE) was higher than 0.5, the value of construct reliability (CR) was higher than 0.7, and the square roots of the AVE of each subscale were larger than the correlation of the specific subscale with the other subscales, supporting the convergent and discriminant validity of the Chinese version of the Hospital Autonomy Questionnaire (CVHAQ). The model fit indices were all acceptable: χ(2)/df=1.73, Goodness of Fit Index (GFI) = 0.93, Adjusted Goodness of Fit Index (AGFI) = 0.91, Non-Normed Fit Index (NNFI) = 0.96, Comparative Fit Index (CFI) = 0.97, Root Mean Square Error of Approximation (RMSEA) = 0.04, Standardised Root Mean Square Residual (SRMR) = 0.07.
CONCLUSIONS: This study demonstrated the reliability and validity of a CVHAQ and provides a quantitative method for the assessment of hospital autonomy. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

To the best of our knowledge, this is the first study to carry out a quantitative investigation of public hospital autonomy in China.

By surveying >400 public hospitals, we developed the first version of a Hospital Autonomy Questionnaire, designed to improve our understanding of the relationship between public hospitals and the government.

We verified the reliability and validity of our questionnaire, which provides a quantitative method for the assessment of hospital autonomy in China.

The survey was conducted only in Guangdong Province; thus, the generalisation of the results to other provinces should be made with caution.

This was a cross-sectional study that could not illustrate the underlying trend of hospital autonomy. In the future, panel data should be collected.

Introduction

In developing countries, public hospitals are normally characterised by inefficient resource management, by low productivity due to their rigid hierarchical structures and by ineffective governmental administrative and financial controls.1 Turning public hospitals into autonomous entities is thought to be able to improve their performance.2 In China, since over 80% of hospitals are run by the state,3 increasing public hospital autonomy, which is also considered to be an effective way to improve the efficiency of public hospitals, is a key issue in health sector reform4 and is a particularly important issue in China. Studies on the evaluation of hospital autonomy in other countries have yielded mixed results. Govindaraj and Chawla,5 using the evaluation toolkit by Chawla et al, carried out five country case studies (Ghana, Kenya, Zimbabwe, India and Indonesia). However, the results varied by country, probably because of the short period of time that elapsed since the granting of autonomy in each country. Bossert et al6 measured hospital autonomy in Indonesia and found little evidence that its objectives were achieved. Sengooba et al7 performed another case study of eight hospitals in Uganda. Given the nature of the case study, the authors did not believe that their results should be used as the sole basis for national-level or international-level policy development. In addition, different authors carried out eight country or area case studies (UK, New Zealand, Australia, Hong Kong, Malaysia, Singapore, Indonesia and Tunisia) based on Harding and Preker's8 conceptual framework on hospital autonomy. Hawkins and Ham summarised these studies and found that autonomisation might improve efficiency in some areas but not in others.9 McPake et al10 reached a positive conclusion by studying five hospitals in Bogotá, Colombia, while Saleem M et al studied the autonomisation of teaching hospitals in Punjab and found that it has not yet yielded the hoped-for benefits.11 London investigated the impact of hospital autonomisation in Vietnam and also found mixed results, mainly due to the small sample size and data constraints.12 Some researchers attributed the mixed results to the small sample sizes and quality of data in these studies, which prevented researchers from conducting more elaborate statistical analyses and reaching robust conclusions.2

In China, public hospitals play a critical role in the healthcare system. According to the China Health Statistics Yearbook (2015),3 they provide medical services to 89.5% of the population, and the proportion of spending in public hospitals is 95.5% of the total national health expenditure in China. Currently, public hospitals are considered less efficient and a serious financial burden to the government.13 To address these problems, China began granting autonomy to some public hospitals in the early 1980s, including shifting partial decision-making control from the government to hospital managers, which has allowed public hospitals to make a profit from the medical services they provide. However, after >20 years, some research suggests that healthcare reform in China has not made significant progress.14 In 2012, China's State Council announced a new phase of healthcare reform, allowing for more private hospitals and granting greater public hospital autonomy to improve efficiency.15 Therefore, studying the impact of hospital autonomy on efficiency is important, and developing an effective tool for measuring such autonomy is the first challenge. With the Chinese government's support, and sufficient hospitals in Guangdong Province to provide a good sample, we carried out this study. The objective of this study was to develop a Chinese version of the Hospital Autonomy Questionnaire (CVHAQ), which can be used to evaluate the level of hospital autonomy and to help policymakers better understand the extent of policy implementation and formulate specific policies to redefine the relationship between health authorities and hospitals.

Methods

Theoretical model and questionnaire development

Although many researchers have proposed evaluation tools for hospital autonomy, there are three generic tools, namely, the Chawla et al,15 Over and Watanabe16 and Harding and Preker8 tools.2 The Chawla et al tool proposes three key areas to be analysed to determine the existing level of autonomy: administration, financing and inputs. The Over and Watanabe tool considers five elements of hospital structure: residual claimant status, decision right, degree of market exposure, availability of accountability mechanisms and extent of unfunded mandates. The Harding and Preker8 tool proposes five dimensions for analysis of the extent of hospital autonomy: decision right, market exposure, residual claimant, accountability and social functions. We modified the three existing tools to develop a CVHAQ and the flow chart of questionnaire development shown in figure 1.

Figure 1

Flow chart of questionnaire development.18 19

A systematic approach was used to generate the final questionnaire.18 19 The development of items for inclusion in the CVHAQ was carried out in five phases. First, an extensive literature review was conducted to select possible subscales for the questionnaire. The theoretical basis for hospital autonomy was the conceptual framework from Harding and Preker,8 which contained five subscales: decision rights, residual claimant, market exposure, accountability and social functions (figure 2). At the same time, we also learnt from the other two related measuring tools, and nine subscales were created to evaluate the extent of Chinese hospital autonomy: strategic management, budget and expenditure, finance, human resources, day-to-day management, residual claimant, market exposure, accountability and social functions. Second, after in-depth interviews and focus-group sessions, 40 potential subitems were developed. Third, 10 carefully selected experts—including 2 lawyers, 4 hospital management experts, 2 government officers, and 2 professors versed in statistics—were invited to discuss relevant items for inclusion or exclusion. We used the Delphi method to evaluate the items. In two rounds, the experts gave their scores in terms of linguistic expression, importance, clarity and correlation for each item, and then we deleted the items whose importance score was <7.5. The remaining 32 items constituted the initial shape of the questionnaire. Fourth, a pilot test with 26 selected hospitals was conducted to judge the appropriateness of each included question. Fifth, in a formal survey, six undergraduate students were selected as questionnaire interviewers, and two researchers in our focus group were designated as quality controllers. After investigation, we used the method of exploratory factor analysis to screen the items, and finally we deleted 10 items and extracted 6 factors. (All loadings were larger than 0.5.) To enhance the interpretability of the factors with high loadings, we rotated the extracted factors to a simple structure using the Varimax procedure (orthogonal rotation) to see if the result was consistent with our theoretical model. Furthermore, we evaluated the reliability of the questionnaire and Cronbach's α coefficients in all subscales higher than 0.70. Meanwhile, we calculated the value of average variance extracted (AVE; >0.5) and square roots of AVE to test the validity of the questionnaire. After some necessary linguistic revisions, the questionnaire, including six subscale (22 items) indicators of Chinese hospital autonomy, was finalised and used to build a Chinese hospital autonomy model.

Figure 2

Critical factors that influence organisational behaviour.8

Responses were rated on a 5-point Likert scale for hospital autonomy (1=totally constrained, 2=very constrained, 3=partly constrained, 4=hardly constrained and 5=not constrained). Items were forward scored on a 1–5-point scale (1=1, 2=2, 3=3, 4=4, 5=5), and subscale scores were computed as the sum of each item score; that is, the higher scores indicated higher autonomy.

Quality control measures: item generation

The items in this questionnaire were generated through literature reviews, in-depth interviews and focus-group sessions. First, we reviewed PubMed and the China National Knowledge Infrastructure (CNKI) to collect pertinent literature. Second, we set up a focus group to analyse the pertinent literature and preliminarily determined the subscales of our questionnaire. Finally, the members of the focus group reviewed the potential respondents or experts in depth. When item generation continued until no new items emerged, we established an item pool with 40 items.

Then the Delphi method was used for two rounds.

Pilot testing

The questionnaire was tested for how well respondents comprehended and correctly answered crucial questions, or to confirm that the questionnaire was user-friendly for the respondent and to ascertain how long it took to complete.

Formal survey

All of the questionnaire interviewers and quality controllers were systematically trained by the project investigator (PI) before the investigation. During the investigation, the questionnaire interviewers were responsible for connecting with respondents who were familiar with questions and reported regularly to the PI. The quality controllers were responsible for checking the integrity and accuracy of the finished questionnaire.

Data entry

Double-blind input, consistency check and logical check were used to control the quality of data entry.

Data collection

This survey was conducted in 2013, and the data collection process consisted of two parts. First, an electronic version of the questionnaire was sent to 404 secondary and tertiary hospitals in Guangdong Province with the support of the Health Bureau of Guangdong Province. Second, senior management staff of these hospitals were contacted and asked to organise persons familiar with these issues to complete the questionnaire. The completed questionnaires were then collected by the research team.

Data analysis

SPSS 19.0 for Windows was used for data analysis. First, Cronbach's α coefficients were calculated to evaluate the internal consistency reliability of the total questionnaire and each subscale. Cronbach's α coefficient ≥0.70 was considered as acceptable. Second, multitrait scaling analysis (using Pearson correlation analysis) was conducted to determine the item–subscale correlations. If the analysis showed a stronger correlation (correlation coefficient ≥0.70) between an item and its hypothesised subscale, and with a coefficient larger than those of other subscales, the questionnaire was considered well defined. To test the validity of the CVHAQ, an exploratory factor analysis was conducted, and the extracted factors were rotated to a simple structure by the Varimax procedure (orthogonal rotation) to see if the result would be consistent with the theoretical model. Varimax rotation can enhance the interpretability of the factors with high loadings. The values of AVE and construct reliability (CR) were calculated to test convergent validity and discriminant validity. A confirmatory factor analysis was then performed with AMOS V.19.0 to assess the fitness of our modified theory model. The fit indices of χ2/df (<2), Goodness of Fit Index (GFI; >0.90), Adjusted Goodness of Fit Index (AGFI; >0.90), Non-Normed Fit Index (NNFI; >0.90), Comparative Fit Index (CFI; >0.90), Root Mean Square Error of Approximation (RMSEA; <0.05) and Standardised Root Mean Square Residual (SRMR; <0.08) were used to assess the fitness of the model. Missing item values from individual surveys were imputed with the mean value.20

Results

Responding groups

In total, 379 questionnaires were collected in this study. The characteristics of the participating hospitals are listed in table 1. The response rate was 93.81%. The percentage of missing values for each item of the subscale varied from 0.00% to 3.63%.

Table 1

Characteristics of the participating hospitals

Variables	N	Per cent
Classification
Specialised hospital	110	29.0
General hospital	269	71.0
Grade
Secondary	293	77.3
Tertiary	86	22.7
Location
PRD	164	43.3
Other areas in Guangdong Province	215	56.7

PRD, Pearl River Delta.

Exploratory factor analysis

To test the validity of the hypothesis, the exploratory factor analysis with Varimax factor rotation was conducted, and six components were extracted, which accounted for 73.05% of the total variance. Results are presented in table 2.

Table 2

Exploratory factor analysis with Varimax factor rotation

	Subscale 1	Subscale 2	Subscale 3	Subscale 4	Subscale 5	Subscale 6
Q1	0.82
Q2	0.92
Q3	0.87
Q4	0.89
Q5		0.83
Q6		0.73
Q7		0.59
Q8			0.84
Q9			0.84
Q10			0.85
Q11			0.65
Q12				0.77
Q13				0.79
Q14				0.76
Q15					0.77
Q16					0.76
Q17					0.83
Q18					0.85
Q19						0.86
Q20						0.88
Q21						0.85
Q22						0.76

Only factor loadings of 0.50 or greater are reported.

Total variance explained=73.05%.

Reliability

The mean scores, SDs and Cronbach's α coefficients are presented in table 3. All internal consistency reliabilities exceeded the minimum reliability standard of 0.70, among which the overall scale coefficient was 0.87, and six subscale coefficients were 0.92 (strategic management), 0.81 (budget and expenditure), 0.85 (financing), 0.75 (financing, medical management), 0.86 (human resources) and 0.86 (accountability).

Table 3

Chinese version of the Hospital Autonomy Questionnaire (CVHAQ) reliability for questionnaire and subscales

Scale/item	Mean	SD	Cronbach’s α
Overall scale	67.47	11.68	0.87
Subscale 1: strategic management	11.92	3.53	0.92
Q1: The extent to which your hospital's desire to develop a vision and purpose is constrained by government departments	3.04	1.10
Q2: The extent to which your hospital’s development goals are constrained by government departments	2.94	0.94
Q3: The extent to which your hospital's attempt to develop a short-term development plan is constrained by government departments	3.07	0.92
Q4: The extent to which the hospital's attempt to develop a long-term development plan is constrained by government departments	2.87	0.96
Subscale 2: budget and expenditure	6.80	2.52	0.81
Q5: The extent to which your hospital’s drug procurement is constrained by government departments	1.94	1.08
Q6: The extent to which your hospital’s equipment procurement is constrained by government departments	2.09	0.98
Q7: Except for drugs and equipment, the extent to which your hospital’s other spending is constrained by government departments	2.78	0.90
Subscale 3: financing	11.16	4.20	0.85
Q8: The extent to which your hospital’s short-term liabilities are constrained by government departments	2.92	1.23
Q9: The extent to which your hospital’s long-term liabilities are constrained by government departments	2.59	1.26
Q10: The extent to which your hospital’s other liabilities (finance leases, bills, etc) are constrained by government departments	2.60	1.32
Q11: The degree to which government departments intervene in your hospital's ability to accept donations	3.05	1.23
Subscale 4: day-to-day management	9.83	2.28	0.75
Q12: The degree to which government departments directly intervene in your hospital's medical business management	3.03	0.95
Q13: The degree to which government departments directly intervene in your hospital's logistics service management	3.51	0.89
Q14: The degree to which government departments directly intervene in your hospital's ability to provide convenience services	3.28	0.96
Subscale 5: human resources	11.46	3.54	0.86
Q15: The extent to which your hospital's ability to recruit employees is constrained by government departments	2.66	1.05
Q16: The extent to which your hospital's policies for staff rewards and punishment are constrained by government departments	3.26	0.99
Q17: The extent to which your hospital employee salary structure is constrained by government departments	2.69	1.07
Q18: The extent to which your hospital's ability to dismiss employees is constrained by government departments	2.85	1.12
Subscale 6: accountability	16.30	2.95	0.86
Q19: Whether the purposes of the inspection set by government departments for your hospital are clear	4.22	0.89
Q20: Whether the assessment indicators set by government departments for your hospital are clear	4.06	0.99
Q21: Whether the assessment indicators set by government departments for your hospital are reasonable	3.68	0.89
Q22: The degree to which recognition of government departments’ inspection affects your hospital	4.34	0.73

Item–subscale correlations

Pearson correlation coefficients between subscales and items are presented in table 4. The results showed that items had higher correlation coefficients as compared with the theoretical subscale than with the other subscales.

Table 4

Item–subscale correlations of the Chinese version of the Hospital Autonomy Questionnaire (CVHAQ)

	Subscale 1	Subscale 2	Subscale 3	Subscale 4	Subscale 5	Subscale 6
Q1	0.86	0.33	0.22	0.32	0.22	−0.04
Q2	0.94	0.33	0.23	0.32	0.27	0.07
Q3	0.90	0.33	0.25	0.37	0.23	0.07
Q4	0.91	0.36	0.21	0.32	0.25	0.07
Q5	0.33	0.86	0.40	0.29	0.31	−0.04
Q6	0.35	0.89	0.55	0.30	0.47	−0.05
Q7	0.28	0.81	0.58	0.34	0.37	0.01
Q8	0.21	0.52	0.87	0.29	0.35	−0.04
Q9	0.18	0.54	0.88	0.29	0.31	−0.00
Q10	0.19	0.48	0.87	0.24	0.30	−0.06
Q11	0.26	0.43	0.72	0.28	0.27	−0.00
Q12	0.34	0.36	0.37	0.85	0.32	−0.03
Q13	0.21	0.29	0.28	0.80	0.25	0.06
Q14	0.34	0.23	0.15	0.80	0.22	0.02
Q15	0.24	0.38	0.28	0.26	0.81	0.04
Q16	0.31	0.40	0.33	0.34	0.82	−0.05
Q17	0.19	0.41	0.34	0.27	0.86	−0.01
Q18	0.18	0.33	0.28	0.21	0.86	0.01
Q19	0.01	−0.05	−0.03	0.02	0.01	0.86
Q20	0.01	−0.04	−0.06	0.01	−0.05	0.89
Q21	0.07	0.01	0.01	0.02	0.04	0.85
Q22	0.06	−0.04	−0.01	0.01	−0.01	0.75

Values in bold type means there were higher correlation coefficients between items and their hypothesised subscales than those with other subscales.

Convergent validity and discriminant validity

If the value of AVE was higher than 0.5 and the value of CR was higher than 0.70, we considered the convergent validity acceptable. To test discriminant validity, we calculated the square root of every AVE value belonging to each latent subscale. The outcomes (table 5) show that the square roots of the AVE of each subscale were larger than the correlation of the specific subscale with any of the other subscales.

Table 5

Convergent and discriminant validity for questionnaire and subscales

Variable	AVE	CR	Strategic management	Budget and expenditure	Financing	Day-to-day management	Human resources	Accountability
Strategic management	0.76	0.93	0.87
Budget and expenditure	0.58	0.81	0.38	0.76
Financing	0.61	0.86	0.25	0.59	0.78
Day-to-day management	0.51	0.75	0.37	0.36	0.33	0.71
Human resources	0.60	0.85	0.27	0.45	0.37	0.32	0.77
Accountability	0.61	0.86	0.04	−0.04	−0.03	0.02	0.00	0.78

On the diagonal, we inserted the square roots of every AVE value to compare it with the other correlation coefficients.

Values in bold type means there was higher square root of AVE value in each subscales than the correlation coefficients with other subscales.

AVE, average variance extracted; CR, construct reliability.

Confirmatory factor analysis

We used AMOS 19.0 to conduct confirmatory factor analysis, and seven indicators were calculated to test the fitness of our theory model. RMSEA and SRMR were <0.08, while GFI, AGFI, NNFI and CFI were higher than 0.9, leading to acceptable construct validity. The results are presented in table 6.

Table 6

Fitness of fit of CVHAQ

χ2/df	GFI	AGFI	NNFI	CFI	RMSEA (95% CI)	SRMR
1.73	0.93	0.91	0.96	0.97	0.04 (0.03 to 0.05)	0.07

χ2/df<2.0, GFI>0.90, AGFI>0.90, NNFI>0.90, CFI>0.90, RMSEA<0.08, SRMR<0.08.

AGFI, Adjusted Goodness of Fit Index; CFI, Comparative Fit Index; CVHAQ, Chinese version of the Hospital Autonomy Questionnaire; df, degree of freedom; GFI, Goodness of Fit Index; NNFI, Non-Normed Fit Index; RMSEA, Root Mean Square Error of Approximation; SRMR, Standardised Root Mean Square Residual; χ2, Minimum Fit Function χ2.

Discussion

In this study, the hospital autonomy survey was modified to adapt to Chinese health policies, and the CVHAQ was developed on the basis of Harding and Preker's conceptual framework on hospital autonomy, as well as the tools developed by Chawla et al15 and Over and Watanabe.16 The reliability and factor structure of the CVHAQ were assessed. The results of this study showed that the CVHAQ was a reliable and valid questionnaire for the measurement of hospital autonomy. The confirmatory factor analysis confirmed the theoretical model, which reflected the extent of hospital autonomy. A higher score suggested the hospital had more autonomy. According to the Chinese Health Statistics Report (2015),3 government fiscal budgets provide about 7% of total revenue for public hospitals. Almost all secondary and tertiary hospitals must earn a profit from the medical services they provide, so there is little variance in residual claimant and market exposure between sample hospitals that could be captured by the questionnaire. Social function is a complementary reform to ensure that services that were previously cross-subsidised continue to be delivered, and does not belong to the scope of hospital autonomy. The six theoretical subscales included in the final CVHAQ well reflected the status of hospital autonomisation. The results of confirmatory factor analysis showed that all indicators of fitness were acceptable.21–23 This study showed that subscales 2 and 3 were correlated (r=0.59), which can be explained by the fact that the autonomy of the two subscales commonly depends on the economic policy for a hospital, which is set by the government; thus, finances, budget and expenditure are synergistically affected. Subscales 5 and 2 were also correlated (r=0.45). The underlying reason could be that the Chinese government spent little on human resources, and the cost was borne by the hospital.24 25

Characteristics of local hospitals in Guangdong made it very suitable for conducting this large-scale quantitative study. China has traditionally managed its public hospitals as administrative units. However, in the past 30 years, the Chinese government has been reforming its health system by reducing the health department's direct control over public hospitals and shifting the day-to-day decision-making from government authority to hospital managers, to reduce the government's financial burden and improve hospital efficiency.26 Guangdong, the frontier of China's economic reform, has >400 secondary and tertiary public hospitals. They are divided into national hospitals, provincial hospitals and municipal or county hospitals. Owing to different levels of government input and degrees of decentralisation, these hospitals vary greatly in autonomy, which created a unique opportunity for us to conduct this study in Guangdong. In addition, this study was supported by the Health Bureau of Guangdong Province, which resulted in a higher response rate from local hospitals. This helped overcome the issues of small sample size and data constraint existing in previous studies.

Although many studies on hospital autonomisation have been conducted, only a few were quantitative. Most previous studies were based on case study, and the results were subject to potential confounding factors, such as hospital size, management style, geographic location, etc.27 Empirical literature to date has done little to explore a Hospital Autonomy Questionnaire to quantify the extent of hospital autonomy. The CVHAQ developed in this study helped fill this gap in the published literature and provided a validated tool that can help overcome these shortcomings and enable reliable conclusions to be drawn.

In addition, the CVHAQ could lay a strong foundation for quantitative analysis of the relationship between hospital autonomy and its effect on healthcare reform. For example, a regression analysis can be done to explore the relationship between hospital autonomy and efficiency by using the hospital autonomy scores as independent factors.

Some limitations must be mentioned regarding our findings. The survey was conducted only in Guangdong Province; thus, generalisation of the results to other provinces should be done with caution. In our future study, we will investigate hospital autonomy in other provinces in China to further validate our questionnaire. In addition, we conducted a cross-sectional study that could not illustrate the underlying trend of hospital autonomy. In future studies, we plan to collect panel data to monitor hospital autonomy and explore the relationship between hospital autonomy and efficiency, to provide an evidence-based foundation for China's healthcare reform.

Conclusions

This study demonstrated the reliability and validity of a CVHAQ, and provided a quantitative method for the assessment of hospital autonomy in China.