Reliability and validity of an indicator system used to evaluate outpatient and inpatient satisfaction in Chinese hospitals.

BACKGROUND AND AIMS: Patient satisfaction is one of the important ways to measure the results of treatment and the quality of medical services. The purpose of the present study was to verify the reliability and validity of the indicator system for evaluating satisfaction of outpatients and inpatients in Chinese hospitals.
METHODS: The study was based on the satisfaction questionnaire program of the national doctor- patient experience research center with data from 99,802 outpatients and 229,215 inpatients, collected in China between 2016 and 2017. We adopted exploratory factor analysis for validity analysis and the method of split-half reliability and Cronbach's α coefficient for reliability analysis.
RESULTS: In the validity analysis of the indicator system for outpatients, the factor loading was between 0.438 and 0.919, and the reliability was excellent. In the validity analysis of the indicator system for nonsurgical inpatients, the factor loading of the secondary indicators was between 0.417 and 0.75, and the reliability was excellent. In the validity analysis of the indicator system for surgical inpatients satisfaction, the factor loading of the secondary indicators range was 0.391-0.751, and the reliability was excellent.
CONCLUSION: The indicator systems for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients all have excellent reliability and good validity. They can be widely used for an outpatient and inpatient satisfaction questionnaire in Chinese hospitals.

Introduction

The patient satisfaction is the patients’ subjective evaluation of the accepted medical service under the influence of various factors after comparing expectations with feelings.1 It is an important evaluation tool for the quality of hospital services. Research on customer satisfaction evaluation systems is of great significance both in theory and practice to improve further the management theory of medical service quality, to guide medical institutions to improve service quality and eventually to win the trust of the patients.2

In other countries outside China, research on the theory and methods of patient satisfaction evaluation started early.3,4 Among them, one of the most classic evaluation scales was the patient satisfaction questionnaire (PSQ) developed by Ware et al, which was the most widely used questionnaire in the 1990s.5 In addition to the evaluation of medical service quality, it can also provide a reference for the overall layout of hospitals, capital utilization and personnel deployment. The Hospital Consumer Assessment of Healthcare Providers and Systems questionnaire developed by the Centers for Medicare and Medicaid Services and Agency for Healthcare Research and Quality is the second most influential.6 Compared with other questionnaires, it includes employee factors and can be used for comparison and analysis of patient satisfaction in different hospitals. It was officially used in the US in 2008.

As patient satisfaction was widely introduced into hospitals in developed countries and achieved significant results, Chinese hospitals have begun to recognize the essence of “patient orientated health care” in recent years. Xia et al (2010)11 followed the basic steps of the measuring tools proposed by Streiner et al7 and designed six questionnaires, such as “the questionnaire on the satisfaction of discharged patients” and “the questionnaire on the satisfaction of outpatients”.11 Chen (2011)8 divided the patient satisfaction evaluation system into three levels, including 6 primary indicators, 13 secondary indicators and 32 three-level indicators, and built a simple outpatient satisfaction indicator system using Analytical Hierarchy Process. Based on the past inpatient satisfaction model, Tan L10 constructed an inpatient satisfaction indicator system of a grade III level A hospital with 1 primary index, 5 secondary indicators and 29 three-level indicators in Jiangkou city.10 Kong and Chen (2007)9 selected 30 experts in different institutions in Beijing, Ji’nan and Shenzhen. Through the investigation of community health services, a reliability test of expert consultations was conducted.

However, due to the late start of patient satisfaction research in China, both theory and practice necessarily are at the exploratory stage, and many problems need to be solved urgently. On the one hand, most of the domestic questionnaires are distributed “on the spot” and are carried out by hospitals themselves. Most hospitals do not entrust third party evaluation agencies to investigate them. It is difficult to construct a reliable mathematical model based on the deviation of measurement data caused by the above reasons.12 On the other hand, there is no scientific and effective PSQ that can be widely used in China to date, and the available PSQs lack scientific credibility and reliability. Furthermore, existing questionnaire research does not address applicability or take practical considerations into account.13 Most of the existing PSQ are not universal and representative, which makes the evaluation results difficult to compare objectively. In addition, the majority of the domestic PSQs used involved medical quality, attitude of medical staff and the medical environment, without mentioning important content such as the patient’s expectations, value perception and loyalty to the medical institution. The evaluation results are therefore difficult to use to provide an effective scientific basis for choosing a medical institution. Therefore, a standardized and reliable indicator system for a PSQ remains to be developed.

Therefore, in the present study, we developed a patient satisfaction indicator system that can be widely used in outpatient and inpatient satisfaction questionnaires (IPSQs) in Chinese hospitals following strict scientific criteria. The study aims to confirm the reliability and validity of the indicator system for evaluating the satisfaction of outpatients and inpatients.

Methods

Compilation of the indicator system

This study is a research project of the national doctor–patient experience research center. In the early stages, the study passed analysis of the medical practice process and many rounds of domestic and foreign expert demonstrations. The indicator systems for evaluating satisfaction of outpatients, surgical inpatients (including all types of surgery) and nonsurgical inpatients were initially constructed. The indicator system of outpatient satisfaction is a two-level indicator system, with 5 primary indicators and 31 secondary indicators (Table S1). The indicator system of inpatient satisfaction is a two-level indicator system, with 6 primary indicators and 43 secondary indicators (Table S2). Among them, the satisfaction indicator system of nonsurgical inpatients had 5 items under the primary index, including hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service, and 38 items under the secondary index. The satisfaction indicator system of surgical inpatients had 6 items under the primary index, including hospitalization service, treatment, auxiliary examination, service attitude, environment and logistic services, surgical anesthesia, and 45 items under the secondary index. We used a balanced Likert 5-point scale to record responses and assigned a score of 1, 2, 3, 4 and 5 for strongly disagree, disagree, neutral, agree and strongly agree, respectively. The higher the score, the higher the satisfaction of the item. Each item score adds up to the total score of the indicator system. With the original indicator system, the full score of the satisfaction questionnaire of outpatients is 155 points, for nonsurgical inpatients, 190 points and for surgical inpatients, 225 points.

Subjects

All the hospitals investigated in our study voluntarily participated in the PSQ of the national doctor and patient experience research center. The 8%–30% of outpatients in a single day were randomly selected from the list of patients who received 3 consecutive days of medical treatment provided by the hospital participating in the project. Between 30% and 60% of hospitalized patients, who fulfilled inclusion criteria in a single day, were selected from the total number of inpatients in the participating hospitals, and the inpatient components of each department and ward. Hospitalized patients who had been admitted in the hospital for >24 hours and therefore had a certain experience of hospitalization were included in this study.

The interviewees were patients or their family members. All patients who were aged ≥18 years old and conscious completed the questions by themselves or with the help of investigators on the mobile computer provided. Only when patients were <18 years old and unable to complete the questions by themselves or with the help of an investigator (medical conditions included severe mental disorder, coma, dementia, etc.), family members answered the questions on behalf of the patient.

On-site investigation process

The indicator system used in our study aimed to understand patient satisfaction on various aspects of the hospital service, thus permitting the evaluation of the overall service quality of the hospital. A third-party company was hired and the investigators conducted paperless questionnaires on subjects using a unified mobile terminal system. During the investigation, the investigators explained the purpose of the study through detailed written and verbal understandable instructions and asked the patients to truly reflect their own thoughts and opinions based on their own experience and feelings. The researchers guaranteed that the information provided by each patient was strictly confidential and that they would never provide any information obtained to others without the consent of the patient. Also, the hospital and department was kept strictly confidential. They did not provide any person with any information on a specific hospital or department, which unintentionally may affect a patient’s medical treatment and perception of the hospital. The investigators used a unified guide to describe the way the mobile terminal questionnaire was filled in, the research content and the purpose of the study, and also informed the patient that there was no right or wrong answer, just follow faithfully their innermost thoughts. Each patient completed the questionnaire independently; patients who had difficulty doing this task, due to physical illness, were interviewed by a researcher or relatives, who completed the questionnaire for them. The investigator waited patiently and requested the patient to answer the questions in an objective, neutral manner. The completed questionnaire was uploaded from the computer as soon as it was completed. The average time required to complete the questionnaire was circa 10–15 minutes.

In total, 99,802 outpatient and 229,215 up-to-standard inpatient data were collected. Among the inpatient questionnaire data, 58,220 came from surgical patients and 170,995 from the nonsurgical patients.

Statistical analyses

This study confirmed the final structure of the indicator system and tested the structural validity of the indicator system by exploratory factor analysis. The appropriateness of factor analysis was tested using the Kaiser–Meyer–Olkin adequacy (KMO) index and Bartlett’s test of sphericity. If the KMO of the indicator system was greater than 0.5 and the Bartlett’s test of sphericity significant (P<0.05), then the indicator system was deemed suitable for factor analysis. After determining the number of factors in each indicator system according to its structure, the cumulative variance contribution rate was obtained by factor analysis. Principal component analysis was used to extract the factor, and then a rotation transformation was carried out to find the best analysis effect. If the factor of the indicator system could explain >50% variation and each item had enough loading on the corresponding factors (≥0.4), the indicator system was considered to have good structural validity. After omitting loadings <0.3 and forming a new indicator system based on expert opinions and the actual situation,14 we used the Spearman–Brown reliability and Cronbach’s α coefficient to test the split-half reliability and internal consistency reliability of the indicator system, respectively.

In general, a split-half reliability coefficient of >0.75 was excellent and <0.4 poor. α>0.8 indicated excellent internal consistency, 0.6<α<0.8 an acceptable level and α<0.6 was indicative of poor internal consistency.15 All statistical analyses were carried out using SAS 9.4 software.

Results

Characteristics of the study subject

This study collected data from 98,192 outpatients in 197 hospitals from 22 provinces and 229,215 hospitalized patients distributed in 413 hospitals in 28 provinces in China (Table S3).

In this study, 19,953 (20.3%) outpatient data were collected in 2016 and 78,243 (79.7%) outpatient data in 2017. Of the outpatients, 40,072 (40.8%) were male, with an age range mainly between 18 and 59 years old (81,960: 83.5%). In addition, outpatients who completed the questionnaire were mainly from grade III hospitals (82.3%), and many of them had local medical insurance (53.8%) (Table 1).

Table 1

Characteristics of outpatients and inpatients

Demographic variables	Outpatients (n=98,196)		Nonsurgical inpatients (n=170,995)		Surgical inpatients (n=58,220)
	Number	Constituent ratio (%)	Number	Constituent ratio (%)	Number	Constituent ratio (%)
Gender
Male	40,072	40.8	84,161	49.2	29,627	50.9
Female	58,124	59.2	86,834	50.8	28,593	49.1
Age (years)
<18	6,347	6.5	11,401	6.7	2,614	4.5
18–59	81,960	83.5	100,042	58.5	40,676	69.9
>60	9,889	10.1	59,552	34.8	14,930	25.6
Source of cost
Local medical insurance	52,877	53.8
Remote medical insurance	6,453	6.6
Self-financed	38,866	39.6
Year
2016	19,953	20.3	93,958	54.9	25,251	43.4
2017	78,243	79.7	77,037	45.1	32,969	56.6
Hospital level
<Grade III	17,361	17.7	62,162	36.4	10,988	18.9
Grade III	80,835	82.3	108,833	63.6	47,232	81.1

Among the outpatient questionnaires collected, 170,995 came from nonsurgical patients and 58,220 from surgical patients. There were 84,161 males (49.2%) in the nonsurgical inpatient group investigated. Patients between 18 and 59 years of age accounted for 58.5% of the total inpatients under investigation, and only 6.7% of the inpatients were under the age of 18. In addition, the nonsurgical inpatients were mainly from grade III hospitals (63.6%). There are 29,627 men (50.9%) in the surgical inpatients. Patients aged between 18 and 59 years of age accounted for 69.9% of the total surgical patients. The surgical inpatients were mainly from grade III hospitals (81.1%) (Table 1).

Results of structural validity

Validity analysis of the indicator system for evaluating outpatient satisfaction

Exploratory factor analysis was used to investigate the structural validity of the indicator system of the outpatient satisfaction questionnaire. The KMO of the indicator system was 0.938 and the Bartlett’s test value of sphericity 2,553,071.12 (P<0.001), which indicated that conducting factor analysis on the data was appropriate.

The secondary index of the indicator system of outpatient satisfaction questionnaire was also analyzed. The total variance contribution rate of the five primary indicators of pre-diagnosis, treatment, auxiliary examination, drug withdrawal, the environment and logistic services was 68.0% (Table 2). The Scree plot showed that the first five factors accounted for most of the variability. From the fifth factor, the characteristic value was small and exhibited a flat downward trend, with each factor accounting for only a small component of the variability (Figure 1).

Table 2

Eigenvalue and contribution rates of the five factors in the indicator system of the outpatient satisfaction questionnaire

Factors	Eigenvalues	Contribution rate (%)	Cumulative contribution rate (%)
1	12,873.47	0.416	0.416
2	3,532.823	0.114	0.530
3	1,845.447	0.060	0.589
4	1,567.751	0.051	0.640
5	1,231.178	0.040	0.680

Figure 1

Scree plot of the outpatient satisfaction indicator system.

In order to facilitate analysis, each index was encoded in the order of b1–b31. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 3). The indicators under the “pre-diagnosis” dimension were b1–b6. They were dominated by the fourth factor, and the factor loading was between 0.531 and 0.707. The indicators under the “treatment” dimension were b7–b12. They were dominated by the second factor, and the factor loading was between 0.488 and 0.803. The indicators under the “auxiliary examination” dimension were b13–b20. They were dominated by the first factor, and the factor loading was between 0.636 and 0.919. The indicators under the “drug withdrawal” dimension were b21–b23. They were dominated by the fifth factor, and the factor loading was between 0.684 and 0.788. The indicators under the “environment and logistics service” dimension were b24–b31. They were dominated by the third factor, and the factor loading was between 0.438 and 0.749.

Table 3

Principal component analysis of the indicator system for evaluation of outpatient satisfaction

Index	Principal components
	1	2	3	4	5
b1	0.138	0.203	0.166	0.576	0.081
b2	0.132	0.213	0.163	0.597	0.081
b3	0.118	0.163	0.169	0.677	0.065
b4	0.105	0.198	0.162	0.707	0.080
b5	0.108	0.242	0.229	0.648	0.059
b6	0.136	0.449	0.203	0.531	0.077
b7	0.170	0.488	0.221	0.383	0.098
b8	0.180	0.670	0.204	0.345	0.109
b9	0.150	0.775	0.158	0.307	0.118
b10	0.176	0.803	0.178	0.275	0.123
b11	0.225	0.722	0.227	0.211	0.206
b12	0.243	0.667	0.256	0.231	0.200
b13	0.635	0.223	0.164	0.153	0.082
b14	0.718	0.165	0.159	0.128	0.094
b15	0.802	0.150	0.143	0.117	0.099
b16	0.819	0.135	0.144	0.107	0.116
b17	0.919	0.088	0.138	0.106	0.117
b18	0.910	0.080	0.149	0.099	0.134
b19	0.698	0.149	0.168	0.133	0.226
b20	0.651	0.168	0.186	0.138	0.270
b21	0.359	0.242	0.235	0.154	0.684
b22	0.350	0.215	0.283	0.138	0.773
b23	0.312	0.230	0.284	0.143	0.788
b24	0.129	0.184	0.471	0.328	0.219
b25	0.137	0.170	0.507	0.323	0.150
b26	0.100	0.157	0.584	0.305	0.113
b27	0.177	0.183	0.749	0.176	0.114
b28	0.190	0.200	0.740	0.165	0.114
b29	0.331	0.137	0.540	0.126	0.123
b30	0.421	0.115	0.438	0.103	0.139
b31	0.249	0.306	0.458	0.208	0.218

The factor of the indicator system can explain >50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), so the indicator system had good structural validity.

Validity analysis of the indicator system for evaluating nonsurgical inpatient satisfaction

Exploratory factor analysis was used to investigate the structural validity of the secondary indicators of the nonsurgical inpatients indicator system. The KMO of the indicator system was 0.984 and the Bartlett’s test value of sphericity 5,441,976.14 (P<0.001), which indicates that conducting factor analysis on the data was appropriate. The total variance contribution rate of the five indicators of hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service was 68.5% (Table 4). The Scree plot showed the first five factors accounted for most of the variability. From the fifth factor, the characteristic value was small and exhibited a flat downward trend, and each factor accounted for only a small component of the variability (Figure 2).

Table 4

Eigenvalue and contribution rates of the indicator system for evaluation of nonsurgical inpatient satisfaction

Factors	Eigenvalue	Contribution rate (%)	Cumulative contribution rate (%)
1	20,464.54	0.539	0.539
2	1,941.828	0.051	0.590
3	1,471.356	0.039	0.629
4	1,176.957	0.031	0.660
5	953.6677	0.025	0.685

Figure 2

Scree plot of the nonsurgical inpatient satisfaction indicator system.

In order to facilitate analysis, each index was encoded in the order of b1–b38. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 5). The index of the maximum absolute value of factor loading in each factor was also grouped. The indicators under the “hospitalization service” dimension were b1–b7. They were dominated by the second factor, and the factor loading was between 0.420 and 0.730. The indicators under the “treatment” dimension were b8–b20. They were dominated by the first factor, and the factor loading was between 0.503 and 0.597. The indicators under the “auxiliary examination” dimension were b21–b26. They were dominated by the third factor, and the factor loading was between 0.649 and 0.775. The indicators under the “service attitude” dimension were b27–b29. They were dominated by the fifth factor, and the factor loading was between 0.417 and 0.579. The indicators under the “environment and logistic services” dimension were b30–b38. They were dominated by the fourth factor, and the factor loading was between 0.485 and 0.713.

Table 5

Principal component analysis of the indicator system for evaluation of nonsurgical inpatient satisfaction

Index	Principal components
3	1	2	3	4	5
b1	0.200	0.545	0.191	0.183	0.139
b2	0.273	0.653	0.182	0.218	0.167
b3	0.252	0.736	0.206	0.230	0.113
b4	0.261	0.732	0.215	0.232	0.108
b5	0.303	0.684	0.208	0.219	0.125
b6	0.229	0.510	0.168	0.184	0.093
b7	0.243	0.420	0.176	0.220	0.039
b8	0.503	0.424	0.188	0.219	0.145
b9	0.544	0.472	0.237	0.262	0.052
b10	0.572	0.445	0.265	0.278	0.027
b11	0.546	0.317	0.320	0.265	−0.021
b12	0.588	0.349	0.311	0.287	0.039
b13	0.597	0.384	0.291	0.289	0.109
b14	0.562	0.374	0.275	0.277	0.163
b15	0.545	0.332	0.310	0.287	0.138
b16	0.547	0.334	0.276	0.292	0.201
b17	0.560	0.303	0.237	0.280	0.313
b18	0.597	0.285	0.261	0.307	0.311
b19	0.549	0.264	0.289	0.336	0.248
b20	0.514	0.284	0.299	0.310	0.345
b21	0.367	0.221	0.649	0.265	0.093
b22	0.314	0.219	0.726	0.264	0.100
b23	0.282	0.238	0.735	0.262	0.141
b24	0.223	0.231	0.775	0.273	0.109
b25	0.215	0.238	0.757	0.300	0.130
b26	0.208	0.253	0.735	0.304	0.180
b27	0.282	0.379	0.428	0.308	0.434
b28	0.323	0.335	0.330	0.309	0.579
b29	0.333	0.339	0.354	0.383	0.417
b30	0.306	0.257	0.332	0.485	0.147
b31	0.266	0.254	0.308	0.487	0.089
b32	0.309	0.264	0.281	0.588	0.216
b33	0.209	0.217	0.197	0.709	0.148
b34	0.214	0.242	0.211	0.713	0.159
b35	0.197	0.200	0.277	0.666	−0.015
b36	0.254	0.222	0.284	0.673	0.057
b37	0.349	0.278	0.271	0.556	0.214
b38	0.362	0.320	0.266	0.513	0.250

The five factors of the indicator system can explain >50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), which indicated that the indicator system had good structural validity.

Validity analysis of the indicator system for evaluating surgical inpatient satisfaction

Exploratory factor analysis was used to analyze the structural validity of the original indicator system of surgical inpatients. The KMO of the indicator system was 0.987 and the Bartlett’s test value of sphericity 2,235,436.36 (P<0.001), indicating sample adequacy.

Exploratory factor analysis was further used to examine the structural validity of the secondary indicators of the surgical inpatients indicator system. The total variance contribution rate of the six primary indicators of hospitalization service, treatment, auxiliary examination, service attitude, environment and logistics service, surgical anesthesia was 71.0% (Table 6). The Scree plot revealed that the first six factors accounted for most of the variability. From the sixth factor, the characteristic value was small and exhibited a flat downward trend, and each factor accounted for only a small part of the variability (Figure 3).

Table 6

Eigenvalue and contribution rates of the indicator system for the evaluation of surgical inpatient satisfaction

Factors	Eigenvalue	Contribution rate (%)	Cumulative contribution rate (%)
1	24,117.75	0.561	0.561
2	1,856.685	0.043	0.605
3	1,493.016	0.035	0.639
4	1,113.706	0.026	0.665
5	1,025.655	0.024	0.689
6	904.2898	0.021	0.710

Figure 3

Scree plot of the surgical inpatient satisfaction indicator system.

In order to facilitate analysis, each index was encoded in the order of b1–b43, and six principal components were taken for principal component analysis. In the rotated factor matrix by varimax rotation, the index of the maximum absolute value of factor loading in each factor was grouped (Table 7). The indicators under the “hospitalization service” dimension were b1–b7. They were dominated by the first factor, and the factor loading was between 0.412 and 0.720. The indicators under the “treatment” dimension were b8–b20. They were dominated by the third factor, and the factor loading was between 0.467–0.578. The indicators under the “auxiliary examination” dimension were b21–b26. They were dominated by the second factor, and the factor loading was between 0.653 and 0.745. The indicators under the “service attitude” dimension were b27–b29. They were dominated by the sixth factor, and the factor loading was between 0.391 and 0.559. The indicators under the “environment and logistics service” dimension were b30–b38. b30–b37 were dominated by the fifth factor, and the factor loading was between 0.422 and 0.688. Factor b38 was dominated by the fourth factor, and the factor loading was 0.465. Its factor loading under the fifth factor was 0.392. The indicators under the “surgical anesthesia” dimension were b39–b43, and the factor loading was between 0.520 and 0.752.

Table 7

Principal component analysis of the questionnaire for evaluation of surgical inpatient satisfaction

Index	Principal components
	1	2	3	4	5	6
b1	0.527	0.231	0.177	0.182	0.188	0.115
b2	0.649	0.209	0.241	0.212	0.208	0.152
b3	0.720	0.222	0.207	0.202	0.215	0.107
b4	0.716	0.221	0.253	0.227	0.191	0.125
b5	0.656	0.207	0.298	0.234	0.188	0.122
b6	0.471	0.171	0.262	0.144	0.162	0.066
b7	0.412	0.180	0.260	0.129	0.226	0.020
b8	0.456	0.203	0.467	0.187	0.203	0.116
b9	0.503	0.244	0.512	0.238	0.212	0.029
b10	0.457	0.265	0.520	0.273	0.232	0.009
b11	0.408	0.280	0.548	0.292	0.227	−0.006
b12	0.391	0.295	0.541	0.285	0.248	0.045
b13	0.418	0.269	0.556	0.253	0.247	0.106
b14	0.392	0.258	0.526	0.282	0.208	0.128
b15	0.360	0.287	0.506	0.276	0.234	0.098
b16	0.327	0.270	0.522	0.293	0.263	0.141
b17	0.303	0.265	0.559	0.217	0.247	0.246
b18	0.304	0.275	0.578	0.247	0.290	0.246
b19	0.283	0.306	0.515	0.230	0.327	0.213
b20	0.303	0.303	0.504	0.241	0.288	0.288
b21	0.246	0.653	0.322	0.210	0.244	0.063
b22	0.245	0.713	0.296	0.225	0.236	0.066
b23	0.255	0.703	0.278	0.236	0.223	0.128
b24	0.244	0.737	0.199	0.202	0.249	0.097
b25	0.241	0.745	0.185	0.211	0.274	0.108
b26	0.254	0.708	0.203	0.234	0.269	0.170
b27	0.365	0.398	0.298	0.308	0.234	0.425
b28	0.311	0.319	0.348	0.263	0.263	0.559
b29	0.347	0.337	0.331	0.303	0.328	0.391
b30	0.270	0.323	0.273	0.298	0.422	0.138
b31	0.261	0.317	0.218	0.280	0.461	0.083
b32	0.250	0.315	0.292	0.340	0.503	0.169
b33	0.232	0.231	0.219	0.191	0.688	0.119
b34	0.255	0.222	0.226	0.234	0.689	0.135
b35	0.194	0.289	0.178	0.251	0.618	0.023
b36	0.245	0.295	0.248	0.301	0.599	0.057
b37	0.257	0.270	0.319	0.429	0.441	0.170
b38	0.291	0.260	0.325	0.465	0.392	0.221
b39	0.305	0.252	0.282	0.520	0.319	0.169
b40	0.287	0.245	0.285	0.643	0.269	0.143
b41	0.277	0.242	0.245	0.679	0.277	0.090
b42	0.243	0.255	0.233	0.752	0.263	0.077
b43	0.239	0.258	0.238	0.720	0.266	0.080

The factors of the indicator system can explain more than 50% of the variation, and each item had enough loading on the corresponding factors (≥0.4), which indicated that the indicator system had good structural validity.

Results of reliability analysis

We adopted the statistical methods of Spearman–Brown split-half reliability and Cronbach’s α coefficient for reliability analysis. The outpatient and inpatient satisfaction indicator system was confirmed by structural validity analysis, and their respective primary indicators were analyzed by the statistical method of split-half reliability and consistency of internal reliability. The results showed that the overall Cronbach’s α coefficient of the outpatient satisfaction indicator system was 0.956, and Cronbach’s α coefficient of each primary index was between 0.860 and 0.946, and the internal consistency reliability was excellent. The overall split-half reliability was 0.956, and split-half reliability of each primary index between 0.906 and 0.976. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient (Table 8).

Table 8

Reliability evaluation of the indicator system for the outpatient and inpatient satisfaction

Type of patient	Primary index	Cronbach’s alpha coefficient	Split-half reliability
Outpatient
	Pre-diagnosis	0.859	0.905
	Treatment	0.920	0.916
	Auxiliary examination	0.946	0.976
	Drug withdrawal	0.931	0.916
	Environment and logistical services	0.875	0.923
	Indicator system	0.952	0.982
Inpatient
Nonsurgical inpatient	Hospitalization service	0.888	0.893
	Treatment	0.952	0.938
	Auxiliary examination	0.945	0.945
	Service attitude	0.882	0.885
	Environment and logistics services	0.927	0.921
	Indicator system	0.977	0.971
Surgical inpatient	Hospitalization service	0.893	0.908
	Treatment	0.958	0.951
	Auxiliary examination	0.947	0.952
	Service attitude	0.891	0.896
	Environment and logistics services	0.932	0.927
	Surgical anesthesia	0.931	0.938
	Indicator system	0.980	0.978

The overall Cronbach’s α coefficient of the satisfaction indicator system of inpatients who did not undergo surgery was 0.975, Cronbach’s α coefficient for each primary index was between 0.887 and 0.947 and the internal consistency reliability was excellent. The overall split-half reliability was 0.967, and split-half reliability of each primary index ranged between 0.882 and 0.944. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient.

The overall Cronbach’s α coefficient of the satisfaction indicator system for surgical inpatients was 0.976, and Cronbach’s α coefficient for each primary index was between 0.882 and 0.951, and the internal consistency reliability was excellent. The overall split-half reliability was 0.976, and split-half reliability of each primary index ranged between 0.885 and 0.946. The split-half reliability was excellent, which was consistent with the evaluation results of Cronbach’s α coefficient.

Discussion

Our research is based on the satisfaction questionnaire project of the national doctor–patient experience research center with questionnaire data obtained from outpatients and inpatients between 2016 and 2017. After determining the indicator system by exploratory factor analysis, the outpatient and inpatient satisfaction indicator systems were analyzed by the statistical method of split-half reliability and consistency of internal reliability. The results showed that all of the satisfaction indicator systems of outpatients, nonsurgical inpatients and surgical inpatients have good structural validity and excellent reliability.

In previous studies, Chen and Zheming had initially developed an IPSQ after reviewing the literature and consulting hospital management experts.16 The first eight factors were admission process, cost, doctor’s service, food supply, assistant department service, nursing, treatment results, medical environment and facilities. However, as the internal consistency was low (α coefficient of admission process and cost was 0.22 and 0.36, respectively), the researchers considered revising the structure of the questionnaire. Wang et al17 extracted five factors through exploratory factor analysis, namely, nursing service, discharge guidance, related services, doctor diagnosis and treatment and logistic support. The overall Cronbach’s α of the questionnaire was 0.868, and the Cronbach’s α of the internal consistency of each primary index was between 0.354 and 0.834.17 There are some deficiencies in the reliability of the indicator systems described above. In recent years, Yangzi has constructed a Chinese patient satisfaction questionnaire containing seven dimensions (doctor–patient relationship, doctor–patient communication, medical service, auxiliary service, environmental sanitation, medical ethics, procedure signs).18 The Cronbach’s α of internal consistency of the questionnaire was 0.975, the Cronbach’s α of seven dimensions was between 0.815 and 0.938, the overall split-half reliability 0.93, and the split-half reliability of each dimension lay between 0.816 and 0.913. Although the structure of the indicator system constructed in the above study was similar to that constructed in the present research, the reliability of the indicator system was better than that of the above study (vide supra).

The exploratory factor analysis of the surgical patients’ indicator system established in our study showed that the factor loading of the secondary index “overall service flow” was 0.395 (<0.4). There was a misclassification in the principal component analysis of the secondary index “medical ethics”. It is not classified under the primary index “environment and logistic services” which it subordinates to. Instead, it is classified under the primary index “surgical anesthesia”. First, our study did not adjust or modify the indicator system directly according to the results of exploratory factor analysis. The main reason is that the indicator system built in our study was based on the Delphi method. Second, the reliability of the indicator system constructed was excellent, indicating that the current indicator system is more than satisfactory. Third, the factor loading of the secondary index “overall service flow” in the surgical patient satisfaction indicator system was <0.4 (>0.3), but it was >0.4 in the nonsurgical patient satisfaction indicator system. Furthermore, it will facilitate the further comparative analysis of the index score of each factor in the surgical and nonsurgical IPSQ. As a result, our study retained the index in the indicator system for evaluating satisfaction of surgical inpatients. On the other hand, the second-level index “medical ethics” has been misclassified in the analysis of disaster factors, but the factor loading in its primary index “environment and logistic services” was 0.392–0.3. This will facilitate the further comparative analysis of surgical and nonsurgical inpatient satisfaction and consider that medical ethics is an important factor influencing patient satisfaction in China.19–21 Finally, based on comprehensive expert opinions and a comprehensive survey of the relevant literature, this study retained the secondary index “medical ethics” in the surgical inpatients satisfaction indicator system and subordinated it in the primary index “environment and logistic services”.

However, there are some limitations in this study. First, we only developed an indicator system for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients in general. Further evaluation for specific diseases still remains to be constructed. We would like to develop an indicator system including different dimensions of departments or diseases in the future research. Besides, in order to improve the health care services of the hospitals, further investigation about impact factors of patients’ satisfaction among different institutions, regions, and characteristic of people need to be further explored.

Conclusion

The indicator systems constructed through exploratory factor analysis for evaluating satisfaction of outpatients, surgical inpatients and nonsurgical inpatients all had excellent reliability. They can be widely used in an outpatient and IPSQ in Chinese hospitals.

Ethics approval and consent to participate

This study was conducted in accordance with the Declaration of Helsinki. Written informed consent has been provided by the patients, or a parent or legal guardian if under the age of 18 years. Data in our study were obtained from electronic hospitalization summary reports (HSRs) of the National Healthcare Data Center of China. The data were masked by the National Healthcare Data Center (data source) and there is no patient name, identification number, contact information and other privacy information in our database. Therefore, any patient privacy was well protected. In addition, the data we used in our study are freely available. Thus, an ethic approval is not necessarily for our study.

Data sharing statement

The datasets analyzed during the current study will be available from the corresponding author on reasonable request.