Measuring Healthcare Quality - Paradigm of Medqual.

Medical activity is assumed to be service activity the effects of which can be measured. The SERVQUAL scale was used as a starting point for our research, which resulted in a new, adopted scale called MEDQUAL. The MEDQUAL scale aims to measure the quality of healthcare provided by medical staff of one hospital department instead of the overall quality of hospital services or parts of services on which medical staff in one department has no influence. The study was conducted in a clinical hospital department in Croatia and included 300 respondents (169 patients and 131 medical staff members). The MEDQUAL scale, designed and tested in the study, showed high reliability in all established dimensions, i.e. trust in doctors (Cronbach's Alpha 0.923), nursing professionalism (0.913), medical professionalism (0.938), and departmental organization (0.810). The scale proposed evaluates both patient satisfaction with the quality of healthcare received and medical staff satisfaction with the quality of healthcare provided. The results were comparable to the groups of respondents, departments, and institutions with potential longitudinal studies of this phenomenon. MEDQUAL is a simple, repeatable and cost-effective scale, applicable to almost all departments and used for measuring the quality of healthcare services both provided and received, the aim of which is to contribute to the assessment of healthcare quality and its improvement.

Introduction

Development of the service sector and its leading role in relation to other types of activities in terms of the number of employees and service users at the end of the 20th century caused development of methods for measuring the level of quality of the services provided (-). This trend has also spread to medicine as a result of an increase in patient expectations, which have ultimately led to the development of private practice, as well as a growing need for assessing economic impacts of medical activities in terms of cost-effectiveness. There has been a significant number of studies measuring healthcare quality that are primarily based on Parasuraman’s SERVQUAL or its minimum adaptation (-). Most studies essentially contain some elements of healthcare services which cannot be influenced by medical staff because these are either inherited problems (e.g., urban plan, parking, size of hospital rooms, access to sanitary facilities, etc.) or within the domain of other hospital departments (e.g., meal delivery services, bed linen cleaning and delivery services, etc.). This indicates the need to develop a scale that would measure the quality of specific healthcare services provided by medical staff in a department instead of the quality of healthcare services in general. The authors tested and proposed the MEDQUAL scale that simultaneously assesses the quality of healthcare provided by medical staff and received by patients in one hospital department. In addition, opinions and ratings of both patients and medical staff referring to the medical service provided or received were analyzed. Both patients and medical staff were asked to rate the trust in doctors, medical professionalism, nursing professionalism, and organization of the department.

Methods

Research setting and sample

The study was conducted at a surgical department in a university hospital in Croatia that covers all surgical specialties (abdominal, trauma, vascular, plastic, cardiac, thoracic and pediatric surgery with emergency and follow-up outpatient care). The department under study has 200 beds and performs approximately 13,000 surgical operations and minor surgery procedures annually and approximately 90,000 examinations through emergencies and follow-ups, as well as examinations in other departments and clinics. Before the study, both approval from the institutional Ethics Committee and Josip Juraj Strossmayer University of Osijek and consent for participation were obtained from medical and nursing staff working at the surgical department and from the patients.

Sample size requirements

Factor analysis was used for testing the constructs and the measurement scale. The study included 300 respondents and met the criterion referring to the sample size of 100 and more respondents (the ratio of 10:1, or 10 respondents for each variable included in factor analysis) since 25 variables of the proposed scale were tested (, ).

Data collection

A structured questionnaire consisting of two parts was developed for the purpose of this study. The first part contained MEDQUAL scale items, while the second part examined socio-demographic characteristics of respondents. The study involved all doctors from the department under study (n=30). The number of participants in the groups of nursing staff and patients was determined proportionally to the ratio between the number of beds and the size of the department. The sample was created by combining a convenience and quota sample. The convenience sample consisted of respondents associated exclusively with the department under study, while the quota sample referring to the quota of medical staff was created based on the ratio (i.e. quota) of the number of medical staff working at the department under study and the number of medical staff employed.

Research instrument

The construct was designed by modifying the SERVQUAL scale. An expert committee consisting of five members (psychiatrist, economist, statistician/methodologist, surgeon, and lawyer) identified measurable elements of quality associated with the department under study. The items and dimensions of the MEDQUAL instrument proposed were adapted by using a theoretical framework of the scale called SERVQUAL and they were used to measure specific terms related to the department where this research was conducted (, ). A specific feature of this study lies in the fact that all those items which healthcare staff of individual departments cannot influence personally were excluded from the research. The structure and hence the analysis of the new instrument proposed differs from the SERVQUAL methodology because the original testing instrument is based on two identical parts of the questionnaire a service receiver has to fill out before (Expectations) and after (Perception) receiving a service. When applying MEDQUAL, the researchers use one questionnaire and start with the assumption that the expectations of a healthcare service are always highest (in case of a 5-point Likert scale, the expectations are measured by number 5) (, , ). Therefore, patients and medical staff were offered only one 23-item questionnaire, which measured the perception of the quality of the healthcare service provided/received. The items are related to the perception of patients/medical staff of the following: (1) building trust in the doctor-patient relationship; (2) professionalism of doctors while providing a healthcare service; (3) professionalism of nurses while providing a healthcare service; and (4) organization of the department.

Statistical analysis

Categorical data were expressed as absolute and relative frequencies. Numerical data were described as the mean and standard deviation in the case of normal distribution, and in other cases as the median and limits of the interquartile range. Exploitation factor analysis was used to test the variable structure and dimensionality of the construct measured. Bartlett’s test of sphericity and Kaiser-Meyer-Olkin (KMO) test were used for investigation of mutual correlation between manifest variables. The Kaiser-Rice criterion (1974) was used to interpret the KMO test, i.e. a value greater than or equal to 0.9 is marvelous, a value of at least 0.8 is meritorious, a value of 0.7 or more is middling, a value of 0.6 or more is mediocre, a value of 0.5 or more is said to be miserable, and a value less than 0.5 is considered unacceptable (). In factor rotation, the Varimax method was applied by using the Kaiser criterion.

We examined internal consistency by means of Cronbach’s alpha reliability coefficient. To test the difference between the average individual MEDQUAL dimensions and the types of respondents that assessed the quality of medical services, the authors used the independent samples t-test or Mann-Whitney test, when appropriate. The normality of distribution was tested by Kolmogorov-Smirnov test. The level of significance was set at 0.05. Data analysis was performed by using the IBM SPSS Statistics 23.0 statistical software.

Results

The study included 300 respondents. The sample consisted of 130 (43.3%) male and 170 (56.7%) female representatives (Table 1).

Table 1

Research respondents

		Medical service recipients (patients), n (%)	Medical service providers (medical staff), n (%)	TotalN (%)
Gender[n (%)]	Male	89 (52.7)	41 (31.3)	130 (43.3)
	Female	80 (47.3)	90 (68.7)	170 (56.7)
	Total	169 (100.0)	131 (100.0)	300 (100.0)
Age	n	169	131	300
	M (SD)	49.7 (16.8)	39.6 (10.3)
Level of education[n (%)]	Elementary school	30 (18.5)		30 (10.3)
	High school	93 (57.4)	81 (62.3)	174 (59.6)
	Junior college	18 (11.1)	17 (13.1)	35 (12.0)
	Bachelor’s degree	18 (11.1)	28 (21.5)	46 (15.8)
	Master’s or PhD degree	3 (1.9)	4 (3.1)	7 (2.4)
	Total	162 (100.0)	130 (100.0)	292 (100.0)

More female representatives (χ2=13.718, df=1, p<0.001) were found in the group of medical service providers. The group of medical service recipients was on average slightly older (49.7 years) than the group of medical service providers (39.6 years).

The scale tested was composed of 25 items. After rotation (Varimax using the Kaiser criterion), items were distributed in 4 factors. Retained items (i.e. 23 items) with the highest loading factors are shown in Table 2. All correlations in the correlation matrix were above 0.5, the value of Bartlett’s test for statistical significance of the correlation matrix was χ2=6833.86 (df=253, p<0.001). The KMO test result was 0.943.

Table 2

Rotated component matrix (23 MEDQUAL items)

	Component
	1	2	3	4
The doctor explained to the patient in an easy and understandable way the procedures and interventions that need to be done.	0.855
The doctor informed the patient about the potential risks.	0.854
The doctor explained to the patient his/her health condition in an easy and understandable way.	0.788
The doctor responds to a patient’s need for conversation and additional information.	0.763
A patient’s right to privacy is ensured.	0.550
A patient has confidence in the doctor.	0.527
Nurses are compliant (willing to help).		0.812
Nurses kindly respond to a patient’s call.		0.810
Nurses do their job professionally.		0.753
Patients have confidence in nurses.		0.706
Nurses look neat.		0.620
A surgical service has been performed professionally.			0.783
You felt safe during the performance of a surgical service.			0.743
Doctors are experts.			0.673
Doctors are compliant (willing to help).			0.604
Doctors take care of patients.	0.516		0.566
Doctors are professional in regard to their relationship with patients.			0.552
Doctors look neat.			0.517
A surgical service was provided at a time agreed upon in advance, with no longer waiting period.			0.517
A surgical department is equipped with modern equipment.				0.737
The rooms for hospitalized patients are adequately equipped.				0.729
A surgical department is well equipped with medical supplies.				0.716
A surgical department is clean (tidy).				0.538
% of explained variance	20.75	18.90	18.21	14.87
Cumulative %of explained variance	20.75	39.56	57.86	72.73

The total variance explained for MEDQCAL was 72.73% (Table 2). Factor 1, which has the highest value of explained variance of the scale presented, may be called ‘Trust in the doctor’ in the patient-medical staff relationship, and is characterized by variables related to building the doctor-patient relationship, which is the basis of every treatment. The second and third factors relate to medical staff professionalism, where the former consists of items related to nursing professionalism, and the latter to medical professionalism. Both factors are characterized by variables related to staff attitude towards their work, which contributes to further strengthening of confidence and patient safety. The items associated with the organization and functioning of the department, as well as its equipment are grouped into the last, i.e. fourth factor called ‘The organization of a department’.

Psychometric analysis of the entire MEDQUAL scale showed internal consistency since the standardized Cronbach’s alpha was 0.961. The inter-item correlation and item-total correlation were examined in the analysis of reliability. The value of the standardized Cronbach’s alpha in certain dimensions of the MEDQUAL scale ranged from 0.810 to 0.938 (Table 6).

Table 6

Review of internal consistency of each dimension of the MEDQUAL scale and total MEDQUAL scale

Dimension	MEDQUAL	1	2	3	4
Number of items	23	6	5	8	4
Cronbach’s alpha	0.961	0.922	0.911	0.933	0.808
Standardized Cronbach’s alpha	0.964	0.923	0.913	0.938	0.810
Mean item value	4.080	3.906	4.144	4.233	3.863
Mean inter-item correlation	0.537	0.668	0.667	0.654	0.516
Mean item-total correlation	0.718	0.781	0.780	0.778	0.631

Differences in the assessment of healthcare service quality (the MEDQUAL scale and every single dimension of the MEDQUAL scale) assigned by patients and medical staff (Table 3), medical staff (Table 4) and patients (Table 5) were examined. In relation to medical staff, all patients (p<0.001) significantly better assessed the quality of the healthcare service received in all dimensions of the MEDQUAL scale and the entire MEDQUAL scale (Table 5).

Table 3

Evaluation of the quality of the healthcare service provided/received in surgical department

MEDQUAL and dimensions		Patients (n=153)	Medical staff (n=131)	Difference between medians	95% CI for difference in medians		p*
		Median			Lower	Upper
MEDQUAL		4.54	3.87	0.65	0.41	0.89	<0.001
Trust in the doctor		4.67	3.67	1.00	0.65	1.35	<0.001
Professionalism	Nurses	4.80	4.00	0.80	0.52	1.08	<0.001
	Medical (doctors)	4.75	3.88	0.87	0.64	1.10	<0.001
Organization of the department		4.25	3.50	0.75	0.47	1.03	<0.001

*Mann-Whitney test; 95% CI = 95% confidence interval

Table 4

Assessment of the quality of the healthcare service provided by surgical department

MEDQUAL and dimensions		Nurses (n=101)	Doctors (n=30)	Mean difference	95% CI of difference		p*
		Mean (SD)			Lower	Upper
MEDQUAL		3.78 (0.70)	3.94 (0.68)	-0.159	-0.45	0.13	0.275
Trust in the doctor		3.62 (0.88)	3.75 (0.91)	-0.129	-0.49	0.24	0.485
Professionalism	Nurses	4.00 (0.74)	3.94 (0.86)	0.062	-0.25	0.38	0.697
	Medical (doctors)	3.90 (0.75)	4.24 (0.60)	-0.136	-0.63	-0.04	0.029
Organization of the department		3.51 (0.81)	3.64 (0.92)	-0.332	-0.48	0.21	0.436

*Independent samples t-test; 95% CI = 95% confidence interval

Table 5

Assessment of the quality of the received healthcare service provided by surgical department according to the type of patients

MEDQUAL and dimensions		Outpatient (n=78)	Inpatient (n=76)	Difference between medians	95% CI for difference in medians		p*
		Median			Lower	Upper
MEDQUAL		4.48	4.66	-0.18	-0.46	0.11	0.067
Trust in the doctor		4.64	4.67	-0.04	-0.44	0.37	0.566
Professionalism	Nurses	4.60	5.00	-0.40	-0.73	-0.07	0.048
	Medical (doctors)	4.67	4.83	-0.16	-0.42	0.10	0.210
Organization of the department		4.25	4.50	-0.25	-0.62	0.12	0.028

*Mann-Whitney test; 95% CI = 95% confidence interval

For all dimensions of the MEDQUAL scale and the total MEDQUAL scale, patients evaluated the quality of services provided higher than medical staff. Doctors assigned a statistically significantly higher mean rating (4.24) to the dimension of professionalism than nurses (3.90) (Table 4).

Finally, in the analysis of satisfaction with the quality of medical services, there was no statistically significant difference in any dimension with regard to the type of patients (i.e. inpatient or outpatient). Since there were no statistically significant differences, it was confirmed that the same standards were used in taking care of all types of patients.

Discussion

Several papers have been published regarding measurement of the quality of medical services so far, but none of them simultaneously tested two groups, i.e. four subgroups of respondents who participate in the process of receiving/providing healthcare services (-). In this study, the MEDQUAL scale was designed and tested. The authors aimed to construct a questionnaire that would collect data easily, be applicable to all participants involved in the process of both receiving and providing healthcare services and to all hospital departments, and be adaptable to future trends in hospital practice.

MEDQUAL items examine personal and collective contribution of medical staff to the quality of healthcare provided in a particular department. The MEDQUAL scale was designed to avoid the impact of external, i.e. paramedical factors on final assessment of the quality of healthcare services. Differences in the attitudes of two groups of medical staff (doctors and nurses) and the divergent views of medical staff and patients, as well as differences between outpatients and inpatients on the provided/received healthcare service were analyzed.

The characteristics of this scale in terms of its application in assessing the quality of a healthcare service provided/received in one or all departments in hospitals are as follows: repeatable, cost-effective, reliable, time-saving (examines only perception), interpretable and comparable. In this study, comparability as a positive feature of the MEDQUAL scale refers to the comparability of its results by groups of subjects, departments, institutions and time.

Analysis of results shown in Tables 4 and 5 shows that the lowest ratings in all groups of respondents were assigned to the organization of the department, on which medical staff had the least direct impact, whereas professionalism of medical staff obtained the highest scores in all examined groups. In the ‘trust in the doctor’ dimension, a lower score could be attributed to the lack of time that doctors devote to their patients to the extent which both the patient and medical staff consider necessary. Given the shortage of medical staff and the fact that they are overburdened, particularly with administrative tasks, such result is not unexpected. Further improvements in the quality of medical services could be achieved by reallocating tasks to the existing administration following appropriate training, better computerization and connection of the health information system that would allow easier and faster access to relevant information and enable doctors and nurses to have more time for patients.

Conclusion

Medical institutions are providers of healthcare services, which are specific in all their segments, particularly when it comes to their impact on the health and life of the service recipient. As quality measurement of the majority of the services that are provided can be developed methodologically, the quality of the medical service provided is also a multidimensional measurable construct. By measuring service quality, a system of indicators has been established to determine those dimensions of the service that require analysis and training in a particular healthcare system. In this study, measurement of the quality of healthcare services was based on the existing theoretical and applicable knowledge of measuring concepts of the service phenomenon that had already been developed. Since SERVQUAL as one of these concepts has experienced numerous adaptations in various service activities, it has been adapted to MEDQUAL in the field of medical services, analyzed and critically commented. The MEDQUAL scale has certain drawbacks. The first is its low specificity. This deficiency can be resolved by adding items that are specific for a certain area or group of patients (e.g., pediatrics, psychiatry). Ongoing research of the MEDQUAL scale could further be adapted and applied to specific groups (PEDQUAL, PSYCHOQUAL), or to the institution as a whole and in order to detect additional factors that affect the quality of healthcare service (HOSPIQUAL). The second drawback of the results presented is that testing was made only in one hospital department. The third disadvantage can be found in the need to collect data in waves (i.e. equal time intervals), which is organizationally demanding. The findings of this study provide a simultaneous assessment of the quality of healthcare services provided/received in one department with the possibility of a longitudinal study of this phenomenon. In order to make further progress in improving the quality of healthcare services, it is essential to engage department managers/stakeholders and to sustain interest in and the need for cooperation with medical staff, which could result in better organization of departments and better equipment, and more time for patients, which will eventually result in higher patient and medical staff satisfaction, and ultimately, increased satisfaction with the healthcare service provided/received.