Patient satisfaction with out-patient rehabilitation therapy: validity and reliability.

[Purpose] The purpose of present study were to develop an out-patient satisfaction questionnaire to be used in health care system, from which the underlying dimensions could be derived and individual patient scores calculated, and to evaluate some of the questionnaire's psychometric properties.
[Subjects and Methods] Forty out-patient of local hospital reply both the questionnaire used in previous study and newly designed questionnaire. To identifying validity, the statistical linear relationship between the total score of the primary questionnaire and newly designed questionnaire, which were analyzed. The test-retest reliability has been investigated by using a single measure intra class correlation.
[Results] The average satisfaction of the previous questionnaire were significantly correlated with newly designed questionnaire. The intra-lass correlation coefficient of the each items of newly designed questionnaire were strong. Total score of the previous questionnaire had the lowest test-retest reliability, Cronbach' s alpha coefficient for the newly designed questionnaire score showed acceptable inter-item reliability.
[Conclusion] The out-patients' satisfaction questionnaire developed in present study, which had appropriate validity, reliability, and acceptability.

INTRODUCTION

Concurrent improvement of the quality of healthcare services is a major interest when evaluating medical facilities1, 2). Classically, an evaluation of medical services did not consider patient preferences, but emphasised technical achievements3). However, there has been growing interest in measuring patient satisfaction in healthcare research, reflecting a move towards patient-centred care2, 4). This trend to patient-centred care has been strengthened by the increasingly competitive marketplace. Patients who are highly satisfied with a medical service are more likely to seek additional services and recommend a medical facility to others4).

The Department of Rehabilitation provides physical, occupational, language, and therapeutic assessments and treatments for outpatients. Patient satisfaction with rehabilitation services and the attitude of the therapist is as important for assessing therapy as diagnostic procedures and the environment of the facility5). Generally included items are location and accessibility, cost, payment, and facility5, 6). Previous researchers who have reported on patient satisfaction related to rehabilitation services have emphasised specific items to measure the therapist’s manner and successful rehabilitation4).

The usefulness of a questionnaire cannot be evaluated with subjective criteria but rather only with objective criteria such as validity and reliability, and most studies try to ensure that the validity of an instrument is well founded7). An uncertain questionnaire item will generate an uncertain reply from a patient. A good questionnaire contains differentiated items for evaluating the environment, therapeutic technique, therapist’s manner, and appropriate rehabilitation process4,5,6).

There is a few study related patient satisfaction of rehabilitation system in Korea. The purpose of present study were to develop an outpatient satisfaction questionnaire to be used in health system, from which the underlying dimensions could be derived and individual patient scores could be calculated.

SUBJECTS AND METHODS

The MEDLINE and Science-Direct databases for the years 2000–2014 were searched to find instruments that had been devised to evaluate patient satisfaction at national and international levels. The keywords used for the search were patient satisfaction, questionnaires, rehabilitation, and validity. An agenda was developed for categorising questions on therapeutic satisfaction, courtesy satisfaction, proceeding satisfaction, environment satisfaction, revisits, and recommendations. The final questionnaire contained 29 positively and negatively worded statements that are scored using a five-point Likert scale ranging from “strongly disagree” to “strongly agree.”

All the patients coming for rehabilitation therapy who were willing to participate in the study were included. In all, 40 outpatients with a mean age of 57.4 ± 3.8 (range 42–62) years completed questionnaire. All of the patients who were unwilling to participate in the study were excluded. Present study did not approved by ethics committee for just gaining user’s satisfaction about the medical service. All the patients gave written informed consent when getting questionnaires so that returned envelop enclosed the consent about joining this study.

After their first visit to the rehabilitation therapy, each patient was given two questionnaires and written informed consent for joining study with a prepaid return envelope. One was the newly designed questionnaire and the other was the Patient Satisfaction with Outpatient Physical Therapy: instrument validation questionnaire of Beattie et al. A cover letter explained the reasons for conducting the survey, encouraged participation, and guaranteed data confidentiality. Follow-up letters with the same questionnaires were sent to the patients 1 week later. If the patient still had not returned the questionnaire 15 days after the first reminder, they received a third letter with a new copy of the questionnaire. The response rate obtained with this method was 50%. Of the 80 patients who answered, 48 (60%) responded to the first mailing, of whom 8 did not reply to the second mailing to investigate reliability, so the final sample was 40.

The test-retest reliability and validity of the questionnaire were analysed using SPSS ver. 18.0 (SPSS, Chicago, IL, USA). To quantify the statistical linear relationship between the total score of the primary questionnaire and each agenda item and the total score of the newly designed questionnaire, Pearson correlation coefficients were used. The a priori α level was set at 0.01 for the determination of statistical significance. A single-measure intra-class correlation (ICC) was used to test the test-retest reliability.

RESULTS

The average satisfaction with the newly designed questionnaire was 18.87 ± 1.65 for therapeutic satisfaction, 19.02 ± 1.52 for courtesy satisfaction, 18.00 ± 2.18 for processing satisfaction, 9.05 ± 1.17 for environmental satisfaction, 9.22 ± 1.12 for hospital satisfaction, and 74.17 ± 6.72 for the total satisfaction score. The average satisfaction with the previous questionnaire was 55.35 ± 5.39, and the values were significantly correlated with those of the newly designed questionnaire (Tables 1 and 2). The test-retest reliability of all items in the newly designed questionnaire was strong (ICC=0.772, 95% CI 0.61–0.87). The Cronbach’s alpha for the newly designed questionnaire score showed acceptable inter-item reliability (α=0.87).

Table 1.

Pearson correlation between newly designed questionnaire and previous questionnaire

Newly designed questionnaire	Previous questionnaire
	Pearson correlation
Therapeutic satisfaction	0.699*
Courtesy satisfaction	0.718*
Processing satisfaction	0.725*
Environmental satisfaction	0.641*
Hospital satisfaction	0.772*
Total satisfaction	0.811*

*Correlation is significant at the 0.01 level.

Table 2.

Intraclass correlation coefficient (ICC) and 95% coefficient interval (95%CI) between Pre and post questionnaire measurement

Item	Test and retest measurement
	ICC	95% CI	Cronbach’s α
Therapeutic satisfaction	0.719*	0.531–0.841	0.838
Courtesy satisfaction	0.785*	0.631–0.880	0.880
Processing satisfaction	0.683*	0.476–0.819	0.810
Environmental satisfaction	0.672*	0.457–0.812	0.800
Hospital satisfaction	0.740*	0.562–0.853	0.854
Total satisfaction	0.772*	0.611–0.872	0.872
Previous questionnaire	0.622*	0.391–0.780	0.768

*Correlation is significant at the 0.01 level.

DISCUSSION

This study examined the validity of a newly designed satisfaction questionnaire based on questionnaires used in previous research and assessed the reliability of the questionnaire in outpatients. This type of investigation is used not only to confirm the present state of a medical service but also to find a way to improve the quality of a medical service to maximise patient satisfaction and visits7,8,9). Rehabilitation service makes use of diverse therapist techniques and sophisticated instruments10). In a team approach, physical, occupational, and speech therapists have to cooperate to improve patient status, and cooperation among the therapists is one of the most important factors leading to patient satisfaction, as well as courtesy and therapeutic techniques. Present results showed that satisfaction related to processing was relatively lower than for the other items.

The correlation between the previous and newly developed questionnaires was significant only for the total score because using a Cronbach’s alpha of 0.8 at the minimum value for regarding statistical means. The previous questionnaire gives a unified score rather than differentiating diverse agendas, and it has the advantage of providing a representative simple value. However, this makes it difficult to clarify the item characteristics, unlike questionnaire in which each agenda item was moderately correlated with the total patient satisfaction score. The aim of this study was not to devalue the previously developed questionnaire but rather to develop a specific questionnaire for examining the state of medical services. Recently, Mas et al.9) reported positive relationships between the level of patient satisfaction and levels of professional competence, interpersonal manner of healthcare staff, and information.

This study confirmed the test-retest reliability of the new and previous questionnaires. In each dimension, Cronbach’s alpha was > 0.8 for all of the dimensions and the inter-subscale correlation was between 0.731 and 0.883. A previous study analysed the internal consistency of the questionnaire dimensions using Cronbach’s alpha, and reported results above the recommended minimum of 0.78). However, the ICC exceeded 0.7 in the therapeutic and courtesy and hospital dimensions only.

One limitation of this study was the limited number of patients. Second, it was almost impossible to determine satisfaction with other hospital departments because this study could not collect information on the presence or absence of other diseases or surgical history.

The outpatient satisfaction questionnaire developed in this study had suitable validity, reliability, and acceptability. Outpatient satisfaction with the rehabilitation service in Busan, Korea, was relatively high in terms of the therapeutic procedure and courtesy, and relatively low in terms of the environment and processing.