Psychometric Properties of the Filipino Version of Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS): A Cross-Cultural Validation Study.

Albeit the importance of patient experience, most questionnaires are only available in English. To understand the hospital experience of Filipino patients, a psychometrically sound instrument in Filipino is warranted. This study culturally adapted and validated the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) into Filipino. A 5-step cross-cultural validation process was conducted. Forward translation, back-translation, and panel reconciliation involved 7 language experts. Pretesting included content validation and pretesting of the Filipino HCAHPS, while field testing involved 64 purposively selected hospitalized patients who completed a 4-part survey from July to December 2018. Content, linguistic, and conceptual equivalence and internal consistency were statistically appraised. Content validation yielded a scale content validity index/average of 1.00. Comparative analysis and Bland-Altman plots indicated good linguistic equivalence. All correlation coefficients were ≥.30, denoting good conceptual equivalence. Cronbach's α for both versions of HCAHPS were ≥0.80, suggestive of good internal consistency. The Filipino HCAHPS is a psychometrically sound and culturally appropriate tool to measure patient experience among Filipinos. This understanding can be utilized for quality improvements on both practice and policy levels.

Introduction

Patient experience, defined as “the sum of all interactions, shaped by an organization’s culture, that influence patient perceptions across the continuum of care” (1,2), has been one of the rising interest in the health-care system to promote patient-centered care. Jha et al (3) even posited that it is one of the measures of patient-centered care which is defined as a “healthcare that establishes a partnership among practitioners, patients, and their families (when appropriate) to ensure that decisions respect patients’ wants, needs, and preferences and that patients have the education and support they need to make decisions and participate in their own care” (4).

In recent years, traditional systems and guidelines are being challenged because patient-related outcome measures (PROMs) have been the basis of health-care reimbursements and incentives. Nevertheless, a growing number of studies claimed that patient-related experience measures (PREMs) is one of the main priorities of health organizations, and nowadays, it is routinely monitored to determine quality of care (5). Additionally, patients are persistently seeking high-quality care and services. For example, the National Health Service of England made endeavors to introduce a national public reporting system of PREMs (6,7). The United States developed a standardized national data using the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey to publicly report patient experience of participating hospitals (8). Recent efforts on patient-centered care also made PREM scores, such as those from HCAHPS, a criterion for reimbursement as part of the Value-Based Purchasing Scheme (9). These assertions emphasize the need to focus not only on PROMs but also on PREMs.

Albeit its importance, studies on patient experience remained scarce. In addition, although a patient-centered approach is widely advocated, hospital performance on PREMs varies. Evidence showed that patients frequently do not receive important information on their condition and options for self-management, and there is insufficient patient involvement in developing quality goals (10,11). To add to this, surveys report that patients are mostly dissatisfied with the manner services are organized in an institution, the lack of time for comprehensive consultation, and the difficulty in understanding information from their doctors (12,13). These evidences have substantial implication in improving the humanity of care and may alter other quality-of-care outcome measures, such as medication adherence, utilization of health services, infections, or unnecessary readmissions (14).

With the increasing attention to PREMs, hospital management and managers need to understand the ways of improving patient experience in their organization. A step to achieving such objective is to develop a culturally valid measure to assess patient experience. As previously mentioned, the United States has used the HCAHPS survey to assess and measure patient experience. However, to the best of the researchers’ knowledge, there has been no established counterpart survey in the Philippines. Since a patient’s hospital experience is highly dynamic with the prevailing health-care system of the country, the medical institution’s organizational culture, and other factors unique to one’s geography, it is imperative to adapt an instrument to measure patient experience. To add to this, patients from the Philippines are culturally and linguistically diverse than those from Western countries; thus using questionnaires that are unfit to the Filipino population may contribute to poorer assessment of quality of care, patient safety, and patient experience (15 -18). Hence, this study culturally adapted and determined the psychometric properties of the Filipino version of the HCAHPS among adult Filipino patients and answered the question “What is the item and scale content validity indices, the reliability score, the linguistic equivalence, and the conceptual equivalence of the Filipino and English versions of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS)?”

Methods

Research Design

A cross-sectional, cross-cultural validation study was conducted. According to Polit and Yang (19), cross-cultural validation studies involve ascertainment of the collective and independent adequacy and equivalence of a translated or culturally adapted measure. Beaton et al (20) and Polit and Yang (19) posited that a cross-cultural validation study involves the following steps: (1) forward translation; (2) back-translation; (3) panel reconciliation of translated tools; (4) pretesting of prefinal translated tools; and (5) field testing of the final tool for measurement equivalence.

Step 1: Forward translation

Forward translation refers to the translation of the items, response options, and instructions from the source language to the target language. In this study, the source language was English, and the target language was Filipino. A decentered or symmetric translation of the items, response options, and instructions was conducted. That is, some words or phrases were modified to ensure contextual relevance and cultural applicability in the Philippines.

As recommended by Beaton et al (20), forward translation was done by a team of 5 translators composed of the investigators and 2 Filipino-language experts. The investigators, who are native speakers of Filipino and are fluent in English, translated and modified the HCAHPS. Any inconsistencies or differences in translation were initially resolved by the investigators before sending the translated tool to 2 language experts.

The preliminary Filipino version was then validated by 2 language experts. These experts were oriented to the purpose of the translation and were given written instructions in carrying out the verification of the forward translation process. The Filipino language experts were tasked to evaluate the accuracy and semantics of the translated questions, ensuring that the meaning of the Filipino HCAHPS encapsulates that of the original English version. These experts conducted their tasks separately and upon completion, their comments and recommendations were summarized and deliberated by the researchers in close coordination with the language experts.

Step 2: Back-translation

The reconciled Filipino HCAHPS was then back-translated, a process in which the synthesized and translated instrument was translated back to the source language (English). This step ensures that the original meaning of the HCAHPS was captured by the translated tool (semantic equivalence). The back-translation team was composed of 2 English-language experts who conducted the back-translation independent of one another (20,21). In addition, these experts were blinded of the original English HCAHPS and were given written instructions.

Step 3: Panel reconciliation

This step involved a consultative meeting with the translators and selected research experts to resolve the identified issues during the forward–back translation processes. Problems were then resolved before preparing the survey questionnaires for pretesting.

Step 4: Pretesting of prefinal tool

This phase involved 2 substeps: content validation of the cultural relevance of the prefinal Filipino HCAHPS and pretest of the prefinal Filipino HCAHPS. The content validation substep was done by 7 content experts in scale development, medical-surgical nursing, family medicine, and internal medicine. These content experts evaluated the cultural relevance of the translated items using a 4-point ordinal scale (22), with 1 being “not relevant” and 4 being “highly relevant.” An item content validity index (I-CVI) of ≥0.78 was used to retain items (23). The scale content validity index (S-CVI)/average was then measured by computing the average I-CVI, and an S-CVI/average of ≥.90 was considered adequate (19). Pretesting involved the administration of the measure to 20 respondents (19). After answering the Filipino HCAHPS, respondents underwent a cognitive interview using targeted verbal probes (19,24) to identify any part of the questionnaire that was vague or needed revisions or adjustments.

Step 5: Field testing of final tool

The field testing of the final Filipino HCAHPS was done for 2 purposes: to assess the extent at which the translated tool meets the quality standards for the original purpose of the tool and to evaluate the equivalence of the translated and original tool (19). Since most Filipinos are fluent and/or conversant in both Filipino and English, the field testing was done in a bilingual sample. During this field testing, both the English and the Filipino versions, with different item order, were administered to the respondents.

Patient and Setting

This study was conducted in a single tertiary hospital in Manila, Philippines, which provides both medical and surgical care to an array of patients. The data collection was conducted in the different medical wards of both the Clinical and Private Divisions of the institution which has 460- and 352-bed capacities, respectively.

This study included adult patients (18-59 years old) who were admitted in a medical ward of the selected hospital for at least 1 overnight stay and have been given a discharge order. However, those who were admitted in a psychiatric ward or emergency unit or have been diagnosed of a psychiatric or cognitive condition were excluded. Consecutive sampling was utilized wherein all respondents who meets the eligibility criteria were included as they arrive or have been identified (22).

Research Instruments

For the field testing, a 4-part survey was administered which was composed of the patient data sheet, the health data sheet, the Filipino HCAHPS, and the English HCAHPS. The patient data sheet profiled the respondent’s demographic characteristics (eg, birthdate, sex, marital status, educational attainment, occupation, monthly household income, and main language/dialect spoken at home). Likewise, the health data sheet profiled the respondents’ chronic disease status, smoking status, alcohol consumption, physical activity status, and medical insurance coverage.

Since the HCAHPS was field tested in a bilingual sample, both English and Filipino versions were administered. The HCAHPS is a 32-item survey, with 25 core items and 7 demographic questions. It has been used in previous studies as a measure of patient experience in 6 composite measures, namely, communication with nurses, communication with doctors, responsiveness of hospital staff, pain management, communication about medicines, and discharge information. The survey also assesses 2 individual items—cleanliness and quietness of hospital environment—and 2 global ratings—hospital rating and willingness to recommend hospital.

Data Collection and Ethical Considerations

The authors initially secured ethical clearance and institutional approval. Afterward, the authors coordinated with the different department heads and nurse supervisors for their endorsement. Formal data collection was conducted from July to December 2018. During each data collection, the authors coordinated with the nurses-on-duty and inquired for a list of patients who met the eligibility criteria. The authors then initially screened these respondents, visited them, and invited them to participate in the study. Full disclosure about the study’s background was provided in the patient’s room, where only the researchers and the potential respondent were present. Afterward, written informed consent was secured. The survey packet was then distributed which took 15 to 20 minutes to complete. All data collection forms were initially screened for completeness before being secured in a sealed, coded envelope. All collected forms were then encoded in a password-protected hard drive that was kept in our research office. Completed questionnaires will be stored for 3 years and will be disposed by shredding.

Data Analysis

Statistical analyses were performed using STATA Statistical Software, version 13 (StataCorp LP, College Station, Texas). A P value ≤.05 was considered significant. Descriptive statistics included mean, standard deviation (SD), frequency, percentage, median, and interquartile range (IQR) depending on the nature of data measurement. Basic item analysis using Spearman’s rank-order correlation were performed to evaluate interitem correlations (19). Content validity indices were assessed using both I-CVI and S-CVI/average (23). Reliability coefficients were assessed using Cronbach’s α with a cutoff score of 0.70 (25). This study also evaluated the equivalence between the English and the Filipino versions of HCAHPS. Bland-Altman plots were used to assess the degree of agreement between the 2 versions. Comparative analyses using 1-sample t test was conducted for the tools’ item, subscale, and overall scale scores (19).

Results

Demographic and Clinical Profiles of the Respondents

Results indicated that most respondents were male (64.06%), were married (60.94%), had completed a tertiary-level education (53.13%), had a monthly household income greater than PHP 20 500.00 (US$410.00; 29.51%), and worked in the service industry (31.58%). The mean age of the respondents was 49.31 years old (SD = 17.75), while the median family size was 4 persons (IQR = 2-5; Table 1). Results also indicated that the most common comorbidity was hypertension (37.50%), and half of the respondents were nonsmokers.

Table 1.

Demographic and Clinical Profiles of the Respondents.a

Characteristic	Frequency (f)	Percentage (%)	Mean (SD) or Median (IQR)
Age			49.31 (17.75)
Sex
Male	41	64.06%
Female	23	35.94%
Marital status
Single	15	23.44%
Married/live-in	39	60.94%
Widow/widower	10	15.63%
Educational attainment
Primary education	10	15.63%
Secondary education	19	29.69%
Tertiary education	34	53.13%
Postgraduate education	1	1.56%
Monthly household income
Less than PHP 3500 (US $70.00)	16	26.23%
Between PHP 3500 and PHP 4999 (US$70.00-US$99.99)	8	13.11%
Between PHP 5000 and PHP 8499 (US$100.00-US$169.99)	11	18.03%
Between PHP 8500 and PHP 20 499 (US$170.00-US$409.99)	8	13.11%
More than PHP 20 500 (US$410.00)	18	29.51%
Occupation
Management, business, and financial	7	12.28%
Professional and related	3	5.26%
Services	18	31.58%
Sales and related	5	8.77%
Office and administrative support	4	7.02%
Farming, fishing, and forestry	3	5.26%
Construction and extraction	2	3.51%
Installation, maintenance, and repair	2	3.51%
Production	2	3.51%
Transportation and materials moving	0	0.00%
Unemployed	11	17.19%
Family size			4 (2-5)
Main language/dialect spoken
Filipino	64	100.00%
English	64	100.00%
Bisaya	9	14.06%
Ilocano	3	4.69%
Pangasinense	1	1.56%
Kapampangan	1	1.56%
Comorbidities
Hypertension	24	37.50%
Diabetes mellitus	16	25.00%
Asthma	2	3.23%
Heart disease	1	1.56%
Smoking status
Current smoker	13	20.31%
Previous smoker	19	29.69%
Nonsmoker	32	50.00%

Abbreviations: IQR, interquartile range; SD, standard deviation.

a N = 64.

Content Equivalence

After 2 iterations of validations, all items were retained but slightly modified to ensure cultural relevance. The computed I-CVI was 1.00 with a scale-content validity/average (S-CVI/average) of 1.00.

Linguistic Equivalence

Table 2 illustrates the descriptive statistics and the mean (SD) and the mean difference (SD) scores (Filipino minus English) for each item and composite of the HCAHPS. It can be gleaned from the table that the mean scores of all composites were above their midpoint values, denoting positive patient experience in these domains. Moreover, all mean differences were less than ±0.25. Comparative analyses also indicated that all items were not statistically different between the Filipino and the English versions. In a similar vein, results showed that the comparative analyses of the mean (SD) scores of all 6 composite measures were not statistically different (P > .05).

Table 2.

Mean (Standard Deviation) Scores and Mean Difference Scores for All Items and Composite Measures of the Filipino and English Versions of the HCAHPS.a

Items and Composite Measures	Mean (SD) of Filipino Version	Mean (SD) of English Version	Mean Difference (SD) of Scoresb
Item Number 1	3.77 (0.46)	3.76 (0.41)	0.01 (0.21)
Item Number 2	3.66 (0.51)	3.66 (0.46)	0.00 (0.17)
Item Number 3	3.64 (0.55)	3.66 (0.52)	−0.02 (0.24)
Item Number 4	2.92 (0.61)	2.96 (0.65)	−0.05 (0.33)
Item Number 5	3.77 (0.46)	3.76 (0.41)	0.01 (0.23)
Item Number 6	3.75 (0.44)	3.75 (0.42)	0.00 (0.11)
Item Number 7	3.83 (0.38)	3.83 (0.36)	0.00 (0.13)
Item Number 8	3.39 (0.68)	3.44 (0.65)	−0.05 (0.32)
Item Number 9	3.36 (0.69)	3.35 (0.68)	0.00 (0.36)
Item Number 10	1.56 (0.49)	1.58 (0.48)	−0.02 (0.20)
Item Number 11	2.27 (1.41)	2.30 (1.39)	−0.03 (0.46)
Item Number 12	1.39 (0.48)	1.41 (0.48)	−0.02 (0.21)
Item Number 13	2.65 (1.40)	2.73 (1.35)	−0.08 (0.42)
Item Number 14	2.67 (1.41)	2.74 (1.36)	−0.07 (0.42)
Item Number 15	1.40 (0.49)	1.36 (0.46)	0.04 (0.19)
Item Number 16	2.82 (1.41)	2.94 (1.29)	−0.12 (0.55)
Item Number 17	2.82 (1.40)	2.94 (1.30)	−0.12 (0.66)
Item Number 18	1.03 (0.18)	1.03 (0.18)	0.00 (0.01)
Item Number 19	0.98 (0.13)	0.98 (0.13)	0.00 (0.18)
Item Number 20	1.00 (0.00)	0.98 (0.13)	0.02 (0.13)
Item Number 21	8.86 (1.54)	8.90 (1.55)	−0.04 (0.18)
Item Number 22	3.39 (0.71)	3.38 (0.71)	0.01 (0.12)
Item Number 23	3.54 (0.48)	3.54 (0.48)	0.00 (0.00)
Item Number 24	3.52 (0.48)	3.52 (0.48)	0.00 (0.00)
Item Number 25	3.47 (0.65)	3.43 (0.73)	0.04 (0.35)
Composite 1: Communication with Nurses	3.71 (0.41)	3.71 (0.39)	0.00 (0.13)
Composite 2: Communication with Doctors	3.78 (0.35)	3.78 (0.33)	0.00 (0.12)
Composite 3: Responsiveness of Hospital Staff	2.57 (0.74)	2.66 (0.79)	−0.09 (0.50)
Composite 4: Pain Management	2.63 (1.40)	2.71 (1.36)	−0.08 (0.42)
Composite 5: Communication about Medicines	2.63 (1.40)	2.71 (1.36)	−0.08 (0.42)
Composite 6: Discharge Information	0.99 (0.05)	0.98 (0.13)	−0.01 (0.14)

Abbreviations: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; SD, standard deviation.

a N = 64.

b Filipino version scores minus English version scores.

The Bland-Altman plots for the English and Filipino HCAHPS are presented in Figure 1. The Bland-Altman plots illustrated the difference in mean scores against the average scores of each composite measure for both versions. The mean difference scores (95% confidence interval [CI]) for the 6 composites were 0.00 (−0.04 to 0.02), 0.00 (−0.03 to 0.03), −0.09 (−0.13 to 0.04), −0.08 (−0.18 to 0.03), −0.08 (−0.26 to 0.02), and 0.01 (−0.03 to 0.04), respectively. Overall, 90.62%, 95.31%, 93.74%, 92.19%, 92.19%, and 96.88% of the points lay within the 95% CI of the 6 composites, respectively. Lin’s concordance correlation coefficients for the 6 composites were 0.96, 0.94, 0.90, 0.95, 0.90, and 0.90, respectively. Bland-Altman plots of all HCAHPS items revealed that 87.10% to 100.00% of points lay within their respective 95% CIs.

Figure 1.

Bland-Altman plots for the different composite measures of the Filipino and English Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS).

Conceptual Equivalence

The correlation analyses for each item and their respective composite measure is presented in Table 3. As illustrated, the correlation coefficients ranged from 0.34 to 1.00 for the Filipino HCAHPS, while in the English HCAHPS, correlation coefficients varied from 0.33 to 1.00. Results also indicated that all correlation coefficients were statistically significant at 0.01 in both versions.

Table 3.

Spearman’s Rank-Order Correlation Coefficients for Each Item on the Filipino and English Versions of the HCAHPS With Their Respective Subscale Score.a

Items and Composite Measures	Spearman’s ρ for Filipino Version of HCAHPS	Spearman’s ρ for English Version of HCAHPS
Composite 1: Communication with Nurses
Item 1	0.79c	0.83c
Item 2	0.93c	0.96c
Item 3	0.88c	0.90c
Composite 2: Communication with Doctors
Item 5	0.74c	0.79c
Item 6	0.86c	0.88c
Item 7	0.77c	0.73c
Composite 3: Responsiveness of Hospital Staff
Item 4	0.34c	0.33c
Item 11	0.91c	0.91c
Composite 4: Pain Management
Item 13	0.98c	0.98c
Item 14	0.97c	0.97c
Composite 5: Communication about Medicines
Item 16	0.98c	0.99c
Item 17	0.99c	0.98c
Composite 6: Discharge Information
Item 19	1.00c	1.00c
Item 20	1.00c	1.00c

Abbreviations: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems.

a N = 64.

b Significant at 0.05.

c Significant at 0.01.

Internal Consistency

The Cronbach’s α for the Filipino and English versions were 0.845 and 0.852, respectively. Comparative analyses indicated that these values were not statistically different (χ2 = .03, P = .860), suggesting good internal consistency for both versions of the instrument.

Discussion

This study translated, culturally adapted, and validated the HCAHPS into the Filipino language. This study, to the best of the researchers’ knowledge, was the first endeavor to translate and culturally adapt the HCAHPS into Filipino. In general, results showed that the Filipino HCAHPS had acceptable content equivalence, linguistic equivalence, conceptual equivalence, and internal consistency.

Findings have shown that all 25 items of the HCAHPS had good content and semantic equivalence after undergoing 2 iterations of content validation. Although the HCAHPS was composed of 32 items, items 26 to 32 were no longer included in the final analysis, since these questions pertained to the respondent’s demographic profile. It is also notable that all 25 HCAHPS items were retained after several repetitions of content validation. However, the second response option for item 18 was slightly modified to make it culturally applicable in the Philippines. These results were achieved after the rigorous process of back-translation, a corner stone strategy in Brislin’s (26) Model of Translation. This approach allowed the identification and modification or elimination of vague, erroneous, and culturally inappropriate items from the original questionnaire (27).

Results also showed that the patients had positive experiences in their communication with the doctor and nurses, the responsiveness of hospital staff, pain management, communication about medicines, and discharge information. These results denote that the patients were satisfied with the communication attitude and adequacy of information provided by the medical professionals (28). Moreover, these results indicate that the patients’ expectations with their medical care were met by the health-care professional and medical staff. Nonetheless, since the current study was conducted in a single institution, these results should be analyzed with caution, since patient experience may vary from one medical institution to another.

It is also interesting to note that the Filipino HCAHPS had acceptable linguistic equivalence, indicative that the translated version was an equivalent of the English HCAHPS in a culturally appropriate fashion. All 25 items and 6 composites of the HCAHPS had mean differences less than the ±0.25 cutoff, denoting acceptable variation between the 2 versions (29,30). Bland-Altman plots for all items and composite measures also provided evidence that there were no significant differences between the 2 versions of the questionnaire (31). Lin’s concordance scores also indicate that at least 90% to 96% of the scores of the Filipino and English HCAHPS were concordant (32). The minor variation in the scores, however, may be attributed to the rewording of the items when these were translated to Filipino. It must be noted that the main objective of the translation and back-translation procedures were to translate the questionnaire into a language which encapsulated the intended meaning of the original English items instead of creating a literal translation.

The results also showed that the Filipino and English HCAHPS were comparable for each item with their respective composite, indicative of conceptual equivalence. Both the Filipino and the English versions were able to assess and measure the same theoretical construct—patient experience—in both languages and cultures. It is also interesting to note that the difference in correlation coefficients between the Filipino and English versions of the HCAHPS ranged from 0.01 to 0.05, which indicates higher conceptual equivalence (29,30).

As for the internal consistency of the Filipino and English versions, the Cronbach’s α values of the 2 versions were 0.845 and 0.852, respectively. Formal statistical analysis indicated that these α coefficients were comparable (33), suggestive that the 2 versions of HCAHPS have acceptable internal consistency (19). These results also denote that the Filipino and English versions were reliable in measuring patient experience.

Albeit the presented results, this study has its limitations. First, the study included a small number of respondents; thus, future research may consider increasing the sample size to further evaluate the psychometric properties of the HCAHPS. Second, over- and underestimations of results is a possibility, since the HCAHPS survey was a self-reported instrument. Hence, the current results must be analyzed with caution. Finally, the study was conducted within Metro Manila, Philippines only, a highly urbanized city, where health-care services and settings are greatly different from other institutions in the provinces of the country.

Conclusion

This study translated, culturally adapted, and validated the HCAHPS into the Filipino language. This study, to the best of the researchers’ knowledge, was the first study to conduct a rigorous cross-cultural validation of the HCAHPS into a different language and culture. By and large, findings indicated that the Filipino HCAHPS has acceptable content equivalence, linguistic equivalence, conceptual equivalence, and internal consistency.

The Filipino HCAHPS can be a valid and culturally appropriate questionnaire to measure patient experience among the Filipino population. To add to this, this Filipino version of the HCAHPS allows non-English-speaking Filipinos to impart their hospital experience to their health-care professionals. This understanding provides avenue for the development of appropriate hospital policies, programs, and interventions that focus on improving hospital experience among hospitalized patients.