Capturing the patient's voice: Kano Analysis of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.

BACKGROUND: Traditional methods are not able to differentiate which feature customers regard as attractive, mandatory, performance, and which feature customers are indifferent about. These categories can only be differentiated based on a specific technique called Kano survey. Specific aim of this study was to categorize the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) patient satisfaction survey questions into Kano categories.
MATERIALS AND METHODS: Design of the study was survey research. It was conducted from 6/2019 to 8/2019 at OSF Saint Francis Medical Centre in Peoria, Illinois, USA. A 34 question Kano survey (17 positive and 17 negative questions) based on HCAHPS patient questionnaire was designed. Surveys were analyzed using Kano analysis template. Comparative analysis of Kano categories based on demographics was also performed.
RESULTS: 39 current patients and 25 caregivers completed the survey. All of the 17 HCAHPS questions except "noise level at night" were classified as mandatory requirement with highest number for information on "indications of medicines." There was a minimum variability in the satisfaction coefficients but large variation in the dissatisfaction coefficients. More patients above 50 years consider "help going to bathroom" as mandatory (70.2% vs. 40.7%, P = 0.01). Sixty-four percent of caregivers considered "explain things (nurse)" as mandatory as opposed to 51.2% of patients (P = 0.03).
CONCLUSION: Current U. S healthcare consumers have high expectations from healthcare delivery and consider most HCAHPS questions as mandatory requirements. Kano analysis needs to be done on a larger, more diverse hospital setting and potentially the HCAHPS survey needs to be modified to reflect prevailing healthcare customer requirements. Copyright:

Introduction

Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores were first introduced in 2006,[1] and were the result of extensive research conducted jointly by the Agency for Healthcare Research and Quality and the Centre for Medicaid and Medicare Services (CMS).[2] HCAHPS was tied to the annual payment update for inpatient prospective payment system for the fiscal year 2008,[1] leading to rapid nationwide adoption of this survey. Currently, hospitals are required to conduct these surveys each year and report results to CMS. Hospitals either conduct these surveys directly or contract with third parties for assistance. Survey results are then reported to CMS which then analyses the results and publishes the analysis on its ‘Hospital Compare’ website. The results are therefore publicly available, and healthcare consumers are able to make decisions based on hospital performance. This process has had a significant impact on hospitals’ approach to performance improvement, with aims to improve HCAHPS scores.[3]

The design of the HCAHPS survey has been questioned regarding its utility in predicting patient outcomes. Although it was designed and validated with traditional research methodologies, it treats all the responses as equal with no qualification between what a consumer may consider a “mandatory” requirement and versus an “attractive” requirement.[4] This limitation in applicability requires an alternative technique to identify which domain(s) a patient truly cares about.[4] Kano modelling is utilized in the manufacturing and service industries to identify consumer requirements for a product. This methodology is based on stratifying consumer requirements of a service or product into categories of mandatory, performance, attractive, indifferent or reverse.[5] A mandatory service is one that consumers expect to be present at all times, and its absence can lead to intense dissatisfaction, while presence does not lead to any additional satisfaction. In the performance category, satisfaction and dissatisfaction are based on incremental availability, thus the more a service is offered, the more satisfied the consumer. In the attractive category, there are services which a consumer does not expect, so absence does not lead to dissatisfaction while presence can lead to intense satisfaction. The indifferent category refers to services that are neither good nor bad, and do not effect satisfaction or dissatisfaction. Finally, the reverse category refers to a service where a high level of service produces dissatisfaction for some consumers, reflecting variable expectations[6] [Figure 1] adapted from.[7]

Figure 1

Kano categories[7]

To utilize Kano categories, a survey must ask each question in a positive and negative format. Specific methodologies including both discrete and continuous analysis can then be used to identify the consumer requirements.[7] Kano methodology has recently been used to categorize healthcare processes.[891011] For healthcare, patients are the main customers who “receive” and “feel” the health services,[12] identification of healthcare customer requirements would be key to improve their satisfaction.

To ascertain these requirements, we conducted a pilot Kano analysis of the HCAHPS survey. The objective of this research was to generate group-level summary statistics to be able to categorize HCAHPS questions into Kano categories, allowing differentiation and prioritization of components of the patient experience based on the patient's survey responses.

Materials and Methods

Study design and patient recruitment

Study design was survey research. It was conducted from 06/2019 to 08/2019 in an academic hospital (OSF Saint Francis Medical Centre) in a Midwestern United States city (Peoria/Illinois). Target population was hospitalized adult patients or caregivers of hospitalized patients (adult or pediatric). We utilized the adaptation of Kano methodology described by Blauth et al.[13] to construct our patient questionnaire. The questionnaire was pretested with three potential participants for length, flow, salience, ease of administration, response, and acceptability to respondents. Survey questions were adapted based on the feedback. The survey protocol and the Kano questionnaire and analysis matrix [Figure 2] was reviewed by the local Institutional Review Board (IRB) and approved (IRB # 14266434-4). Written informed consent was waived by the IRB, and agreement to complete the survey was considered an implied consent to participate.

Figure 2

Kano question and analysis matrix

Patients and caregivers were recruited for the survey by the co-investigator (IK for adult hospital and CR for parents of children admitted in the Children's Hospital) utilizing a convenient random sampling. Patients were eligible if they were greater than or equal to 18 years of age, admitted in the hospital for more than two midnights and deemed able to complete the survey by nursing staff. If the patient was not able to complete the survey due to age or other factors, then caregivers were approached. Survey respondents were given the option of completing the survey while the investigators waited, or to return the survey after completion to the nurses’ station. The number of people who were approached was not maintained, so response rate cannot be calculated. Survey respondent privacy was ensured by not requesting any form of identifying information in the survey. Only baseline demographics that is collected as part of HCAHPS survey was included in the Kano questionnaire.

Kano analysis

The survey was completed in a paper format, and results of each survey were transcribed to a Kano analysis template. For ease of description we have abbreviated the HCAHPS questions in this manuscript [Supplemental Digital Content Table A], and the abbreviated form is utilized throughout the manuscript. We utilized a Kano analysis matrix described by Pouliot for the discrete analysis.[14] All the information was then entered into a password protected Excel sheet available to research personnel only.

Supplemental Digital Content Table A

Hospital Consumer Assessment of Healthcare Providers and Systems questions and their abbreviated form used in this manuscript

HCAHPS category	HCAHPS question	Abbreviation in this manuscript
Your care from nurses	During this hospital stay, how often did nurses treat you with courtesy and respect?	Courtesy and respect (nurses)
	During this hospital stay, how often did nurses listen carefully to you?	Listen carefully (nurses)
	During this hospital stay, how often did nurses explain things in a way you could understand?	Explain things (nurses)
	During this hospital stay, after you press the call button, how often did you get help as soon as you wanted it?	Help when pressing call button
Your care from doctors	During this hospital stay, how often did doctors treat you with courtesy and respect?	Courtesy and respect (doctors)
	During this hospital stay, how often did doctors listen carefully to you?	Listen carefully (doctors)
	During this hospital stay, how often did doctors explain things in a way you could understand?	Explain things (doctors)
The hospital environment	During this hospital stay, how often were your room and bathroom kept clean?	Bathroom and room clean
	During this hospital stay, how often was the area around your room quite at night?	Quiet at night
Your experiences in this hospital	How often did you get help in getting to the bathroom or in using a bedpan as soon as you wanted?	Help in going to bathroom
	Before giving you any new medicine, how often did the hospital staff tell you what the medicine was for?	Told what the medicine was for
	Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand?	Side-effect of medicine explained
When you left the hospital	During this hospital stay, did Doctors, nurses or other hospital staff talk with you about whether you would have the help you needed when you left the hospital?	Enquiry about help you may need when you leave the hospital
	During this hospital stay, did you get information in writing about what symptoms or health problem to look out for after you left the hospital?	Information in writing about symptoms/health problem to look out for
Understanding your care when you left the hospital	During this hospital stay, staff took my preference and those of my family or caregivers into account in deciding what my healthcare needs would be when I left	Staff took preference into account at discharge
	When I left the hospital, I had a good understanding of the things, I was responsible for in managing my health	Understanding of the things responsible for/managing health at discharge
	When I left the hospital, I clearly understood the purpose of taking each of my medications	Understanding the purpose of taking medications at time of discharge

HCAHPS=Hospital Consumer Assessment of Healthcare Providers and Systems

After analyzing each positive and negative question on the Kano matrix, responses were classified as mandatory, attractive, performance, indifferent, reverse, or questionable for each of the HCAHPS questions. Responses for all the patients were then entered into an excel sheet, with patients as rows and the 17 HCAHPS questions as columns. This allowed us to calculate the most common categorization in a column (mode). The mode was designated as the categorization of the whole survey group for that question. The number of performance, attractive, indifferent, reverse, and questionable responses for each question was calculated. The proportion of mandatory categorization for each question was also calculated. We further calculated the satisfaction coefficient and dis-satisfaction coefficient for each question based on the formula described by Timko:[15] Satisfaction coefficient = (Attractive + Performance)/(Attractive + Performance + Mandatory + In different); Dissatisfaction coefficient = (Performance + Mandatory)/(Attractive + Performance + Mandatory + Indifferent). Satisfaction and dissatisfaction coefficient implies the percentage satisfaction or dissatisfaction in a customer with the presence or absence of a service.

Statistical analysis

Standard descriptive and comparative analysis was performed. Values are provided as number and percentages as applicable. Chi-square test was used to compare categorical variables. Patient demographic information was analyzed for trends within different Kano categories. Because of lower numbers in different categories of educational level and race this analysis was only performed for age (<50 or more than 50 years and for respondent type (patient versus caregiver)). A total of all mandatory and nonmandatory responses for the two categories was also charted on a 2 × 2 table and compared using Chi-square test. All statistical analysis was performed on JMP Pro V14.2.0 (SAS Institute Inc., Cary, NC, USA).

Results

Demographics

A total of 64 Kano surveys were completed during the study (39 patients and 25 caregivers). Among the total survey participants, 27 (42%) were <50 years of age, and five respondents were >75 years of age. There were no respondents who had not completed at least high school level of education with 14 (21.8%) 4-year college-level education and 9 (14%) with >4-year college-level education. 85.9% of all survey participants identified themselves as Caucasian, 9.3% as African American and 4.6% as Hispanic [Table 1].

Table 1

Demographics of the survey respondents

Category	Subcategory	n (%)
Type	Patient	39 (60.9)
	Caregivers	25 (39.0)
Age	<25	2 (3.1)
	25-50	25 (39.0)
	51-75	32 (50.0)
	>75	5 (7.8)
Education	High school	20 (31.2)
	Some college	21 (32.8)
	Four year college	14 (21.8)
	>4 years college	9 (14.0)
Race	White	55 (85.9)
	African American	6 (9.3)
	Hispanic	3 (4.6)

Kano categorization

All of the 17 HCAHPS questions except “quiet at night” were classified as mandatory. Among all the questions, “told what the medicine was for” had the highest number of mandatory responses (78.1%) followed by “side-effect of medicines explained” (75.0%). The lowest proportion of mandatory responses were for “quiet at night” (21.9%) and “help when pressing call button” (31.3%). There was little variability in the satisfaction coefficient for various questions with the highest satisfaction coefficient (66.5%) for “told what the medicine was for” and the lowest satisfaction coefficient (56.7%) for “inquiry about help you may need when you leave the hospital”. The dissatisfaction coefficient, however, had a much broader range with the highest dissatisfaction coefficient of 94% for “told what the medicine was for” followed by 80.3% dissatisfaction coefficient for “courtesy and respect (nurse).” The lowest dissatisfaction coefficient was 37.5% for “quiet at night” [Table 2].

Table 2

Categorical analysis of customer requirements from healthcare delivery

Category	M	P	A	I	R	Q	Cat	% M	SC	DC
Nurse
Courtesy and respect	45	9	4	6			M	70.3	66.3	80.3
Listen carefully	36	10	8	7	2	1	M	56.3	62.3	67.5
Explain things	28	12	12	8	2	2	M	43.8	61.0	62.3
Help when pressing call button	20	7	19	13	1	4	M	31.3	59.4	48.1
Doctor
Courtesy and respect	41	8	4	9	1	1	M	64.1	64.0	76.3
Listen carefully	41	8	6	9			M	64.1	65.3	72.8
Explain things	41	7	8	3	1	4	M	64.1	59.9	63.1
Environment
Bathroom and room clean	35	10	8	9		2	M	54.7	63.3	68.4
Quiet at night	14	4	30	15		1	A	21.9	63.1	37.5
Experience
Help in going to bathroom	37	11	9	6		1	M	57.8	64.2	69.1
Told what the medicine was for	50	9	2	1	1	1	M	78.1	66.5	94.0
Side-effect of medicines explained	48	6	6	2		2	M	75.0	63.0	70.0
Information on discharge
Enquiry about help you may need when you leave the hospital	40	4	6	6		8	M	62.5	56.7	60.7
Information in writing about symptoms/health problem to look out for	45	4	8	2		5	M	70.3	59.5	60.6
Understanding your care when you leave the hospital
Staff took preference into account discharge	30	9	11	9	1	2	M	48.4	59.8	59.7
Understanding of the things responsible for/managing health at discharge	42	6	5	7	1	2	M	66.7	61.2	69.4
Understanding the purpose of taking medication a time of discharge	45	7	4	4	1	2	M	71.4	61.8	74.3

M=Mandatory, P=Performance, A=Attractive, I=Indifferent, R=Reverse, Q=Questionable, Cat=Category, SC=Satisfaction coefficient, DC=Dis satisfaction coefficient

Out of possible 1088 possible responses (17 questions × 64 surveys), four responses were missing/”questionable.” Among the valid 1084 responses on the Kano analysis, 638 were categorized as mandatory (58.8%). There was no difference in the proportion of total mandatory versus nonmandatory categories by patient type (patient/caregiver) or patient's age (less than/more than 50 years) [Table 3].

Table 3

Number of respondents selecting mandatory versus non mandatory for questions on the kano survey

Category	N	Mandatory (%)	Non mandatory (%)	Total	P
Type
Patient	39	396 (60.0)	263 (40.0)	659	0.30
Care giver	25	242 (57.0)	183 (43.0)	425
Age
<50	27	254 (55.7)	202 (44.3)	456	0.07
>50	37	384 (61.1)	244 (38.9)	628

Demographic differences in patient preferences

There was no difference in the proportion of respondents who categorized the various questions into mandatory category based on age less than or more than 50 except for “help in going to bathroom,” which was considered as a mandatory requirement by 70.2% of respondents more than 50 years of age as compared to only 40.7% of respondents <50 years of age (P = 0.01). Similarly, there was no significant difference in the proportion of respondents who selected mandatory for the various questions between current patients and caregivers except for “explain things (nurse)” which a higher proportion of caregivers (64%) considered as a mandatory requirement compared to 51.2% of current patients (P = 0.03). The other question which was approaching statistical significance was “bathroom and room clean” which was considered mandatory by 64.1% of current patients compared to 40% of caregivers (P = 0.05) [Table 4].

Table 4

Percentage of respondents with mandatory categorization by category

Category	Age			Type
	<50 Years (n=27), n (%)	>50 Years (n=37), n (%)	P	Inpatient (n=39), n (%)	Caregiver (n=25), n (%)	P
Nurse
Courtesy and respect	21 (77.7)	24 (64.8)	0.25	26 (66.6)	19 (76.0)	0.42
Listen carefully	16 (59.2)	20 (54.0)	0.67	20 (51.2)	16 (64.0)	0.31
Explain things	14 (51.8)	14 (37.8)	0.26	13 (33.3)	15 (60.0)	0.03
Help when pressing call button	6 (22.2)	14 (37.8)	0.18	15 (38.4)	5 (20.0)	0.12
Doctor
Courtesy and respect	18 (66.6)	23 (62.1)	0.71	24 (61.5)	17 (68.0)	0.59
Listen carefully	17 (62.9)	24 (64.8)	0.87	24 (61.5)	17 (68.0)	0.59
Explain things	17 (62.9)	24 (64.8)	0.87	24 (61.5)	17 (68.0)	0.59
Environment
Bathroom and room clean	12 (44.4)	23 (62.1)	0.15	25 (64.1)	10 (40.0)	0.05
Quiet at night	4 (14.8)	10 (27.0)	0.24	11 (28.2)	3 (12.0)	0.12
Experience
Help in going to bathroom	11 (40.7)	26 (70.2)	0.01	26 (66.6)	11 (44.0)	0.07
Told what the medicine was for	21 (77.7)	29 (78.3)	0.95	30 (76.9)	20 (80.0)	0.77
Side-effect of medicines explained	18 (66.6)	30 (81.0)	0.18	31 (79.4)	17 (68.0)	0.30
Information on discharge
Enquiry about help you may need when you leave the hospital	17 (62.9)	23 (62.1)	0.94	23 (58.9)	17 (68.0)	0.46
Information in writing about symptoms/health problem to look out for	16 (59.2)	29 (78.3)	0.09	30 (76.9)	15 (60.0)	0.14
Understanding your care when you leave the hospital
Staff took preference into account discharge	12 (46.1) (n=26)	18 (50) (n=36)	0.76	19 (51.3) (n=37)	11 (44.0)	0.56
Understanding of the things responsible for/managing health at discharge	16 (61.5) (n=26)	26 (70.2) (n=36)	0.46	26 (68.4) (n=38)	16 (64.0)	0.71
Understanding the purpose of taking medication a time of discharge	18 (69.2) (n=26)	27 (72.9) (n=36)	0.74	29 (76.3) (n=38)	16 (64.0)	0.28

Discussion

In this pilot survey, we have categorized the various HCAHPS questions into Kano categories. This is the first such reported analysis on broad healthcare delivery in the United States. Our analysis showed all the questions as mandatory except for “quiet at night” with maximum mandatory responses for indications of medicines. This could be interpreted as patients having a stronger desire to know the details of the medications they are taking more than they expect the environment to be quiet at night. Not informing the patients about the indications for the medicine they are taking, would lead to a 94% decrease in the satisfaction. Importance of education on medication side effect and usage has been shown in prior studies also to have a large impact on patient satisfaction.[16]

Based on Kano analysis, patients expect doctors to listen and explain their medical conditions. A prior qualitative study on customer requirements from healthcare also showed intense dissatisfaction when healthcare providers don’t listen and answer questions appropriately.[17] Patients expect nurses to be more respectful and polite (70.3%) than they expect their doctors to be (64.1%), and are more likely to be intensely dissatisfied if that is not the case. This may reflect an underlying long held bias regarding “proper” physician and nursing behavior, or may reflect random chance in the convenience sampling being of a relatively small sample size.

Overall we have shown that a substantial proportion of patients in a tertiary care academic/community hospital in the Midwest United States consider all the questions in the HCAHPS Survey as mandatory requirement from healthcare. For healthcare industry leaders it may mean that consumers are not apt to be more satisfied if these requirements are met. However, it is very feasible to make the consumers extraordinarily dissatisfied if any of these requirements are not met. Similarly, it is also not possible to make the consumers happier by providing more of the mandatory requirements. Different patients may give priority to different aspects of the HCAHPS survey, and hospitals can prioritize improvement efforts based on specific services and demographics. For example, Patients >50 years are more likely to demand “help in going to bathroom” as compared to younger patients. Geriatric and nursing home facilities may prioritize this requirement to enhance patient satisfaction.

Because of the competitive environment of the healthcare industry, a high degree of importance is placed on achieving “top box” scores in patient satisfaction domains on the HCAHS survey.[18] The correlation of these scores with patient outcomes is mixed, with some studies showing strong correlation, while others show none to minimal.[192021] Much has changed since 2005, when the HCAHPS survey was developed as the society and patient expectations change rapidly, and our results suggest a need to update the HCAHPS survey to better reflect current patient expectations. Our study suggests a trend in healthcare consumers similar to that in other industries, namely what was once an attractive quality, for example the ability to take pictures from a phone, has now become a mandatory requirement. As the health care industry strives to enhance patient satisfaction results while improving patient care outcomes, a deeper understanding and interpretation of the voice of the patient may be the most highly productive place to begin.

To the best of our knowledge, this is the first report on Kano analysis of the global US customer requirements from healthcare. Our study however is limited with its small sample size and geographic limitation. The US population is much more diverse in education, race and socioeconomic spectrum distribution compared to our convenience sample population and our results may not be reflective of US population as a whole. Due to absence of any valid method to conduct sample size estimation for Kano analysis, no a priori sample size estimation was performed. However, we designed this study as a pilot to be hypothesis-generating, with further work needed to apply the Kano methodology to a broader population before more firm conclusions can be drawn. In addition, each Kano survey can take more than 30 min to complete, and a large multi-center study would require appropriate funding to complete.

Conclusion

Current U. S healthcare consumers have high expectations from healthcare delivery and consider most HCAHPS questions as mandatory requirements. Kano analysis needs to be done on a larger, more diverse hospital setting and potentially the HCAHPS survey needs to be modified to reflect prevailing healthcare customer requirements. Results of a large multicentre study could have a large impact on healthcare delivery, reimbursement, and patient experience.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.