A Qualitative Exploration to Understand Hospitalists' Attitude Toward the Patient Experience Scoring System.

Patient's perception of their inpatient experience is measured by the Center for Medical Services' (CMS) administered Hospital Consumer Assessment of Healthcare Providers & Systems (HCAHPS) survey. There is scant existing literature on physicians' perceptions toward the HCAHPS scoring system. Understanding hospitalist knowledge and attitude toward the HCAHPS survey can help guide efforts to impact HCAHPS survey scores by improving the patient's perception of their hospital experience. The goal of this study is to explore hospitalists' knowledge and perspective of the physician communication domain of the HCAHPS survey at an academic medical center. Seven hospitalists at an academic medical center were interviewed for this report using a semistructured interview. Thematic analysis approach was used to analyze data. Open, line-by-line coding was performed on all 7 transcripts. Categories were derived in an inductive fashion. Categories were refined using the techniques of constant comparison and axial coding. We generated themes reflecting hospitalists' knowledge of the HCAHPS scoring system, their perception of the HCAHPS scoring system and the impact of the HCAHPS scoring system on their practice. While hospitalists acknowledged physician-patient communication is a challenging area to study, they are unlikely to embrace the feedback provided by HCAHPS surveys. There is a need to deploy tactics that provide timely and actionable feedback to providers on their bedside communication skills.

Introduction

Effective provider–patient communication is a key component of providing high-quality patient care. For the purpose of this article, providers are defined as attending physicians. When carried out appropriately, communication with patients encourages compliance and improves clinical outcomes (1 –3). Given patients’ preference for shared decision-making, it is crucial that physicians continually focus on a bidirectional flow of information in order to build strong therapeutic alliances with their patients. This can be especially challenging in the inpatient set up where care is increasingly provided by physicians who do not have a preexisting therapeutic relationship with the patient. A review of existing literature suggests that the current state of patient experience with provider communication is suboptimal (4 –6). Hospitalized patients are often in the dark with respect to their medical team and management plan. Up to 75% (2110 of 2807) of patients are unable to name anyone when asked to identify an inpatient physician in charge of their care (4). In fact, 38% (87 of 229) of patients are unaware of tests planned on any given day, while only 45% (104 of 231) are in complete agreement with their physicians about their primary diagnosis (5). Even when patients know their providers and plan of care, they are often disgruntled with the quality of communication with their providers. Around 42% (37 of 89) of patients report their physicians do not explain things in a comprehensible way, while 54% (48 of 89) state their fears and anxieties were never addressed by physicians in the hospital (6).

Lack of time at the bedside is a potential explanation for this disconnect. According to one study, inpatient providers spend 4 minutes with a patient and 20 seconds with a relative per day (7). Another study broke down a hospitalist’s day by tasks, and found that 69% of a physician’s time was spent in indirect patient care activities while only 18% was dedicated to direct patient care (8). In addition to time constraints, lack of ongoing training and feedback is another possible reason for attrition in communication skills over a period of time (9). While some studies have shown a sustained improvement (up to 2.5 years) in communication skills with discrete interventions such as communication skills training workshops (10,11), others show a regression to baseline as soon as 6 months after the intervention (12).

Ongoing communication between inpatient providers and hospitalized patients is an important measure of patient experience scores. Patient satisfaction with their inpatient stay is one of the drivers of value-based purchasing (VBP) as introduced by Center for Medical Services (CMS) in the United States. Value-based purchasing was introduced in 2010 as part of the Affordable Care Act to improve the quality of care received by Medicare beneficiaries (13). For the purpose of VBP, patient experience is determined by the CMS administered Hospital Consumer Assessment of Healthcare Providers & Systems (HCAHPS) survey. Hospitals that score below the 50th percentile of performance when compared to all hospitals may receive a lower Diagnosis Related Group-based payment from CMS, whereas hospitals that score above the 50th percentile have the potential to receive incentives. Questions that address physician communication on the HCAHPS survey are presented in Table 1.

Table 1.

Physician communication questions on the HCAHPS survey.

Question	Response options
During this hospital stay, how often did doctors treat you with courtesy and respect?	Never, Sometimes, Usually, Always
During this hospital stay, how often did doctors listen carefully to you?	Never, Sometimes, Usually, Always
During this hospital stay, how often did doctors explain things in a way you could understand?	Never, Sometimes, Usually, Always

Abbreviation: HCAHPS, Hospital Consumer Assessment of Healthcare Providers & Systems.

Value-based purchasing, in part, drove researchers to attempt interventions to improve HCAHPS survey scores, mainly by offering bedside communication tools or communication skills training (14,15). Any improvement in communication skills as a result of such training is unlikely to endure unless preexisting attitudes and beliefs that are in conflict with the behaviors promoted during training are addressed (10,16). Scant literature exists on physicians’ attitudes and beliefs toward the HCAHPS survey. One indirect study reported only 15% of chief experience officers at 143 health-care institutions felt that their physicians were supportive of efforts to improve the patient experience (17). Without providers’ active collaboration to improve the physician communication scores on the HCAHPS survey, any improvement effort is unlikely to last for long. Hospitalists are providers who take care of patients in the inpatient setting. Understanding hospitalist knowledge and attitude toward the HCAHPS survey can help guide efforts to impact HCAHPS survey scores. Accordingly, the purpose of this study is to explore hospitalists’ knowledge and perspective of the HCAHPS scoring system.

Methods

Study Design

From a constructivist point of view, we conducted a qualitative research study to explore how hospitalists construct the current patient experience scoring system by conducting semistructured interviews. In the constructivist framework, all knowledge is created by the learner and influenced by context (learner’s prior experiences, attitudes, and beliefs) and social interaction (learner’s peers and environment) (18). This theoretical framework is applicable when researchers are interested in (1) how people interpret their experiences, (2) how they construct their worlds, and (3) what meaning they attribute to their experiences (19).

We interviewed a total of 7 hospitalists between March and December of 2016 at a 550-bed tertiary university-based hospital in Milwaukee, Wisconsin. There are 14 inpatient medicine teams that admit patients to the general internal medicine floors. On average, 11 to 12 of 14 teams are staffed by 26 hospitalists. The remaining teams are staffed by primary care physicians or specialists. We interviewed hospitalists as they are the primary providers for most of the patients admitted to the general internal floors. Since hospitalists staff greater than 80% of hospitals with more than 200 beds in the United States (20), our study sample is broadly reflective of inpatient providers in large hospitals (21).

Interviews were audio-recorded by the primary author. Interviewer took field notes making a note of nonverbal cues. Participants were not identified by name during the interview or in field notes. Interviews lasted an average of 26 minutes. The secondary authors only had access to deidentified transcripts of the interview to preserve confidentiality and maintain anonymity. Informed consent was obtained from all participants. All participants were aware they could terminate the interview at any point. The interviewees were told that the purpose of the study was to explicate their world view on the HCAHPS survey. Hospitalists were interviewed in their individual offices which offered a quiet environment.

Study Participants

A convenience sample of 7 hospitalists were interviewed for this report. Informed consent was obtained from all participants. Institutional informant characteristics are displayed in Table 2. Three of seven interviewees had nonclinical or administrative roles. One was the medical director of the section of hospital medicine, while the remaining 2 ran medical student courses. The medical director practices clinical medicine at the bedside for 70% of their professional effort, while 1 of the 2 medical student course directors is 85% clinical. The second medical student course director is 100% clinical. The medical director receives monthly reports on the HCAHPS scores for the hospitalist group. They also have access to HCAHPS scores of individual hospitalists on an as needed basis.

Table 2.

Characteristics of survey respondents.

Characteristic	Description
Gender	3 females, 4 males
Average hospitalist experience	7.2 years (range 4-10 years)
Medical school background	4 international medical graduates, 3 American medical graduates

Description of Data

The primary author designed a semistructured interview and asked questions about hospitalists’ knowledge and attitudes toward the physician communication domain of the HCAHPS survey. The individual in-depth interview format was used to help recreate participants’ perceptions and attitudes related to the HCAHPS survey (22). We asked 3 open-ended questions and then diverged to pursue an idea or response in more detail (23). This was done by using nonleading prompts by repeating the words used by the interviewee (22) in order to avoid biasing the interview by introducing the interviewer’s perspective. The open-ended questions helped get the interviewee talking and ease the apprehension inherent in the interview process (24). We created 4 clarification questions, 3 to further explore the participants’ attitude toward the HCAHPs scoring system, and 1 to delve into any specific training participants’ may have received to improve their HCAHPS scores (22). Clarification questions were asked only if participants did not comment on them in response to the initial broad questions (25). For each question, participants were asked to confirm they did not have any residual thoughts on the question before moving on to the next question. This report draws on participant responses to these questions. The 3 broad questions and 4 specific questions are listed below:

1. What do you know about the HCAHPS scoring system?

2. What do you think about the HCAHPS scoring system?

Do you think HCAHPS scoring system is fair?

Do you think the HCAHPS scoring system is helpful?

Do you think the HCAHPS scoring system is effective?

3. Has value-based purchasing, particularly HCAHPS, had any impact on the way you practice?

Have you received any other training to improve your patient satisfaction scores on HCAHPS?

Data Analysis

Thematic analysis approach was used to analyze data. The primary author transcribed all 7 interviews verbatim. The quality of audio-recordings was excellent. Data management and analysis were performed using Microsoft word. Open, line-by-line coding was independently performed on all 7 transcripts by the primary author and one of the secondary authors in order to improve the reliability of our findings. Categories were derived in an inductive fashion. After deriving categories from the first interview, responses from each subsequent interview were used to either supplement or modify an existing category or create a brand-new category. Particular attention was accorded to contradictory views by including them even if they were expressed by a single participant. Any examples cited by participants to support their views similarly received close examination. Categories were refined in the next step, using the techniques of constant comparison and axial coding. Categories with similar themes were solidified as one. Categories that were thought to explain or enrich a different theme rather than represent their own theme were subsumed into that category as subthemes. Categories were reviewed to ensure that all the manifestations of each theme had been considered and compared.23 The primary author kept track of the number of hospitalists who expressed views in support of a particular theme. Most themes listed under the results section are qualified by “most” or “some.” “Most” indicated a theme that was validated by 5 or more, and “some” indicated a theme expressed by 3 or 4 hospitalists, respectively. When an opinion was endorsed by “one” or “all” hospitalists, this is indicated as well. Ideas expressed by a single participant are reported when they explicate a contrary or unique point of view. Any conflict was resolved by discussion and mutual agreement. This report describes the various themes that were generated. Relationship between themes (23) as predicated by informant characteristics (administrative versus nonadministrative) are described as well.

Results

1. What do you know about the HCAHPS system? The following themes were elucidated when hospitalists were asked of their knowledge of the HCAHPS scoring system:

Survey-based system: All hospitalists expressed knowledge of HCAHPS scoring system as a survey-based system. Some respondents further described it as a survey of patient perceptions of their inpatient stay

Financial implications: Most hospitalists expressed awareness of the negative financial impact of low HCAHPS scores on Medicare reimbursement. One hospitalist expressed awareness that HCAHPS scores are publicly reported: “They (hospitals) can either be penalized for poorer scores, or they can get incentives if their scores are higher than benchmarks that have been set up by Medicare” One hospitalist incorrectly identified negative personal financial implications of low HCAHPS scores “I know that it counts towards our personal incentive bonus package.” This is inaccurate as individual HCAHPS scores are not tied to financial incentives at our institution.

Multiple domains: Some hospitalists expressed understanding that the HCAHPS survey assesses patient perceptions across multiple domains related to their inpatient stay. This knowledge appeared to be limited to hospitalists with an administrative role. “I believe there is a total of 9 sections to HCAHPS and they ask about, like I said physician communication, nursing communication, whether the patients’ requests for pain medications were met, call lights answering…environmental questions are in there as well about cleanliness and quietness and things like that. And there is, I believe a question on discharge instructions, whether they got instructions that were appropriate for discharge, at the time of discharge as well.”

Top-down system: Most hospitalists perceive HCAHPS as a top-down, regulatory system: “We are attaching a negative, you know, reinforcement at higher levels, and that’s being passed down to physicians”

Quality improvement initiative: Two of 3 hospitalists with administrative roles consider the HCAHPS scoring system to be a part of a larger quality improvement initiative. Nonadministrative hospitalists did not articulate this view. “…not just in terms of morbidity and mortality metrics but also in terms of consumer items, such as communication, and whether their needs were met as a customer in the hospital setting”

2. What do you think about the HCAHPS scoring system?

Tackles a challenging area: Two of 3 hospitalists with administrative roles recognized that the HCAHPS scoring system addresses a challenging area since patient experience scores are not automatically captured by the electronic medical record. The collection of patient experience data was regarded to be a challenge: “Seems to be the best you could do in this setting” Nonadministrative hospitalists did not hold this view.

Validity concerns: Most hospitalists expressed concerns regarding the validity of physician–patient communication scores measured by the HCAHPS survey. These validity concerns can be grouped into the following subthemes:

Negative anchoring: Most hospitalists suspect one negative experience of significant consequence to the patient has the potential to drag down responses to all questions on the survey. Particular examples cited were food quality, pain control, and unrealistic expectations in terminal situations. “Patients might have one bad experience during the stay and just mark the entire survey down poorly” “They (patients) are going to have a bad impression of the hospital because they are going to say they can’t make decent food …(is my score low because I helped) get that patient to hospice when there was no chance of them surviving and the family had such a hard time with it. I remember I had one of those in the fall and it was hard…I think I did the best thing for the patient, tried my best to make the family happy, but I am not super sure they were, through the whole process”.

Lack of specificity: Most hospitalists felt that patients’ respond to all domains on the HCAHPS survey based upon their overall sense of how their stay went and do not adjudicate each domain on its own merit. Areas that impact a patient’s overall stay cited by participants include pain control, quietness of room, kind of floor patient is admitted to, and nursing care. “.…their pain control, quietness of the room, surroundings so…those some environmental factors would also pay a role based on where the patient is located or what type of nursing staff are taking care of these patients” One hospitalist expressed concerns regarding the wrong physician being evaluated since HCAHPS surveys are linked to the discharging physician who might have only cared for the patient for a very limited part of their stay. One hospitalist stated that HCAHPS scoring system was a comparative tool that helped physicians compare themselves to national and local trends. On the other hand, another hospitalist felt that the HCAHPS scoring system had no sound basis to be used as a tool to compare hospitalists with each other.

Poor efficacy: Most hospitalists stated they had doubts regarding the effectiveness of the HCAHPS scoring system. The following subcategories explicate the reasons cited:

Time lag: Some hospitalists noted that the 3-month turnaround time made it hard to reflect upon what was done at the time the survey was administered: “If it (HCAHPS scores) came in real time it would be far more helpful”

Lack of actionable information: Lack of remedial suggestions and absence of guidance on tools to improve the patient experience are considered as reasons behind the poor efficacy of the HCAHPS scoring system by most hospitalists: “There is no guidance for how to be a better physician, nothing there tells me. Did I not explain things correctly, did I actually end up denying somebody narcotics and they got really mad?”

Unfair system: Most hospitalists report HCAHPS scoring system to be reflective of overall care a patient receives and not an accurate evaluation of their hospitalist’s communication skills: “I don’t know if it’s fair to the practicing physician, especially if you tie it into their own financial incentives, or their own financial performance, or even psychologically, to hold them up to a score, without really explaining it to them, or giving them the tools to make it better”. One hospitalist stated that the HCAHPS scoring system was fair at the level of the hospital system: “There should be some kind of an accounting system that hospitals should be held accountable to”. One hospitalist stated the HCAHPS scoring system was unfair at the level of the hospital system: “You are paying the hospitals that might not be giving the best hospital care, than giving the hospital that might be doing a better job”.

Unintended consequences: Most hospitalists ascribed negative repercussions to the single-minded focus on improving HCAHPS scores. These include time being taken away from other patient care activities to focus on improving HCAHPS scores, and the perils of making HCAHPS scores a priority over sound medical care: “It’s tricky to want to get good scores, at the risk of offering pain medicines”

3. Has value-based purchasing, particularly HCAHPS, had any impact on the way you practice? No themes were generated for this question since responses were mostly homogenous. Most hospitalists reported HCAHPS survey has had no impact on their practice. “I don’t know if it stays with me on a day by day basis” “It leaves much to be desired, in terms of being translated to…how its perceived, by physicians. I don’t think they have done a great job in terms of disseminating what it actually means”. One hospitalist reported HCAHPS surveys have impacted their practice. Another hospitalist reported they found HCAHPS survey to be effective and helpful as it allows for “self-improvement by identifying areas of weakness.” They gave the example of sitting down and talking to the patient more so after the introduction of HCAHPS survey. All hospitalists reported undergoing Acknowledge, Introduce, Duration, Explanation and Thank you (AIDET) training (15) as the sole training they had received on improving HCAHPS scores. This training was mandatory for all hospitalists at our institution. Table 3 summarizes the themes elucidated in our study.

Table 3.

Summary of themes generated in various domains.

Knowledge about HCAHPS What do you know about the HCAHPS scoring system? Survey-based system Financial implications Multiple domains Top down system Quality improvement initiative

Perceptions toward HCAHPS What do you think about the HCAHPS scoring system? Tackles a challenging area Validity concerns Negative anchoring Lack of specificity Poor efficacy Time lag Lack of actionable information Unfair system Unintended consequences

Has HCAHPS had any impact on the way you practice? No

Abbreviation: HCAHPS, Hospital Consumer Assessment of Healthcare Providers & Systems.

Discussion

A review of the themes generated and relationships between themes in this study of 7 practicing hospitalists provides a window into hospitalist attitudes, perceptions, and beliefs with regard to provider communication scores on the HCAHPS survey. We found a much deeper and more nuanced understanding of the HCAHPS scoring system among hospitalists with an administrative responsibility as opposed to hospitalists with a purely clinical role, as indicated by themes 1C, 1E, and 2A. Only hospitalists with administrative roles saw the survey as an improvement initiative that addresses a key, underaddressed aspect of providing high-quality care in the inpatient setting. In addition, only hospitalists with an administrative background verbalized understanding that the HCAHPS survey covers many domains. Information dissemination on how the HCAHPS survey was derived and the way it is applied to all hospitalists might improve provider buy-in.

The concerns expressed by participants regarding the validity and efficacy of the HCAHPS survey, along with its perception as a top–down, regulatory system, are related to the HCAHPS survey being viewed as unhelpful, ineffective, and unfair. Since stakeholder engagement is essential to the success of a quality improvement initiative (26), educating hospitalists on the rigorous scientific processes that went into the design of the HCAHPS survey (27) might help temper their skepticism regarding the survey. Some elements that went into the design of the HCAHPS survey included cognitive interviews and focus groups with patients, focus groups, extensive psychometric testing, and pilot testing in 3 states (27). In addition, hospitalists should be educated on how the survey is administered. The knowledge that the survey is administered by CMS and cannot be altered by an individual hospital can improve their understanding and hence perception of the survey. The concerns raised under our “validity” theme highlight the concern hospitalists have regarding the ability of HCAHPS to measure the patient experience. While hospitalists would like that the survey go out to more patients with fewer but more directed questions regarding different domains of the inpatient experience, altering the HCAHPS survey and its administration is not feasible. Innovative approaches such as asking physician specific questions at the bedside and providing feedback to hospitalists (28) can help address validity concerns of “negative anchoring” and “lack of specificity” identified above. Another approach to addressing validity concerns is making sure patients know the name of their provider. Only 40% (281 of 697) of patients in the hospital are able to correctly identify their inpatient provider (4). Several studies have successfully evaluated techniques such as writing provider’s name on the whiteboard, sharing headshots of providers with patients and simply reminding the patient of their provider’s name in improving patient’s recall of their provider’s name (29 –32).

This study brings forth a need for tools that provide more ongoing and actionable feedback to hospitalists in order to address the “poor efficacy” theme that is characterized by “time lag” and “lack of actionable information.” Existing techniques such as scorecards, monthly feedback, emotional intelligence training, and communication skills training have yielded inconsistent results so far (14,33 -38). One promising approach seems to be gathering and communicating ongoing patient concerns to hospitalists on a daily basis (28). In one study (28), patient experience scores and patient comments on hospitalist communication skills were obtained by daily bedside surveys of patients. Hospitalists were given in-person feedback and coaching based on the information collected from patients. There was a substantial improvement in HCAHPS percentile ranks for physician-specific questions over 6 months. There is a pressing need to derive, validate, and disseminate more such innovative processes to improve patient experience in the inpatient setting. Ongoing feedback can also address the questionable sustainability of discrete communication skills training programs for providers (39). This area appears rife for further experimentation and study.

One of the areas of concern identified in 2 themes (“negative anchoring” and “unintended consequences”) is prescription of opioids. Appropriate denial of opioids has the potential to negatively impact HCAHPS scores, and financial and regulatory pressure on achieving high patient experience scores increases the risk of inappropriate prescription of opioids. In the face of the ongoing epidemic of opioid abuse in this country (40), it is reassuring that Medicare has made the decision to take patient’s perception of pain control out of the HCAHPS scoring system.

Limitations

Three of our 7 informants are physicians with administrative responsibilities. Since a majority of practicing hospitalists do not have administrative roles, our findings may have been further enriched by interviewing more frontline hospitalists. However, our sample does represent a roughly even split by gender and type of medical school training and both early and midcareer hospitalists (see Table 2). Such a range of perspectives helps reduce bias in qualitative studies (23). To further reduce bias, we have presented the views of a single participant when they differ substantially from the view of other participants (23). All the participants in this study have also participated in an as yet unpublished quantitative study on improving HCAHPS scores. This could have impacted their knowledge and perceptions of the HCAHPS scoring system. The reliability of our findings would have been further enhanced if we would have been able to interview more than 7 hospitalists and deploy more than 2 coders to code our data (23).

Conclusion

Our findings suggest that the HCAHPS survey system is poorly understood by hospitalists. Educating hospitalists on the methodology underlying the design of the survey, the questions that make up the survey and way it is delivered can help improve their understanding of the HCAHPS survey. In addition, hospitalists are concerned about validity and efficacy of provider communication scores reported by the HCAHPS survey and do not believe it impacts their practice. Creating processes that deliver real time, ongoing and actionable feedback to frontline provider may help address these concerns.