Does Coaching Hospitalists in a Community Hospital Improve the Measured Experience of Care?

Hospitals initiate physician communication training programs expecting to improve patient experience measures. However, most efforts have relied on methods with limited attention to bedside physician-patient interactions. We conducted an intensive in-person hospitalist coaching program to improve patient experience in a community hospital. Full-time hospitalists were coached twice monthly by physician-coaches using a structured process featuring direct observation of care and immediate recommendations. Coach-observed care measures improved marginally. Difference-in-differences analysis of 1137 Hospital Consumer Assessment of Healthcare Providers and Systems surveys revealed no significant improvements by trained hospitalists in preintervention versus intervention comparisons, calling into question the strategy of using coaching programs to improve hospitals' doctor communication measures.

Introduction

As patient experience has become an integral component to the Hospital Value-Based Purchasing program, hospitals have attempted to improve doctor communication domain scores on the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey by implementing training programs designed to enhance interactions between physicians and patients. Training methods usually involve some combination of lecture, simulations, coaching, rounding, reminders, and recognition (1 –5).

The results of these efforts have been mixed. Some have failed, while others have demonstrated higher doctor communication scores after the training. Improvements have been modest or limited to 1 or 2 survey questions (6 –9). Furthermore, most studies that assess training effectiveness have taken place in academic medical centers with research capabilities that enable formal evaluations to be conducted. There is scant evidence of formal evaluations of HCAHPS-inspired doctor communication training in community hospital settings. Additionally, prior research has focused on training aimed at internists, residents, and specialists whose HCAHPS surveys could be tracked by unit within the hospital (10). To our knowledge, no published studies have addressed doctor communication training specifically targeted to hospitalists in community hospitals, who have emerged as important providers of inpatient care. Although anecdotal evidence suggests that hospitalists’ HCAHPS scores fall below those of other attending physicians, no formal studies have examined differences between HCAHPS ratings of hospitalists and other admitting specialists (11,12).

In this article, we report a quantitative assessment of an intensive hospitalist training program designed to improve a community hospital’s scores on the HCAHPS doctor communication survey section (questions 5-7). Our initiative relied on multiple sessions of in-person coaching at the bedside with immediate feedback. The HCAHPS surveys attributed to these hospitalists were analyzed before and after the training and were compared to surveys attributed to physicians in multiple specialties who did not undergo the training.

Method

The study was conducted in a 438-bed community hospital that experienced lower than average HCAHPS scores over several quarters in 2017 to 2018, and ratings on the doctor communication domain emerged as a concern. In 2019, hospital management and physician leadership initiated a program in which hospitalists received coaching and feedback in joint rounding sessions by 2 consultants, both highly experienced internal medicine physicians, using a prescribed instructional method. After an in-hospital pilot test, the program began in November 2019 and ended in February 2020. The program concluded earlier than planned because the Covid-19 pandemic disrupted normal hospital operations.

Twenty employed hospitalists were offered 2 coaching sessions each month. Although not mandatory, the program was strongly recommended by physician leadership, and 15 hospitalists participated in the training during the intervention period.

Prior to the intervention, 24 measured elements of the coaching were determined based on literature review and past experience (13). In the training program, both coaches accompanied each hospitalist to the bedside and observed the actual patient encounter, noting everything from hand hygiene to the physical examination with special attention paid to clear communication as well as sitting versus standing at the bedside. After each coaching session, immediate feedback was provided and the results were recorded in a database. Interobserver bias was controlled for by comparing aggregate measurements and reconciling scoring for each coach. Encounter-focused scores among hospitalists participating in the program improved marginally from preintervention to post-intervention periods (summary provided in online Appendix, Tables A-1 and A-2).

We conducted a retrospective analysis of 1307 HCAHPS surveys that assessed inpatient experience from June 2019 through October 2019 (preintervention period) and November 2019 through February 2020 (intervention). One hundred seventy surveys were excluded because of missing data, resulting in a final sample of 1137 patients; no significant demographic differences were observed for patients with missing data. Surveys were conducted by an approved external vendor within the hospital’s usual HCAHPS data collection periods, following established HCAHPS protocols. The primary outcome measures were “top box” scores on the HCAHPS questions from the doctor communications domain as well as 2 global questions, overall hospital rating and recommend the hospital. Analyses of top box scores are recommended over arithmetic means from continuous measures because a high proportion of respondents give favorable ratings, skewing the distributions (4,14).

The HCAHPS scores were attributed to the provider of record at time of admission, and we used this attribution to link physicians to patients’ HCAHPS surveys for both preintervention and intervention periods. As noted in prior research, the physician identified as the provider of record for HCAHPS may not be entirely responsible for the care of the patient during their stay, and this represents a limitation of the study data (10). Since hospitalists are not coded as a specialty in the HCAHPS survey and reporting process, we identified hospitalists using the hospital’s internal provider job classification and attributed HCAHPS surveys to each physician using their National Provider Identifier number. The HCAHPS surveys attributed to nonhospitalist physicians formed a control cohort (“specialists”) who did not receive the training protocol. All analyses were performed on raw HCAHPS data supplied by the survey vendor, unadjusted for patient mix, survey mode, or hospital characteristics.

We used χ2 and t statistics to test for differences between the trained hospitalist and specialist cohorts and between preintervention and intervention phases. A difference-in-differences analysis was performed using multivariate logistic regression on the 2 global HCAHPS measures and 3 doctor communication domain scores for the hospitalist (intervention) and specialist (control) samples. Five hospitalists who declined the training were not included in the analysis. Covariates included patient characteristics, self-rated overall and mental health, admission from emergency department (ED), and discharge to home. Stata version 13 and SPSS version 25 were used for statistical analysis.

Results

The trained hospitalists showed modest improvements on key measures of patient interaction at the bedside (ie, seated, reassurance, use of communication whiteboard), based on trainers’ assessments (see Table A-1 in online Supplemental Appendix).

Patient characteristics from HCAHPS surveys are presented in Table 1. Self-rated health and mental health were significantly lower among patients attributed to hospitalists versus specialists. In the preintervention period, education level of patients attributed to specialists was higher than hospitalists, and in both periods, patients attributed to hospitalists were less likely to be discharged to home. Additionally, the proportion of patients admitted from the ED was significantly higher for hospitalists than for specialists. No differences were found for race or ethnicity, and these covariates were excluded from the multivariate analysis.

Table 1.

Summary Patient Characteristics From HCAHPS Surveys Attributed to Hospitalists and Specialists.

	Preintervention period (July 1, 2019 to October 31, 2019)		Intervention period (November 1, 2019 to February 29, 2020)
Measures	Trained hospitalists, N = 195	Specialists (control), N = 402	Trained hospitalists, N = 180	Specialists (control), N = 360
Self-rated overall health, mean (SD)a	2.92 (1.00)c	3.53 (1.01)c	2.95 (0.99)c	3.45 (1.01)c
Self-rated mental/emotional health, mean (SD)a	3.63 (1.03)c	3.93 (1.01)c	3.69 (1.03)c	3.92 (0.94)c
At least some college education (%)	56.9c	68.4c	66.1	61.9
Spanish, Hispanic, or Latino (%)	14.9	14.4	12.8	11.4
Admitted through emergency department (%)	83.6c	38.6c	87.8c	42.5c
White race (%)	87.2	85.6	88.9	89.4
Discharged to home (%)	81.0b	87.6b	81.7c	90.3c

Abbreviation: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems.

a Scale: 5 = excellent, 1 = poor.

b Difference between specialists (control) and hospitalists is significant at P < .05.

c Difference between specialists (control) and hospitalists is significant at P < .01.

In the difference-in-differences regression analysis that included covariates, adjusted top box scores on both doctor communications ratings and global HCAHPS measures were not significantly different between the preintervention and post-intervention periods (Table 2). However, in both periods, the trained hospitalists had significantly lower HCAHPS scores than specialists after adjusting for overall health status, mental health status, discharge to home, and ED admission. The significantly higher proportion of patients admitted from the ED by hospitalists may have affected these results, since over 80% of HCAHPS surveys attributed to trained hospitalists were completed by patients admitted through the ED. In both preintervention and intervention periods, patients admitted through the ED gave significantly lower HCAHPS scores than patients admitted directly, with top box percentages on both doctor communication ratings and global HCAHPS measures differing by almost 10 to 15 percentage points between the 2 groups (see Appendix Table A-2).

Table 2.

Predicted Probabilities of Top Box HCAHPS Scores From Difference-In-Differences Logistic Regression.

HCAHPS measuresa,b	Trained hospitalists	Specialists (control)	Difference in differences (P value)
Doctor communication
Preintervention	49.2% (47.0%-51.3%)	71.3% (69.9%-72.7%)
Intervention	42.9% (40.7%-45.0%)	70.5% (69.1%-71.8%)
Pre–post difference	−6.30%	−0.80%	−5.5% (.354)
Doctors explain things you can understand
Preintervention	59.2% (56.9%-61.5%)	80.2% (78.9%-81.4%)
Intervention	56.0% (53.6%-58.3%)	79.0% (77.8%-80.2%)
Pre–post difference	−3.2%	−1.2%	−2.0% (.766)
Doctors listen carefully to you
Preintervention	60.3% (58.2%-62.4%)	81.0% (79.9%-82.0%)
Intervention	56.0% (53.7%-58.2%)	79.8% (78.8%-80.9%)
Pre–post difference	−4.3%	−1.2%	−3.1% (.619)
Doctors treat you with courtesy and respect
Preintervention	71.5% (70.0%-73.0%)	87.7% (87.0%-88.3%)
Intervention	69.6% (68.0%-71.3%)	87.8% (87.2%-88.4%)
Pre–post difference	−1.90%	0.10%	−2.0% (.741)
Recommend the hospital
Preintervention	47.5% (45.6%-49.4%)	69.7% (68.5%-70.9%)
Intervention	44.6% (42.6%-46.7%)	75.6% (74.5%-76.6%)
Pre–post difference	−2.90%	5.90%	−8.8% (.093)
Rate this hospital
Preintervention	48.0% (46.1%-50.0%)	67.6% (66.3%-68.8%)
Intervention	47.0% (45.0%-49.1%)	73.3% (72.2%-74.4%)
Pre–post difference	−1.00%	5.70%	−6.7% (.175)

Abbreviation: HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems.

a Outcome measures adjusted for emergency department admission, discharge to home, self-rated overall health, and self-rated mental health.

b Values in bracket denote 95% CI.

Discussion

Our analysis did not show significant improvement in patient experience by the coached hospitalists, nor between hospitalists and specialists, after a concentrated communication training program intended to enhance patient experience. The findings are consistent with other studies that have failed to show significant improvement in HCAHPS doctor communication among providers who participated in communication training programs. Because the training was more intensely focused on hospitalists in a community hospital setting, administrators were optimistic that the program could produce gains that have eluded earlier efforts, since hospitalists are more involved with daily inpatient care than specialists who spend less time in the hospital. However, hospitalists received lower HCAHPS scores than specialists, both before and after the training intervention. Notably, hospitalists were much more likely than specialists to be evaluated by patients admitted through the ED, who gave lower HCAHPS scores than patients admitted directly. Additionally, patients attributed to hospitalists rated their overall and mental health lower (ie, poorer) than patients attributed to specialists, while demographic profiles of hospitalists’ and specialists’ patients were not significantly different. These findings suggest the need for further research about the complex relationships among the variables that affect hospitalists’ service and subsequent HCAHPS ratings (15).

Limitations

Our study has several limitations. Attribution of HCAHPS surveys to individual physicians is imprecise, and patients who see multiple providers during their stay may complete their HCAHPS survey based on their experience with one doctor while their survey was attributed to another. Further, although the longitudinal data allowed comparisons between trained and untrained groups, the design was not experimental and caution must be used when interpreting the findings. Additionally, our sample size was limited due to the smaller community hospital setting and size of attending staff. Because of Covid-19, the training program ended earlier than planned; however, substantial training had occurred and sufficient data had been collected for our analyses to proceed.

Finally, during the intervention period, the hospital’s HCAHPS percentile scores improved broadly, prompting the hospital’s administrators to declare the program a success. The findings from this study call this conclusion into question, reminding us that a myriad of factors influences patient experience, and evaluation of more comprehensive hospital-wide initiatives is needed.

Click here for additional data file.

Supplemental Material, sj-docx-1-jpx-10.1177_2374373521996964 for Does Coaching Hospitalists in a Community Hospital Improve the Measured Experience of Care? by David Pratt, Aaron Wu and John W Huppertz in Journal of Patient Experience