The Correlation of Communication Effectiveness and Patient Satisfaction.

This study assessed the correlation of 9 questions addressing communication effectiveness (the Communication Effectiveness Questionnaire [CEQ]) with other patient-reported experience measures (PREMs; satisfaction, perceived empathy) as well as patient-reported outcome measures (PROMs; pain intensity, activity tolerance) in patients with musculoskeletal illness or injury. In a cross-sectional study, 210 patients visiting an orthopedic surgeon completed the CEQ and measures of satisfaction with the visit, perceived empathy, pain intensity, and activity tolerance. We evaluated correlations between CEQ and other PREMs and CEQ and PROMs. We measured ceiling effects of the PREMs. Communication effectiveness correlated moderately with other PREMs such as satisfaction (ρ = 0.54; P < .001) and perceived empathy (ρ = 0.54; P < .001). Communication effectiveness did not correlate with PROMs: pain intensity (ρ = -0.01; P = .93) and activity tolerance (ρ = -0.05; P = .44). All of the experience measures have high ceiling effects: perceived empathy 37%, satisfaction 80%, and CEQ 46%. The observation of notable correlations of various PREMs, combined with their high ceiling effects, direct us to identify a likely common statistical construct (which we hypothesize as "relationship") accounting for variation in PREMs, and then develop a PREM which measures that construct in a manner that results in a Gaussian distribution of scores. At least within the limitations of current experience measures, there seems to be no association between illness (PROMs) and experience (PREMs).

Introduction

A genuine, trusting relationship between clinician and patient is associated with better adherence, a stronger placebo effect (which enhances both inert treatments and active treatments), and fewer disputes or lawsuits (1,2). The instruments used to measure the experience of care from the patient’s perspective—including measures of satisfaction with care, communication effectiveness, and perceived clinician empathy—seem to measure a single underlying construct and have strong ceiling effects (3). Orthopedic surgeons have the lowest average outpatient satisfaction rating among 28 specialties (4). To help specialists and their organizations learn and improve, we need patient-reported experience measures (PREMs) that address coachable, trainable factors, and we need to have enough spread in the values to allow for meaningful benchmarking and analysis (2,3,5,6).

The questions regarding communication effectiveness in the Clinician & Group Consumer Assessment of Healthcare Providers and Systems questionnaire and other questionnaires addressing communication effectiveness are subject to strong ceiling effects (7,8). And they are in some ways too general to lead to direct coaching and training of specific tactics (5,6). We reviewed existing measures of communication effectiveness and picked 9 items that were specific and coachable. We refer to these 9 items as the Communication Effectiveness Questionnaire (CEQ). Our purpose with this first study was to confirm correlation with other PREMs and see if the CEQ also has a substantial ceiling effect. Specifically, among patients with musculoskeletal illness or injury, we tested the null hypothesis that the CEQ does not correlate with other experience measures (satisfaction with the visit and perceived empathy). We also tested the correlation of the CEQ with patient-reported outcome measures (pain intensity and activity tolerance). Third, we assessed the number of factors the CEQ measures and if there was a subset of questions that maintained the construct validity and reliability of the full CEQ. Fourth, we assessed ceiling effects and internal consistency using Cronbach α. Fifth, we assessed factors independently associated with CEQ.

Materials and Methods

Study Design

After institutional review board approval of this cross-sectional study, patients who visited 1 of 4 participating orthopedic offices in a large urban area over a 6-month period in 2019 were invited to participate. A research assistant not involved in patient care recruited patients directly after their visit. Inclusion criteria were new and returning patients with musculoskeletal illness or injury in both upper and lower extremity, aged between 18 and 89 years, and English fluency and literacy.

Study Population

A total of 210 patients completed the questionnaire. Everyone who started the questionnaires completed them, and none of the patients were excluded from the analysis. The mean age (and standard deviation [SD]) was 51 ± 17 years (range 18-88 years; Table 1). Ninety-five (45%) patients were men. The most common race/ethnicity was white (70%), and 60% of the patients were married or unmarried couples.

Table 1.

Patient and Clinical Characteristics.a

Variables	N = 210
Age, years	51 ± 17 (18–88)
Men	95 (45)
Race/ethnicity
White	147 (70)
Latino/Hispanic	36 (17)
Black/African American	12 (5.7)
Asian	8 (3.8)
Other	7 (3.3)
Level of education
High school or less	33 (16)
Some college	56 (27)
Bachelor’s degree	58 (28)
Graduate or professional degree	63 (30)
Marital status
Married/unmarried couple	125 (60)
Single/dating	60 (29)
Divorced/separated/widowed	25 (12)
Work status
Employed	133 (63)
Retired	46 (22)
Disabled/unemployed	15 (7.1)
Other	16 (7.6)
Pain	4.4 ± 2.8 (0-10)
PROMIS Physical Function CAT	42 ± 9.6 (23–76)
JSPPPE	33 (29–35)
Ceiling effect 37%
CEQ	61 (56–63)
CEQ 1	7 (7–7)
CEQ 2	7 (7–7)
CEQ 3	7 (6–7)
CEQ 4	7 (6–7)
CEQ 5	7 (6–7)
CEQ 6	7 (5–7)
CEQ 7	7 (7–7)
CEQ 8	7 (7–7)
CEQ 9	7 (6–7)
Satisfaction	10 (10–10)
Ceiling effect 80%

Abbreviations: CEQ, Communication Effectiveness Questionnaire; JSPPPE, Jefferson Scale of Patient’s Perceptions of Physician Empathy; PROMIS, Patient-Reported Outcomes Measurement Information System; CAT, Computer Adaptive Test.

a Continuous variables as mean ± standard deviation (range) or median (interquartile range [IQR]); discrete variables as number (percentage).

Outcome Measures

Patients were asked to complete a set of questionnaires in the following order: (1) a demographic questionnaire consisting of gender, age, level of education, marital status, race/ethnicity, and occupation; (2) the CEQ; (3) an 11-point ordinal rating of satisfaction with the visit; (4) Perceived empathy using Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE); (5) an 11-point ordinal rating of pain intensity (0-10); and (6) magnitude of physical limitations, measured with Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) Computer Adaptive Test (CAT). After the questionnaires were completed, a research assistant recorded the office and the clinician. The vast majority of care was nonoperative.

The CEQ is a new questionnaire consisting of 9 statements about the visit (Appendix A). In an attempt to limit ceiling effects, the CEQ used 7-point Likert-type scales, from 1 (“Strongly Disagree”) to 7 (“Strongly Agree”). The total score is the sum of all item scores (9 to 63) with higher scores representing a greater degree of perceived communication effectiveness.

Satisfaction with the visit was measured using an 11-point ordinal rating scale, with 0 being not at all satisfied and 10 being completely satisfied with the visit.

The JSPPPE is a 5-item questionnaire, measured with 7-point Likert-type scales. It assesses agreement with statements about the physician. The scores range from 1 (“Strongly Disagree”) to 7 (“Strongly Agree”), with the total score being between 5 and 35. Higher scores represent greater empathic perception about the physician (9).

The magnitude of physical limitations was measured with PROMIS PF CAT. Higher scores indicate better physical function, with a mean of 50 for the general US population (10,11).

All questionnaires were completed on a tablet via secure, Health Insurance Portability and Accountability Act (HIPAA)-compliant electronic platform called REDCap (Research Electronic Data Capture: a secure web-based application for building and managing online surveys and databases) (12). All questions required an answer to prevent unanswered questions.

Statistical Analysis

The distributions of continuous variables and assumptions concerning normality were assessed to determine the appropriateness of the statistical tests. Descriptive statistics are presented as the mean ± SD for continuous variables and proportions for discrete variables. Bivariate analyses were conducted to test the association of each explanatory variable (demographics, satisfaction, pain intensity, perceived empathy, and magnitude of physical limitations) with the total score of the CEQ. We used Spearman correlation coefficient for continuous variables, Kruskal-Wallis tests for categorical variables, paired Student t tests, and Mann-Whitney U tests to assess differences between continuous variables, and Fisher’s exact tests for discrete variables. We aimed to create a multivariable linear regression model to assess factors independently associated with CEQ; however, due to a paucity of significant variables on bivariate analyses, multivariable analysis was omitted. Because perceived empathy, satisfaction, and communication effectiveness are expected to correlate a priori, we excluded these from the model too.

A scree plot of eigenvalues revealed that the 9 questions of the CEQ loaded onto one construct (Figure 1). Confirmatory factor analysis identified items 2, 5, 7, and 8 as having the highest factor loadings (0.91, 0.90, 0.91, and 0.92, respectively; Table 2). We also looked at Cronbach α for each item. Corrected item total correlations were highest for items 7 and 8. We tested the performance of these 2 items, referred to as the CEQ-2 (Appendix A).

Figure 1.

Screeplot of eigenvalues of the communication effectiveness questionnaire. An Eigenvalue <1 indicates that the factor contains more information than the average item.

Table 2.

Rotated Factor Loadings of the CEQ.

Items	Factor 1a
CEQ 1	0.85
CEQ 2	0.91
CEQ 3	0.76
CEQ 4	0.83
CEQ 5	0.90
CEQ 6	0.63
CEQ 7	0.91
CEQ 8	0.92
CEQ 9	0.85

Abbreviation: CEQ, Communication Effectiveness Questionnaire.

a Only 1 factor identified.

We measured the score distributions, mean scaled scores, Cronbach α, and floor and ceiling effects of the CEQ and the CEQ-2 (Table 3).

Table 3.

Number of Items, Score Distributions, and Floor and Ceiling Effects of the CEQ and CEQ-2.a

Questionnaire	Number of items	Median score	Range	Possible range	Median scaled scoreb	Median scaled range	Correlation (r)	Cronbach α	Floor effect	P value	Ceiling effect	P value
CEQ	9	61 (56–63)	9-63	9-63	61 (56–63)	9-63	.75c	0.93	1 (0.48)	.010	96 (46)	<.001
CEQ-2d	2	14 (13–14)	2-14	2-14	63 (59–63)	9-63		0.94	2 (0.95)		152 (72)

Abbreviation: CEQ, Communication Effectiveness Questionnaire.

a Bold indicates statistically significant difference; Spearman correlation indicated by r; Continuous variables as median (interquartile range [IQR]); Discrete variables as number (percentage).

b Scaled scores converted to maximum of 63.

c P < .001.

d CEQ short form with items 7 and 8 of the original CEQ.

We directed our a priori power analysis to the factor analysis because we knew that it would need more patients. The analysis indicated that a sample of 199 patients would provide 80% statistical power, with α set at 0.05 to detect a difference between the long and short form of 2 points with an SD of 10, tested with a paired t test. Including 5% extra to account for incomplete or incorrect data, a total of 209 patients were enrolled.

Results

Correlation CEQ With Patient Satisfaction

We found a moderate correlation between CEQ and patient satisfaction (ρ = 0.54, P < .001; Table 4).

Table 4.

Bivariate Analyses of Factors Associated With CEQ.

Variables	CEQ	P value
Age, years (r)	0.04	.529
Sex
Women	63 (57–63)	.071
Men	60 (55–63)
Race/ethnicity
White	60 (55–63)	.150
Non-white	63 (57–63)
Level of education
High school or less	63 (57–63)	.834
Some college	61 (56–63)
Bachelor’s degree	61 (56–63)
Graduate or professional degree	60 (55–63)
Marital status
Married/unmarried couple	61 (55–63)	.647
Single/dating	61 (56–63)
Divorced/separated/widowed	63 (58–63)
Work status
Employed	61 (56–63)	.786
Retired	60 (53–63)
Disabled/unemployed	63 (57–63)
Other	61 (57–63)
Pain (r)	−0.01	.934
PROMIS Physical Function CAT (r)	−0.05	.444
JSPPPE (r)	0.54	<.001
Satisfaction (r)	0.54	<.001

Abbreviations: CEQ, Communication Effectiveness Questionnaire; JSPPPE, Jefferson Scale of Patient’s Perceptions of Physician Empathy; PROMIS, Patient-Reported Outcomes Measurement Information System; CAT, Computer Adaptive Test.

a Bold indicates statistically significant difference; Spearman correlation indicated by r; continuous variables as median (interquartile range [IQR]).

Correlation CEQ With JSPPPE, Pain Intensity, and PROMIS PF

We found a moderate correlation between CEQ and perceived empathy as measured with JSPPPE (ρ = 0.54; P < .001; Table 4). We found no significant correlation between CEQ and pain intensity (ρ = −0.01; P = .934) or magnitude of limitations measured with PROMIS PF (ρ = −0.05; P = .44).

Factor Analysis

A scree plot of eigenvalues (Figure 1) and factor analysis (Table 2) revealed only one construct (factor) being measured by the CEQ items. Cronbach α for items 7 and 8 were 0.91 and 0.92, respectively, and they were selected for the CEQ-2. There was a strong correlation between the CEQ and the CEQ-2 (α = 0.75; P < .001; Table 3). The median scaled score was 61 (56–63) on the CEQ and 63 (59–63) on the CEQ-2 (Table 3).

Floor and Ceiling Effects and Internal Consistency

The CEQ and CEQ-2 both had minimal floor effects (0.48% and 0.95%, respectively; P = .010; Table 3). The CEQ had a ceiling effect of 46% and the CEQ-2 of 72% (P < .001).

Factors Independently Associated With CEQ

Due to a paucity of significant variables, we omitted multivariable analysis of factors independently associated with CEQ (Appendix A).

Discussion

The study and improvement in patient experience might benefit from the development of measures that have a limited ceiling effect and provide useful feedback to clinicians about specific and coachable strategies that can help them learn and improve (2,3). We took initial steps in developing a communication effectiveness questionnaire for use in assessing and coaching our clinicians.

Limitations of this study include the fact that both our comparator measures (JSPPPE and satisfaction) have high ceiling effects, respectively, 37% and 80% (Table 1). We used nonparametric statistics to account for this rather than categorizing the results (as is commonplace in research), but future work must identify ways to limit ceiling effects in all patient experience measures. The findings might best apply to our practice setting of orthopedic specialty care in a single urban location, although the fact that the patients were recruited from several different surgeon offices helps address this limitation. The results might be different in a more homogenous sample. In our opinion, the heterogeneous sample may increase generalizability. And if there is variation in patient experience by diagnosis, that variation may represent a strength. This preliminary work chooses questions that address specific features of communication that can be coached and practiced and also an attempt to reduce the ceiling effect (tendency for a high percentage of top scores). Formal validation will await the development of a questionnaire that has less ceiling effect. The analysis was based on knowledge that PREMs tend to correlate. We used satisfaction and empathy measures to establish that we were measuring what we intended rather than other communication measures, in part due to these known correlations, in part because of potential overlap of questions with the other questionnaires, and in part because of known ceiling effects with other communication questionnaires.

The observation that the CEQ has a moderate correlation with patient satisfaction supports the criterion and construct validity of the questionnaires (13). The American Academy of Orthopedic surgeons found that orthopedic surgeons have the lowest average outpatient satisfaction because of their relatively ineffective communication with patients (4,13,14). In some contexts, including patients after total knee arthroplasty, satisfaction is correlated with the fulfillment of expectations (15,16). Effective communication impacts how expectations are formed and managed and is likely to increase satisfaction (15,17 –19).

The observation that the CEQ has a moderate correlation with perceived empathy is also supportive (1,20,21). Longer questionnaires addressing communication effectiveness, perceived empathy, and patient satisfaction have overlapping questions that support content validity by pointing out the difficulty in separating these concepts (2,3,5). Consistent with prior research, patient experience measures such as the CEQ have little or no correlation with patient-reported outcome measures (pain intensity and magnitude of physical limitations) in cross-sectional studies (1 –3,22 –24). There may be more of a relationship over time or in response to intervention (6).

Factor analysis demonstrated that the 9 questions of the CEQ were all addressing a single underlying construct. Given the strong relationship of CEQ to other PREMs, it may be that they are all measuring a single underlying construct. A shorter, 2-item questionnaire may provide enough information for both quality improvement and research endeavors, but we need to find a way to diminish the ceiling effect of these measures. It is likely worthwhile to remove the ceiling effect before selecting the questions for a shorter questionnaire.

The CEQ had a notable ceiling effect, particularly in its 2-question form. Likert-type scales—even those with extreme expressions at the top end—have not been able to decrease ceiling effects in the use of patient-reported experience measures (3). We have had some success with Guttman-type questions in our work on satisfaction measures and plan to use Guttman answer structures to try to remove the ceiling effect from the CEQ. We will do that before we settle on a 2-question measure.

Attempts to isolate factors independently associated with CEQ were unsuccessful. The PREMs all correlate with one another, and no other demographic, illness-specific, or PROM measures had an appreciable influence. It is notable how difficult it is to identify factors associated with PREMs (5). That might relate in part to the high ceiling effects but also perhaps to some inherent aspect of patient experience.

This work is a step in developing a questionnaire with specific items that are coachable and trainable in order to help improve clinician communication strategies by identifying areas for improvement. We aim for a measure that leads to tailored feedback, coaching, and training. This preliminary work confirmed the large correlation between PREMs such as satisfaction, perceived empathy, and communication effectiveness, which directs us to identify a likely common statistical construct accounting for variation in PREMs. Given the mounting evidence of a common factor underlying PREMs as currently formulated, it might prove possible to measure patient experience using just a few simple questions. First, we need to develop a PREM, which measures that construct in a manner that results in a Gaussian distribution of scores, and then we can attempt to reduce the questionnaire to a small number of actionable items that can contribute to clinician coaching and personal development. At least within the limitations of current experience measures, there seems to be no association between illness (PROMs) and experience (PREMs).

Table 1.

The CEQ: Communication Effectiveness Questionnaire and Selected Questions for the Short Form; CEQ-2.

Please rate the following phrases. After today’s appointment.	1 = Strongly disagree	2	3	4 = Neutral	5	6	7 = Strongly agree
1. I have all of the information I need.	1	2	3	4	5	6	7
2. I understand what to do next for my health.	1	2	3	4	5	6	7
3. I am more motivated to address my health.	1	2	3	4	5	6	7
4. I believe I can improve my overall sense of wellness.	1	2	3	4	5	6	7
5. I have a trusting relationship with my health care providers.	1	2	3	4	5	6	7
6. I feel more relaxed than I did before my appointment.	1	2	3	4	5	6	7
7. I am certain that I can be open and honest with my health care providers (CEQ-2).	1	2	3	4	5	6	7
8. I know that my health care providers care about my well-being (CEQ-2).	1	2	3	4	5	6	7
9. I look forward to following up with my health care providers when necessary.	1	2	3	4	5	6	7