Does Hospital Consumer Assessment of Healthcare Providers and Systems survey correlate with traditional metrics of patient satisfaction? The challenge of measuring patient pain control and satisfaction in total joint replacement.

BACKGROUND: Medicare reimbursements have been tied to patient satisfaction measures. Despite these measures' influence on reimbursements, the relationship between pain management and patient satisfaction remains unclear. This study aims to evaluate the relationship between traditional patient perception of pain control and their overall satisfaction after joint replacement.
METHODS: This study is a retrospective review of consecutive primary total hip and total knee replacements. A total of 286 patients who underwent primary total hip (N = 106) and total knee (N = 196) replacements with completed Hospital Consumer Assessment of Healthcare Providers and Systems surveys were evaluated. Pain control, communication, and hospital satisfaction were stratified into satisfactory or unsatisfactory groups. These 2 groups were compared in terms of visual analog scale (VAS), opioid use in morphine equivalents, length of hospital stay, anesthesia type, and demographics.
RESULTS: Average VAS and opioid use did not differ between patient groups for any of the questions evaluated. Those who responded "always" to pain domain questions had a statistically shorter length of stay compared to patients with other response. On average, those who endorsed "always" on communication question were younger.
CONCLUSIONS: Patients who endorsed satisfactory pain control and communication with staff had shorter lengths of stay. There was no relationship between survey scores and traditional pain control measures such as VAS and opioid use. This questions the relevance of our primary pain measures in assessing patient perception of pain control. Length of stay may be a better surrogate marker for patient satisfaction of pain control.

Introduction

Pain management is increasingly recognized as a key component of health-care quality and improved patient outcomes. Appropriate pain management has been linked to earlier mobilization, shorter hospital stay, and reduced costs [1]. In 2001, the Joint Commission on Accreditation of Healthcare Organizations published standards for pain management in response to their estimation of widespread undertreatment of pain [2], [3]. Their recommendations included establishing clear metrics to assess pain and incorporating these standards into daily practice. As a result, hospitals have used postoperative pain management as a marker for overall patient satisfaction.

In 2007, the Centers for Medicare and Medicaid Services (CMS) required health-care organizations to report patient satisfaction through the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey [4]. This survey given to patients consists of questions about their perception of pain management during their treatment course. The 9 domains measured include communication with doctors, communication with nurses, responsiveness of hospital staff, communication about medications, discharge information, cleanliness of the hospital environment, quietness of the hospital environment, transition of care, and pain management.

The HCAHPS survey has been used as a comparative metric to evaluate quality of care and has been tied to hospital reimbursement as a portion of the hospital value-based purchasing program [5]. The current CMS guidelines give credit when patients award hospitals a score of greater than 9 of 10 or an “always” response. As a result, hospital organizations have sought to improve HCAHPS scores, focusing much of their attention on pain management. Despite the survey's profound impact on reimbursements, the relationship between pain management and patient satisfaction has not been critically evaluated in the orthopaedic literature.

In this study, we used the HCAHPS data to evaluate if there is a relationship between patients' perception of pain management and general satisfaction with their level of care. This study aims to specifically look at the 2 most widely reported pain measures, VAS and opioid usage, and their relationship to an arthroplasty patient's response on their HCAHPS. It is our hypothesis that patient satisfaction would be directly related to the perception of pain management and length of stay and traditional pain measures may have a more variable relationship.

Material and methods

This study is a retrospective review of all primary total hip and total knee arthroplasties performed by 3 fellowship-trained joint reconstruction surgeons from October 2013 to January 2015. All surgeries were done at the same suburban hospital within a tertiary referral health system. During October 1, 2013 and January 1, 2015, 560 HCAHPS surveys were mailed to patients who underwent knee or hip arthroplasty. Patients were eligible to receive a HCAHPS survey if they had not received one from a prior inpatient encounter at our particular hospital within 90 days before the operative date. Patients with outpatient clinic visits during this 90-day period or admission to another business unit within our health system were still eligible to receive this survey according to the set algorithm. A total of 302 patients returned the HCAHPS survey, which was included in our analysis. From these 302 patients, 106 patients underwent hip arthroplasty and 196 patients underwent total knee arthroplasty. Diagnosis-related group codes 469 and 470 were used by our HCAHPS vendor to generate a report of HCAHPS survey results, correlated to our 3 hip and knee arthroplasty surgeons. The final analysis consisted of 191 patients with data in both the demographics and HCAHPS data sets. Twenty-six patients were excluded from our analysis because of incomplete data in the HCAHPS data set. Ten patients were also excluded from our analysis, because of duplicate medical record numbers in the HCAHPS data set and presumed contralateral surgery. Up to 75 patients were excluded from individual question analysis due to unanswered questions on the survey. Responses to pain control (questions 13 and 14), communication (questions 16 and 17), and hospital satisfaction (questions 21 and 22) questions were stratified into either satisfactory or unsatisfactory responses. Satisfactory responses were achieved when patients reported that their expected standard of care was always met per the different domains. Anything less than that is considered undesirable in the CMS reimbursement system. Surveys were collected by Press Ganey, our CMS-approved survey vendor. This study was conducted with approval from the institutional review board.

Pain control (questions 13 and 14)

Patients were asked to respond to 2 questions regarding pain control: (1) “how often was your pain well controlled?”and (2) “how often did staff do everything they could to help with pain control?” Responses were provided using a 4-point Likert scale: never, sometimes, usually, and always. For analysis, pain control was considered satisfactory if the answer on the survey was always. All other responses (never, sometimes, or usually) were considered unsatisfactory.

Communication (questions 16 and 17)

Patients were asked to respond to 2 questions regarding hospital staff communication: (1) “before giving you any new medicine, how often did hospital staff tell you what the medicine was for?” and (2) “before giving you any new medicine, how often did hospital staff describe possible side effects in a way you can understand?” Responses were provided using a 4-point Likert scale: never, sometimes, usually, and always. For analysis, pain control was considered satisfactory if the answer on the survey was always. Always is the only score credited for reimbursement. All other responses (never, sometimes, or usually) were considered unsatisfactory.

Satisfaction (questions 21 and 22)

Overall satisfaction was the primary outcome variable for our hypothesis. The HCAHPS questionnaire included 2 independent questions regarding satisfaction: using any number from 0 to 10, where 0 is the worst hospital possible and 10 is the best hospital possible, what number would you use to rate this hospital during your stay? Patients were classified as satisfied with their hospitalization if their answer to the question was 9 or 10. All other answers were classified as unsatisfactory. The second question asked patients, would you recommend this hospital to your friends and family? Responses were provided using a 4-point scale: definitely no, probably no, probably yes, and definitely yes. Patients were classified as satisfactory if they answered definitely yes. All other responses were considered unsatisfactory. The complete survey can be found online at www.hcahpsonline.com [4].

Pain management protocols for our total joint replacements were standardized. All patients were compliant with preoperative protocol consisting of sustained-release morphine 15 mg, meloxicam 15 mg, and oral acetaminophen 975 mg. All patients received spinal anesthetic placed by a senior staff anesthesiologist unless otherwise contraindicated. Intraoperatively, a periarticular injection of 120 mL of diluted ropivacaine 300 mg with epinephrine 1 mg and ketorolac 30 mg was used for local administration. After surgery, patients were transferred from the postoperative anesthesia care unit to their respective private rooms where nurses began to monitor pain levels. Postoperative pain regimen consisted of sustained-release morphine 15 mg every 8 hours for 24 hours, meloxicam 15 mg daily, scheduled oral acetaminophen 975 mg every 8 hours, and oxycodone 5-10 mg as needed with morphine 1-2 mg intravenously as needed for breakthrough pain. VAS scores were recorded by nurses into the electronic medical record (Epic, Verona, WI) every 2-4 hours. Per protocol, pain assessment is required every 4 hours. However, if any intervention was administered, such as pain medication, another pain assessment was triggered. A standardized multimodal pain regimen was administered. This regimen consisted of the same scheduled and as-needed oral opioid medication, intravenous opioid medication for breakthrough, an oral anti-inflammatory and acetaminophen around the clock. Nonmedical relief measures such as ice, repositioning, and early mobilization to the side of the bed were used. We were not mobilizing with physical therapy until postoperative day 1 at that time.

Statistical analysis

Based on the survey, patients were stratified into satisfactory and unsatisfactory groups. The 2 groups' VAS, opioid use in morphine equivalents, and length of stay were analyzed. We collected demographic data including race, type of anesthesia (general vs regional), operating room time, surgical time, body mass index, height, weight, age, and gender. Statistical significance was set at P ≤ .05. All analyses were computed using R 3.2.2 (R Foundation, Vienna, Austria). We used Welch's t tests and Mann-Whitney U tests to find differences in mean responses between the 2 groups for numerical data. We used chi-squared tests to find differences in mean responses between the 2 groups for categorical data.

Results

Table 1, Table 2, Table 3, Table 4, Table 5, Table 6 show the results of each individual question. Average pain scores, total opioid units administered, operating room time, surgical time, and anesthesia type did not differ between the 2 groups for any of the 6 questions. On average, those who endorsed always for the communication questions were younger than those who responded with a different answer. No other demographics showed statistically significant differences.

Table 1

Question 13.a

Variableb	Total number of responses	Overall mean value	Number of other responses	Mean value in other response	Number of always responses	Mean value in always response	P value
Pain score	221	3.33 ± 1.71	53	3.19 ± 1.55	168	3.37 ± 1.76	.485
Opioid total	221	146.71 ± 83.82	53	148.15 ± 90.69	168	146.26 ± 81.81	.892
Length of stay	209	2.24 ± 0.56	48	2.50 ± 0.65	161	2.16 ± 0.51	<.001

Bold value indicate statistical significance below the threshold of 0.05.

Question 13 was “how often was your pain well controlled?” Responses were stratified to other and always. Other and always responses were correlated with pain, opioid use, and length of stay.

Pain score was measured with visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days. There was a significant difference in length of stay in the other response and always response.

Table 2

Question 14.a

Variableb	Total number of responses	Overall mean value	Number of other responses	Mean value in other response	Number of always responses	Mean value in always response	P value
Pain score	223	3.33 ± 1.70	18	2.65 ± 1.66	205	3.39 ± 1.70	.088
Opioid total	223	146.83 ± 83.77	18	120.39 ± 100.71	205	149.15 ± 82.00	.254
Length of stay	211	2.23 ± 0.57	14	2.43 ± 0.51	197	2.22 ± 0.57	.102

Question 14 was “How often did staff do everything they could to help with pain control?” Responses were stratified to other and always. Other and always responses were correlated with pain, opioid use, and length of stay.

Pain score was measured on a visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days.

Table 3

Question 16.a

Variableb	Total number of responses	Overall mean value	Number of other responses	Mean value in other response	Number of always responses	Mean value in always response	P value
Pain score	181	3.23 ± 1.88	36	3.06 ± 1.55	145	3.27 ± 1.96	.502
Opioid total	181	145.56 ± 87.88	36	131.78 ± 67.59	145	148.98 ± 92.10	.210
Length of stay	169	2.20 ± 0.55	35	2.40 ± 0.60	134	2.15 ± 0.53	.024

Bold value indicate statistical significance below the threshold of 0.05.

Question 16 was “Before giving you any new medicine, how often did hospital staff tell you what the medicine was for?” Responses were stratified to other and always. Other and always responses were correlated with pain, opioid use, and length of stay.

Pain score was measured on a visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days. There was a significant difference in length of stay in the other response and always response.

Table 4

Question 17.a

Variableb	Total number of responses	Overall mean value	Number of other responses	Mean value in other response	Number of always responses	Mean value in always response	P value
Pain score	176	3.13 ± 1.65	89	2.96 ± 1.58	87	3.30 ± 1.71	.166
Opioid total	176	145.27 ± 88.26	89	136.52 ± 81.04	87	154.22 ± 94.70	.185
Length of stay	165	2.20 ± 0.55	82	2.32 ± 0.56	83	2.08 ± 0.52	.008

Bold value indicate statistical significance below the threshold of 0.05.

Question 17 was “Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you can understand?” Responses were stratified to other and always. Other and always responses were correlated with pain, opioid use, and length of stay.

Pain score was measured on a visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days. There was a significant difference in length of stay in the other response and always response.

Table 5

Question 21.a

Variableb	Total number of responses	Overall mean value	Response <9	Mean value in response <9	Response ≥9	Mean value in response ≥9	P value
Pain score	228	3.36 ± 1.87	19	3.21 ± 1.29	209	3.38 ± 1.92	.616
Opioid total	228	145.13 ± 84.06	19	142.87 ± 87.29	209	145.34 ± 83.97	.907
Length of stay	215	2.23 ± 0.56	18	2.22 ± 0.43	197	2.23 ± 0.57	.950

Question 21 asked the patient to rate the hospital from 0 to 10, with 0 as the worst hospital possible and with 10 as the best hospital possible. Responses were stratified to responses less than 9 and responses greater than or equal to 9. These responses were correlated with pain, opioid use, and length of stay.

Pain score was measured on a visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days.

Table 6

Question 22.a

Variableb	Total number of responses	Overall mean value	Response <9	Mean value in response <9	Response ≥9	Mean value in response ≥9	P value
Pain score	228	3.37 ± 1.87	12	3.44 ± 1.68	216	3.37 ± 1.89	.888
Opioid total	228	145.41 ± 83.95	12	174.61 ± 97.37	216	143.79 ± 83.10	.304
Length of stay	215	2.23 ± 0.57	11	2.27 ± 0.47	204	2.23 ± 0.57	.718

Question 22 asked the patient to rate his or her satisfaction from 0 to 10, with 0 as completely unsatisfied and with 10 as completely satisfied. Responses were stratified to responses less than 9 and responses greater than or equal to 9. These responses were correlated with pain, opioid use, and length of stay.

Pain score was measured on a visual analog scale. Opioid use was measured in morphine equivalents. Length of stay was measured in days.

The distribution of length of stays showed statistically significant differences between groups for questions 13, 16, and 17. Patients who responded always to either questions 13, 16, or 17 had shorter lengths of stay compared to those who responded with any other response for those 3 questions.

Table 7 was built to stratify the always answer to discharge disposition. In this table, we included the percentage of always response for both home discharge and rehabilitation center discharge. Home discharge includes discharge to home with home health services. In addition, we performed a chi-square test to determine if there were any significantly different responses depending on whether patients were discharged home or to a rehabilitation center.

Table 7

Correlation of discharge disposition with survey responses.

Question	Percent always response for home discharge (%)	Percent always response for rehabilitation center discharge (%)	Chi-square test (P value)
Question 13	77.2	69.0	.31
Question 14	91.9	89.7	.68
Question 16	84.2	71.4	.12
Question 17	56.2	38.1	.10
Question 21	92.7	87.1	.26
Question 22	95.0	87.1	.06

Discussion

This is the first study to analyze the relationship between HCAHPS patient survey scores with traditional pain metrics in total joint replacement patients. This study did not find a relationship between HCAHPS survey scores and traditional inpatient pain control metrics such as VAS. This study also did not find a relationship between opioid use and patient's perception of adequate pain control (question 13). Clinicians often find themselves being pulled in 2 directions in managing pain. On 1 end, clinicians are encouraged to limit opioid use to prevent future dependency. On the other end, clinicians are wary that limiting opioid medications may negatively impact patient-reported scores, which may influence reimbursement. This study is useful in questioning whether limiting opioid use will negatively impact patient-reported pain scores. This study did find that a shorter length of hospital stay was related to a patient's perception of adequate pain control (question 13) and staff's communication to the patient about pain management (questions 16 and 17). Also, much of our literature comparing various surgical approaches, pain control methods such as catheter-based anesthetic or periarticular with standard vs liposomal bupivacaine relies on these traditional metrics. This does call into question the utility of these 2 traditional metrics that we often use to assess superiority in this new patient-centric world of outcomes.

One of the largest driving forces in implementing HCAHPS was to ensure that the patient's pain was well controlled during hospital stays. Approximately, one-third of the 2% Medicare reimbursement adjustment is traditionally tied to management of pain control. Since no relationship was found between HCAHPS scores and traditional pain metrics, this study calls into question the validity of traditional pain measures themselves as they relate to the joint replacement patient's perception of pain control. Surveys are given to patients up to 6 weeks after being discharged from the hospital [6]. This prolonged time interval allows for recall bias on how well a patient's pain was managed during their hospital stay. Since there is a time lag between discharge and surveys being sent, the HCAHPS survey may better reflect the patient's current level of satisfaction after total joint replacement. Several studies have shown that satisfaction after total joint replacement is multifactorial, extending beyond patient care in the perioperative period. A patient's psychological state, socioeconomic status, understanding of degenerative joint disease, and expectations of outcomes are important predictors of satisfaction after total joint replacement [7], [8], [9]. As a result, there are many confounding factors that may influence the HCAHPS surveys. On the other side, if any intervention we try to analyze is linked to traditional pain measures such as VAS and opioid use, we may not be truly understanding the effect from the patient's side of reported outcome measures.

Since HCAHPS scores are tied to reimbursements, these surveys play an important financial role for health systems [5]. The motive behind this arrangement was to align reimbursements to patient outcomes. Although the arrangement may be well intended, there are mixed signals on how the incentives are designed to influence hospital and physician behavior. If the domains measured in HCAHPS intend to reflect added value to the patient, then the change in preoperative and postoperative clinical and functional outcomes would be the most appropriate metric. If the domains measured in HCAHPS are intended to measure the patient's overall experience during the hospital stay, then completing the survey at the time of discharge may yield more actionable data. However, the CMS does not allow surveys to be submitted at the time of hospital discharge [6]. Clarification on intent will allow hospitals to successfully implement processes and protocols designed to meet these metric targets. As health-care reform progresses, it is key to make sure our desired outcome measures are indeed valid if we are making value-based decisions centered around their results. This is especially true with the wide array of options for multimodal pain control after joint replacement.

This study did find that patients with shorter hospital stays were more satisfied with their pain management and level of staff communication. Although this study found a significant correlation, we must be prudent when drawing conclusions, as correlation does not necessarily mean causation. However, there are some reasonable inferences that can be made from this. First, if patients believed that their pain was well controlled, then they would feel safe being discharged home under their own care. This may explain why these patients were discharged earlier than patients who did not believe their pain was well controlled. This study cannot conclusively infer that improved communication results in shorter hospital stays, but the link is intuitive. As far as the traditional use of VAS and opioids as the final metric by which we are to judge pain control, this study illustrates the need for an updated patient-centric system.

There are several weaknesses in this study. First, this is a retrospective study, which by design has inherent limitations in drawing firm conclusions. The data are supplied by nursing records, which is often the standard in these pain studies. An effort was made to standardize the scoring records by implementing a dry-erase pain board using a modified Mankoski pain scale. Second, there may be a potential sampling bias based on demographic differences in patients responding to the survey and patients who did not respond to the survey. While demographic data points for patients responding to the survey are included in the HCAHPS survey report, we were unable to obtain demographic data points for patients who did not respond to the HCAHPS survey. Third, there was no preoperative pain metrics obtained on any of these patients. Previous studies have found that patients who had more severe preoperative symptoms were more likely to be satisfied after total joint replacement [10], [11]. In addition, patients severely deconditioned before joint replacement were less likely to be satisfied with their outcome [11]. Measuring how patients improve after a total joint replacement by comparing preoperative to postoperative metrics may provide some additional insight. For example, if there is not a strong relationship between HCAHPS survey scores and quality of life improvement after a total joint replacement from that particular hospital, then further analysis can be done to assess how sensitive such patient reporting actually is after total joint replacement. This new analysis may be a more reliable marker for delivering value to the patients. Linking reimbursement to this type of patient outcome is the best way to create the most value in health care [12]. Finally, obtaining longer clinical follow-up results would help us better characterize the HCAHPS survey. For example, if there is a strong correlation between HCAHPS survey scores and clinical outcomes at 6-month follow-up, then we could analyze if such survey scores could be used as a surrogate for outcomes or just predict psychological and social influences that may need more modification.

Conclusions

Patient satisfaction scores reported in the HCAHPS survey have directly influenced the CMS reimbursements. This retrospective review further characterizes the relationship between patient satisfaction scores and the perception of pain control in patients who received total joint arthroplasty. The data reveal that patient perception of satisfactory pain control and staff communication of pain management are related to shorter lengths of stay. There was no relationship between survey scores and traditional pain control measures such as VAS and opioid use. This questions the clinical relevance of traditional pain measures in the joint replacement patient. Length of stay may be a better surrogate marker for patient satisfaction of pain control and supports further investigation into more holistic pain outcome measures.