Long-term Clinical Outcomes Among Responders and Nonresponders to the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) Survey.

RATIONALE &
OBJECTIVE: The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) survey, introduced into the End-Stage Renal Disease Quality Incentive Program, is the only patient-reported outcome currently used for value-based reimbursement in dialysis. Current response rates are ∼30% and differences in long-term clinical outcomes between survey responders and nonresponders are unknown. STUDY
DESIGN: Retrospective cohort study. SETTING & PARTICIPANTS: Patients from all Dialysis Clinic Incorporated facilities from across the United States who met survey eligibility (aged ≥18 years and had been treated at their facility for at least 3 months). EXPOSURES: Patient-level demographic, clinical, and treatment-related characteristics. OUTCOMES: Mortality, all-cause hospitalization, and kidney transplantation. ANALYTICAL APPROACH: Time-to-event analyses using competing-risks models. Sensitivity analyses performed after multiple imputation for missing covariate data.
RESULTS: Among 10,395 eligible patients, 3,794 (36%) responded to the survey. During a median follow-up of 33 months, 4,588 patients died, 7,638 patients were hospitalized at least once, and 789 patients received a transplant. In multivariable models, survey response was associated with lower mortality (subdistribution hazard ratio [sHR], 0.80; 95% CI, 0.75-0.86) and hospitalization (sHR, 0.94; 95% CI, 0.89-0.99) and higher likelihood for a kidney transplant (sHR, 1.27; 95% CI, 1.10-1.46). Results were consistent across sensitivity analyses after multiple imputation for missing covariates. LIMITATIONS: Small amount of missing covariate data, baseline covariate data assigned at the first month of the 3-month survey administration period, reasons for nonresponse unknown.
CONCLUSIONS: Response to the ICH CAHPS survey is associated with lower risk for mortality and hospitalization and higher likelihood for kidney transplantation. These findings suggest that survey responders are healthier than nonresponders, emphasizing the need for caution when interpreting facility-level survey results to inform quality improvement and public policy efforts and the critical need to better capture patient-reported outcomes from more vulnerable patients.

With high rates of kidney failure and high costs of dialysis, the US Centers for Medicare & Medicaid Services (CMS) moved to a value-based purchasing model with the introduction of the End-Stage Renal Disease Quality Incentive Program (ESRD QIP) in 2012., The In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH CAHPS) survey was the first patient-reported quality metric added to the ESRD QIP. After introduction of this survey into the ESRD QIP in 2014, mandatory twice-yearly reporting began in calendar year 2016 and survey results began affecting dialysis facility reimbursement in payment year 2018.

The response rate to the ICH CAHPS survey was 46% during development and has since decreased to ∼30% nationally.4, 5, 6, 7, 8 Given these declining response rates, there is concern about the generalizability of survey results due to nonresponse bias, which is a type of selection bias. This selection bias is a frequent problem in health survey data, including CAHPS survey data, and cannot be remedied statistically.10, 11, 12, 13 There are data that point to health survey nonresponders being younger, sicker, and poorer.14, 15, 16, 17 In keeping with these findings, our previous cross-sectional analyses showed that ICH CAHPS survey responders are generally healthier, socioeconomically more advantaged, and more adherent to hemodialysis treatment than nonresponders. There has been limited research to address ICH CAHPS survey nonresponse, either through qualitative research to understand the reasons behind nonresponse or through trials of interventions aimed at reducing nonresponse. This deficit is compounded by regulatory changes enacted in 2014 that prohibit dialysis providers from obtaining patient-level ICH CAHPS survey results for independent research.

Despite these concerns, assessing patient experience is an important aspect of medical care, and the ICH CAHPS survey results have been increasingly emphasized by CMS. Initially, facilities received credit for simply administering the ICH CAHPS survey. Starting in 2016, ICH CAHPS survey scores affected facility reimbursement and were publicly reported online (www.medicare.gov/dialysisfacilitycompare); these scores have substantial influence on ESRD Seamless Care Organizations shared savings and are used by CMS to generate publicly reported Star ratings.,, To date, there is no evaluation of response status to the ICH CAHPS survey and long-term clinical outcomes. Using data from a large real-world national sample of in-center hemodialysis (ICH) patients from Dialysis Clinic Incorporated (DCI) facilities, we examined the relationship between ICH CAHPS survey response status and mortality, hospitalization, and kidney transplantation.

Methods

Study Population

ICH patients 18 years and older who had been at their facility for at least 3 months were eligible for the ICH CAHPS survey in 2012. According to contemporaneous Agency for Healthcare Research and Quality (AHRQ) guidelines, surveys with at least 50% of predefined key questions answered and those that indicated no receipt of proxy help with survey completion were eligible for scoring. In 2012, institutionalized patients were eligible for this survey. We included patients from DCI facilities across the country who were eligible for the survey at the start of August 2012 and remained ICH patients at DCI through the end of the survey period in October 2012.

Study Design

In this longitudinal study, primary analyses assessed the relationship between response status to the 2012 ICH CAHPS survey administration and time to death. Secondary analyses evaluated time to first hospitalization and time to kidney transplantation. Patient-level survey results were obtained by DCI for research purposes only under a Respondent Identifiable Information Disclosure Agreement with DCI’s survey vendor. A member of the DCI information technology team, not affiliated with the research team, provided merged and deidentified data to the authors. This study was approved by the Tufts Health Sciences Investigational Review Board (IRB#11150) and underwent review by the DCI Administrative Review Office.

Survey and Administration

Details regarding survey administration among DCI facilities in 2012 have been published previously., In brief, the ICH CAHPS survey administered in 2012 had 58 questions and was available in English and Spanish (Item S1). Three questions rated the nephrologist, dialysis staff, and dialysis facility (on a 0-10 scale, with 10 being the best). The remaining questions were used to assess patient characteristics and compile the 3 following composite scores: Nephrologists’ Communication and Caring, Quality of Dialysis Center Care and Operations, and Providing Information to Patients.

Patients were sent a prenotification letter before the first survey mailing. Nonresponders were sent the prenotification letter again followed by a second survey mailing. Those who had still not responded were contacted by telephone to complete the survey. The survey administration period extended from August 1, 2012, to October 22, 2012. Dialysis facility staff were strictly not allowed to discuss the survey with patients or help them complete the survey.

Covariates

Covariates thought to be either confounders or related to the outcome were chosen a priori based on the prior literature and clinical knowledge. All covariate data were obtained from the DCI electronic medical record system. Because the precise date of individual survey completion is not known, all covariate data were obtained from the month the survey administration began (August 2012). Any covariate data that were missing in August prompted a 3-month look back and the most proximate value was used. Body mass index was calculated using the most recent estimated goal weight set by the patient’s nephrologist. Hospitalization for any reason during survey administration was defined as any hospital stay longer than 1 day between August 1 and October 22. “Treatment shortened” was defined as having any dialysis treatment that was 15 minutes shorter than prescribed for any reason. “Unexcused absence” was defined as having missed any dialysis treatment without rescheduling and without a known reason such as a hospitalization.

Outcomes

Outcome data were obtained from the DCI electronic medical record system through January 18, 2018. Follow-up time began the day after the end of the survey administration period (October 22, 2012) to avoid immortal time bias. Hospitalization was defined as any hospital stay longer than 1 day after the end of the survey administration period. We used competing-risks models for the primary and secondary analyses. For the outcome of death, we treated transplantation as a competing risk because we did not have follow-up data for patients after transplantation. We treated death as a competing risk for the outcome of hospitalization and for the outcome of transplantation. Patients were censored at the date of their last treatment if they withdrew from hemodialysis, stopped hemodialysis after recovering kidney function, transferred out of DCI care to a different dialysis provider, or were lost to follow-up. All remaining patients were censored at the end of the follow-up period (January 18, 2018). Those receiving more than 1 transplant or having more than 1 hospitalization were censored at the time of their first event for analyses evaluating transplantation or hospitalization, respectively. Patients who withdrew from hemodialysis were grouped with patients known to have died for the purpose of analysis.

Statistical Analyses

The association between continuous covariates and each outcome was explored; covariates with nonlinear associations were analyzed as categorical variables. ESRD vintage was truncated at 100 months before evaluation. Event rates for each outcome were modeled using Poisson regression. Time-to-event analyses were performed and subdistribution hazard ratios (sHRs) were obtained for each outcome using Fine and Gray competing-risks models. Models were fitted sequentially starting with a parsimonious model with only demographic covariates followed by the addition of clinical and treatment-related covariates. Additionally, we included robust sandwich covariance matrix estimates to account for the intracluster dependence among dialysis facilities. The assumption of proportional hazards between the covariate of interest and outcome was evaluated using Schoenfeld residuals. Unadjusted associations between ICH CAHPS response status and each outcome were assessed using cumulative incidence derived from the Fine and Gray competing-risk models (using the CMPRSK package in R). Adjusted cumulative incidence curves were created using the mean (or median if skewed distribution) value for continuous covariates and the most common value for categorical covariates. For sensitivity analyses, multiple imputation technique using 10 imputations was used to handle missing data. The imputation model included all covariates used for adjustment along with the outcomes. We obtained averaged effect estimates using Rubin’s rule.

All analyses were performed using R, version 1.1.414 (R Foundation for Statistical Computing), and SAS Enterprise Guide (SAS Institute; version 7.12).

Results

DCI facilities were located in 28 states in 2012 (Fig S1). Of 11,055 patients eligible in 2012, a total of 10,395 (94%) remained DCI ICH patients at the end of the ICH CAHPS survey administration period (Fig 1). Of the 660 excluded patients, most either died or transferred from DCI to a different dialysis provider (Table S1). Included patients were on average 61 years old with median ESRD vintage of 40 months, 44% were women, and 45% were African American (Table 1). Mean age and cause of ESRD were similar to national estimates from 2012, whereas the percentage of African American patients was higher. Of the 3,419 responders, 2,846 (83%) responded by mail versus telephone (Table S2). Those responding by telephone compared to mail were younger, more likely to be African American, more likely to be receiving dual Medicare/Medicaid, less educated, and with lower treatment adherence. Overall, responders compared with nonresponders were older, more likely to be women, and more likely to be married. Responders were also more likely to be active on the transplant list, have arteriovenous fistulas for vascular access, have shorter ESRD vintage, and have better treatment adherence. About 20% of patients were hospitalized during the survey administration period; responders were less likely to have been hospitalized than nonresponders (Table 1).

Figure 1

Flow diagram. Abbreviations: AHRQ, Agency for Healthcare Research and Quality; DCI, Dialysis Clinics Inc.

Table 1

Baseline Demographics Stratified by Response Status

	Total (N = 9,141)	Responders (n = 3,419)	Nonresponders (n = 5,722)
Age, y	61.0 ± 14.7	62.1 ± 13.8	60.3 ± 15.2
Women	4,022 (44.0%)	1,574 (46.0%)	2,448 (42.8%)
Race
African American	4,090 (44.7%)	1,316 (38.5%)	2,774 (48.5%)
White	4,386 (48.0%)	1,940 (56.7%)	2,446 (42.7%)
Other	665 (7.3%)	163 (4.8%)	502 (8.8%)
Insurance
Medicare/Medicaid	3,258 (35.6%)	970 (28.4%)	2,288 (40.0%)
Medicare only	3,581 (39.2%)	1,566 (45.8%)	2,015 (35.2%)
Medicaid only	533 (5.8%)	153 (4.5%)	380 (6.6%)
Other	1,769 (19.4%)	730 (21.4%)	1,039 (18.2%)
Marital status
Married	3,499 (38.3%)	1,477 (43.2%)	2,022 (35.3%)
Divorced/separated	1,920 (21.0%)	710 (20.8%)	1,210 (21.1%)
Widowed	1,426 (15.6%)	488 (14.3%)	938 (16.4%)
Single	2,296 (25.1%)	744 (21.8%)	1,552 (27.1%)
Education
Grade school	1,193 (13.1%)	272 (8.0%)	921 (16.1%)
High school	5,570 (60.9%)	2,106 (61.6%)	3,464 (60.5%)
≥College	2,378 (26.0%)	1,041 (30.4%)	1,337 (23.4%)
Hospitalized during survey administration	2,090 (22.9%)	591 (17.3%)	1,499 (26.2%)
Active on transplant waitlist	1,030 (11.3%)	460 (13.5%)	570 (10.0%)
BMI, kg/m2	28.5 ± 7.5	29.2 ± 7.6	28.1 ± 7.4
Cause of ESRD
Diabetes	3,950 (43.2%)	1,384 (40.5%)	2,566 (44.8%)
Hypertension	2,585 (28.3%)	970 (28.4%)	1,615 (28.2%)
Other	2,606 (28.5%)	1,065 (31.1%)	1,541 (26.9%)
Vascular access
Catheter	1,462 (16.0%)	465 (13.6%)	997 (17.4%)
Graft	1,955 (21.4%)	716 (20.9%)	1,239 (21.7%)
Fistula	5,724 (62.6%)	2,238 (65.5%)	3,486 (60.9%)
Albumin, g/dL	3.8 ± 0.4	3.9 ± 0.4	3.8 ± 0.4
Kt/V	1.62 ± 0.28	1.63 ± 0.27	1.62 ± 0.29
ESRD vintage, mo	40.3 [19.5-76.3]	37.5 [18.2-72.0]	41.9 [20.5-78.3]
Treatment shortened	4,537 (49.6%)	1,497 (43.8%)	3,040 (53.1%)
Unexcused absences	1,575 (17.2%)	477 (14.0%)	1,098 (19.2%)

Note: Values expressed as mean ± standard deviation, median [25th-75th percentiles], or number (percent). Kt/V is a unitless measure of dialysis adequacy (national goal > 1.2).

Abbreviations: BMI, body mass index; ESRD, end-stage renal disease.

Primary and Secondary Analyses

During a median follow-up of 33 months, 4,588 (50.2%) patients died (Fig 2). Following multivariable adjustment, responders had a lower risk for death (sHR, 0.80; 95% confidence interval [CI], 0.75-0.86; Table 2). During the follow-up period, 7,638 (83.6%) patients were hospitalized at least once (Fig 2). Responders had a lower risk for hospitalization after multivariable adjustment (sHR, 0.94; 95% CI, 0.89-0.99; Table 2). During follow-up, 789 (8.6%) patients received a kidney transplant (Fig 2). Following multivariable adjustment, responders were more likely to receive a transplant than nonresponders (sHR, 1.27; 95% CI, 1.10-1.46; Table 2).

Figure 2

Cumulative incidence curves for each outcome.

Table 2

Event Rates and Multivariable Association Between Survey Response Status and Outcomes

	No Response (N=5722)			Response (N=3419)
	Years/Events	Event Rate, 100-py (95% CI)	sHR (95% CI)	Years/Events	Event Rate, 100-py (95% CI)	sHR (95% CI)
Death
Unadjusted	14,933/3,022	19.7 (19.0-20.4)	1 (reference)	9,814/1,566	15.5 (14.8-16.3)	0.77 (0.73-0.82)
Model 1		18.1 (17.1-19.2)	1 (reference)		12.7 (11.8-13.6)	0.68 (0.64-0.73)
Model 2		17.4 (16.3-18.5)	1 (reference)		14.0 (13.0-15.1)	0.79 (0.74-0.85)
Model 3		17.2 (16.2-18.3)	1 (reference)		14.0 (13.0-15.1)	0.80 (0.75-0.86)
Hospitalization
Unadjusted	5,334/4,798	83.7 (81.3-86.1)	1 (reference)	3,944/2,840	68.7 (66.2-71.3)	0.88 (0.85-0.92)
Model 1		82.6 (79.1-86.1)	1 (reference)		63.6 (60.3-67.0)	0.86 (0.83-0.91)
Model 2		87.9 (83.9-92.0)	1 (reference)		76.6 (72.5-80.9)	0.93 (0.89-0.98)
Model 3		87.4 (83.5-91.5)	1 (reference)		76.9 (72.8-81.3)	0.94 (0.89-0.99)
Transplant
Unadjusted	14,933/427	2.8 (2.5-3.1)	1 (reference)	9,814/362	3.6 (3.2-4.0)	1.43 (1.27-1.62)
Model 1		1.7 (1.4-2.0)	1 (reference)		2.4 (2.0-2.8)	1.55 (1.37-1.74)
Model 2		1.4 (1.1-1.6)	1 (reference)		1.7 (1.4-2.0)	1.27 (1.11-1.46)
Model 3		1.4 (1.1-1.6)	1 (reference)		1.6 (1.4-2.0)	1.27 (1.10-1.46)

Note: Model 1 adjusted for age, sex, race, insurance type, marital status, and education level. Model 2 adjusted for covariates in model 1 and hospitalization during survey administration, transplant waitlist status, body mass index (per 2 kg/m2), cause of ESRD, vascular access type, serum albumin level (per 0.2 g/dL), Kt/V (per 0.2), and ESRD vintage. Model 3 adjusted for covariates in model 1 and 2, treatments shortened in last month, and unexcused absences in last month. Kt/Vurea is a unit less measure of dialysis adequacy (national goal>1.2).

Abbreviations: ESRD, end-stage renal disease; Events, total number of events; sHR, subdistribution hazard ratio; BMI, body mass index; Years, total years of follow-up; CI, confidence interval.

Missing Data

Overall, 1,254 (12%) patients had missing data for at least 1 covariate and were not included in primary and secondary analyses. Nonresponders were more likely to have missing data than responders (13.3% vs 9.9%) and most of the missing data were either demographic characteristics or kidney transplant waitlist status (Tables S3 and S4). Demographic characteristics and long-term outcomes were similar between those with and without missing data (Table S3 and S5). The associations between survey response and all 3 outcomes were essentially unchanged after multiple imputation for missing covariates (death: sHR, 0.80; 95% CI, 0.75-0.85; hospitalization: sHR, 0.95; 95% CI, 0.90-0.99; transplantation: sHR, 1.28; 95% CI, 1.12-1.45).

Discussion

In a large national population of US hemodialysis patients, responders to the ICH CAHPS survey of patient experience had lower risk for mortality and hospitalization and higher likelihood of kidney transplantation than nonresponders, suggesting that survey responders are healthier than nonresponders. These associations were consistent across sensitivity analyses. Our findings emphasize the need for caution when interpreting facility-level survey results to inform quality improvement and public policy, including for assessing value-based care quality. Additionally, these findings highlight the critical need to better capture patient-reported outcomes from more vulnerable patients who likely have worse experiences.

Increasingly, payers in the United States are moving toward value-based purchasing or pay-for-performance payment models within the context of increasing health care costs. Payment for dialysis underwent this change in 2012 with the advent of the ESRD QIP. Quality metrics within the QIP, chosen largely by policy makers, initially were clinical and laboratory based., The 2014 introduction of the ICH CAHPS as the first standardized and mandatory metric assessing patient-reported outcomes was an important step in making QIP measures more patient-centered. Presently, dialysis facility ICH CAHPS scores are publicly reported on the CMS Dialysis Facility Compare website and added to the total performance score of each facility, with the latter determining dialysis facility reimbursement.,,

Despite the importance of this patient-reported outcome, patient response rates to the ICH CAHPS have declined and plateaued during the past several years, resulting in possible nonresponse bias, which is a specific type of selection bias present in survey data. This nonresponse bias in turn has the potential to bias value-based purchasing models that incorporate ICH CAHPS results. In an attempt to mitigate this bias, CMS started adjusting facility scores starting in 2015 by survey administration mode, proxy help, and 12 patient self-reported characteristics. To date, there has been no published research, either qualitative or quantitative, assessing reasons for nonresponse. Future efforts will be hindered by a prohibition on using patient-level ICH CAHPS data for research since 2014.

This analysis represents the only longitudinal assessment of clinical outcomes among both ICH CAHPS responders and nonresponders. There has not been any standardized and mandatory patient experience survey used in the US dialysis population before this for comparison. The ICH CAHPS survey is one of several patient experience CAHPS surveys currently used in the United States to evaluate different areas of health care and health plans. The association between response status and long-term outcomes for these other CAHPS surveys has not previously been assessed either. Cross-sectional data from hospital and Medicare CAHPS data have shown nonresponse to be associated with male sex, nonwhite race, younger age, and lower socioeconomic status.,, In a previous cross-sectional analysis, we showed that several patient-level characteristics, including younger age, nonwhite race, dual Medicare/Medicaid eligibility, recent hospitalization, unexcused absences, and shortened treatments, were associated with nonresponse to the ICH CAHPS survey. This work strengthens our previous findings and shows worse long-term clinical outcomes among nonresponders compared with responders. Because nonresponse to the survey is not causally linked to long-term outcomes, these associations are probably explained by differences in health status, socioeconomic status, and treatment adherence.

Publicly reported survey response rates through 2018 remain ∼30% nationally.5, 6, 7, 8 Our findings raise the concern that the current survey results do not include the experience of the most vulnerable hemodialysis patients. Based on our prior work, certain characteristics are associated with both nonresponse and lower ICH CAHPS scores, including younger age and shortened hemodialysis treatments. How these differences affect facility-level survey scores is difficult to predict because CMS adjusts scores based on several patient self-reported characteristics. Our findings here suggest that we are missing responses from patients who are the least adherent and have the poorest experience and worst long-term outcomes. Therefore, it is plausible that responders likely have higher experience scores than nonresponders, potentially resulting in inflated facility-level scores. Nevertheless, this limits the use of the ICH CAHPS survey as a standardized patient-reported outcome measure for comparison of dialysis facilities or for quality improvement. Additionally, this raises questions about the adequacy of using survey results for public reporting and dialysis facility financial reimbursement.

Going forward, improving response rates among these at-risk patients may increase the utility of facility-level results by helping identify facilities that may need interventions to improve patient experience. At the national level, improvement in response rates will better inform policy making, quality improvement, and facility and provider rankings. Further research and innovation are needed to develop and implement ways to better engage these vulnerable hemodialysis patients to allow them to share their experience. Potential strategies to explore include decreasing survey length, increasing font size on paper surveys, and offering additional survey modalities. Ultimately, engaging dialysis patients in qualitative work assessing reasons for nonresponse would be extremely instructive in our understanding of the obstacles these patients face.

Our study has several limitations. Because of subsequent regulatory restrictions, all survey response data represent patients treated by 1 national dialysis provider (DCI) in 2012. At the time, there were relatively few DCI clinics in states that had the highest prevalence of ESRD (Fig S1). We did not have reliable data for patient comorbid conditions or adherence to medications and diet. There was a small amount of missing covariate data; however, sensitivity analyses after multiple imputation showed similar results. Reasons for nonresponse were not collected by AHRQ and are unknown. Finally, baseline data were assigned at the first month of survey administration because the actual date of survey completion within the survey administration period is not obtained in the survey.

Our study also has several strengths. This is the first analysis examining response status to the ICH CAHPS survey and long-term outcomes. We included a large sample of real-world hemodialysis patients along with extensive baseline clinical data obtained within the dialysis facility and during 5 years of follow-up. Our outcomes are clinically relevant and our analytic models use multivariable adjustment for patient factors, account for competing risks, and also address clustering at the dialysis facility level. Last, this study is unique because patient-level survey data can no longer be obtained.

At present, the ICH CAHPS survey has a response rate of only 30%, with healthier and more adherent patients disproportionately providing data about their hemodialysis experiences. This nonresponse bias potentially results in a missed critical opportunity to gather additional informative and important data and engage more vulnerable hemodialysis patients. These results highlight a critical need for initiatives to encourage patient engagement at the facility level, efforts that may not only improve responsiveness to patient-reported outcome reporting but may also help overcome barriers to poor treatment adherence.