Measuring patients' experience with renal services in the UK: development and validation of the Kidney PREM.

BACKGROUND: Patient experience is a recognized aspect of quality of care for people with chronic kidney disease (CKD), but current patient-reported experience measures (PREMs) only focus on dialysis care. We developed and validated the Kidney PREM to assess patients' experience with renal services in secondary care for any CKD stage or treatment (transplant, haemodialysis and peritoneal dialysis).
METHODS: We developed the Kidney PREM in two phases, informed by a multidisciplinary expert group to ensure face validity. We organized three national data collections (2016-8) to investigate item response profiles and to conduct exploratory and confirmatory analyses to assess internal consistency. We also explored content validity in cognitive interviews and evaluated test-retest reliability. Finally, we developed the Kidney PREM Short Form for more frequent measurement of patient experience to inform local service improvements.
RESULTS: We analysed 32 959 responses across data collections, with the 2018 collection covering all 71 UK renal centres. The Kidney PREM final version consisted of 38 items grouped into 13 themes, all pertaining to one underlying dimension reflecting the construct of 'patient experience' with high internal consistency (Cronbach's α = 0.94). The Kidney PREM Short Form consisted of 15 items across the same 13 themes.
CONCLUSIONS: The Kidney PREM supports the collection of reliable information on patient experience that people with CKD consider relevant, regardless of CKD stage or treatment modality. Kidney PREM data have the potential to guide local and national initiatives to improve patients' experiences with renal services in the UK and other countries.

What is already known about this subject?

Patient-reported experience is a recognized aspect of quality of care for people with chronic kidney disease (CKD).

A better patient experience is associated with better quality of care, higher treatment adherence and improved patient outcomes.

Available validated patient-reported experience measures (PREMs) are limited to dialysis care and consist of multiple underlying dimensions with often suboptimal internal consistency.

What this study adds?

With >32 000 survey responses, this is the largest study to date internationally to develop and validate a PREM for people with CKD, regardless of disease stage or treatment modality.

The Kidney PREM consists of 38 items grouped into 13 themes, all pertaining to one underlying dimension reflecting ‘patient experience’ with high internal consistency.

The Kidney PREM Short Form consists of 15 items across the same 13 themes to enable renal centres to measure patient experience more frequently to inform local service improvements.

What impact this may have on practice or policy?

The Kidney PREM is suitable for national collection of reliable patient experience data that people living with CKD consider relevant and meaningful. It thereby has the potential to guide local and national initiatives to improve patients’ experiences with renal services in secondary care in the UK and in other countries.

The Kidney PREM has been adopted by the UK Kidney Association as part of the national audit scheme for renal services. It will support the identification of areas of care where there is unwarranted variation within and between renal centres and where there may be room to improve patient experience.

INTRODUCTION

Patient-reported experience has become a recognized aspect of quality of care for people with chronic kidney disease (CKD) [1, 2]. Experience refers to how people perceive their interactions with healthcare professionals and facilities [1, 3]. Measuring kidney patient experience as part of performance monitoring and audit schemes supports patient-centred approaches to evaluating and improving the quality of renal services [3-6]. In other clinical areas, patient experience has been linked to the quality of care processes and to outcomes [7, 8]. In CKD, an enhanced patient experience is associated with improved treatment adherence and outcomes [9-11].

Robustly measuring patient experience requires validated questionnaires to elicit feedback on topics such as communication with the healthcare team, information provision, social and emotional support and coordination of care [2, 3]. Reviews [2, 12] identified two validated, dialysis-specific patient-reported experience measures (PREMs): the Consumer Assessment of Healthcare Providers and Systems (CAHPS) questionnaire for in-centre haemodialysis (HD) [13, 14] and the Consumer Quality Index for in-centre HD and peritoneal dialysis (PD) [15, 16]. In addition, Sanabria-Arenas et al. [17] reported the development and validation of a Scale for Evaluation of Haemodialysis Patients’ Satisfaction (ESUR-HD).

Currently no PREMs have been developed and validated in other CKD populations, such as people living with a kidney transplant or those with advanced CKD not yet requiring kidney replacement therapy (KRT). Hence renal services lack the means to assess patient experience across the full range of treatment modalities. Although centres may have their own instruments to measure experience within specific services, the UK Kidney Association (UKKA) and Kidney Care UK commissioned the development of an instrument that could measure patient experience across entire service provisions, aiming to support evaluations of renal services and inform national and local quality improvement initiatives.

This study aimed to develop and validate the Kidney PREM, an instrument to measure patients’ experiences with renal services in secondary care for any CKD stage or treatment modality.

MATERIALS AND METHODS

Figure 1 illustrates our phased approach in developing the Kidney PREM (available in colour as online supplementary material). We consulted with a multidisciplinary expert group throughout to ensure face validity and inform item selection. The group included 13 patient and carer representatives, three health professionals, three methodologists and one policy maker; all were involved in a national programme promoting person-centred approaches to kidney care [18, 19].

FIGURE 1:

Flowchart of our phased development of the Kidney PREM.

Data collection across phases

In this section we describe the collection of data across all phases of Kidney PREM development. Data were integral to the evolution of the instrument, with results of each Kidney PREM version used to inform the next. We organized three annual data collections (2016–8). Each took 2 months and used the most recent Kidney PREM version then available. The UK Renal Registry (UKRR), part of the UKKA, distributed paper copies to all renal centres, focusing on England in 2016 and extending to Wales, Northern Ireland and Scotland from 2017. All CKD patients were attending outpatient clinics in secondary care settings and had responded to the Kidney PREM following an invitation from their secondary care provider. They comprised a heterogenous group, including those living with advanced CKD (CKD stages 3B, 4 and 5), those with less advanced disease requiring management of conditions such as glomerulonephritis, connective tissue disease, vasculitis and other immune-mediated diseases, and those with progressive conditions such as diabetic nephropathy and adult polycystic kidney disease. Patient guidance stated that the Kidney PREM was anonymous and would take ∼15 min to complete, assisted, if required, by a friend or family member. From 2017, the PREM was also available online, translated into Welsh (facilitated by the Welsh Renal Network) and Gujarati and Urdu [20]. Paper copies returned to the UKRR were scanned into electronic format and merged with online responses into one dataset. Responses with two or fewer items completed were rejected as scanning errors.

Phase 1: development of the Kidney PREM preliminary version

The initial version (i.e. preliminary version 1; Supplementary data, Table S1) consisted of 43 candidate items across 13 themes, selected with reference to other validated and non-validated PREMS [13, 15, 21–24] identified through an informal review of published and grey literature. The expert group commented on relevance, completeness, wording and response scales. We evaluated the initial version's item response profile and internal consistency using the 2016 data and content validity through cognitive interviews. Test–retest reliability was evaluated using a Welsh dataset collected for that purpose. Findings from phase 1 informed changes to the preliminary version 1, resulting in the Kidney PREM preliminary version 2.

Item response profile and exploratory factor analysis. We evaluated each item's response profile by calculating its mean, median and the percent missing and ‘not applicable’ values. To assess internal consistency, we conducted an exploratory factor analysis with Varimax rotation. Eigenvalues and scree plots were used to determine the number of underlying dimensions of patient experience. We considered the following widely accepted [25] fit statistics and thresholds to identify items whose removal might improve internal consistency: factor loading (<0.4), uniqueness (<0.4), item-test and item-rest correlation (<0.5), average interitem covariance (<0.15) and any increase in the dimension's Cronbach's α after item removal. Candidate items for removal were discussed during the cognitive interviews.

Test–retest reliability. To evaluate the reliability of preliminary version 1 over time, we conducted a test–retest exercise with patients from five renal centres in Wales who did not take part in the 2016 data collection. Three weeks after completing a first Kidney PREM, participants were asked to complete a second, together with a questionnaire to identify significant changes in health, treatment or service delivery since initial completion (Supplementary data, Table S2). For each dimension identified in exploratory factor analysis, we calculated test and retest mean scores and the correlation between them, excluding responses with >20% of items missing. Dimensions with correlation coefficients (r) <0.7, indicating poor reliability [26], were explored in the cognitive interviews.

Cognitive interviews. We conducted cognitive interviews to further improve content validity [27]. We invited 13 patients from four geographically convenient renal centres, using purposive sampling to achieve variation in age, ethnicity and modality [28]. The interview guide included all Kidney PREM items, focusing on those with high proportions of missing or ‘not applicable’ responses and those with suboptimal fit statistics in the exploratory factor analysis. We asked participants to respond to each item while verbalizing their thoughts, with the researcher using concurrent and immediate retrospective probing to explore potential problems with item structure and presentation (e.g. negative wording and framing) [29]. We audio recorded and transcribed all interviews before coding and analysing the transcripts to identify general, as well as item- and response-specific issues.

Phase 2: development and validation of the Kidney PREM final version

Phase 2 aimed at improving the internal consistency of preliminary version 2 by reducing the number of items, using the 2017 data. This resulted in the Kidney PREM final version, for which we confirmed internal consistency using the 2018 data.

Item response profile, exploratory factor analysis and interitem correlation. To identify candidate items for removal and revision, we analysed items’ response profiles and conducted an exploratory factor analysis to evaluate their fit statistics. We used an approach similar to phase 1, adding high item mean and low standard deviation (SD; indicating limited response variation) and discrimination (>4.5; indicating participants’ tendency to disproportionately use the scale's upper response options) as criteria for nominating items for removal. For each theme we also investigated interitem correlations, considering an r >0.85 to indicate item redundancy [25]. We presented suggestions for item removal to the expert group, who selected items for inclusion in the Kidney PREM final version, aiming to retain at least one item per theme to preserve face validity.

Confirmatory factor analysis and sensitivity analyses. To confirm the robustness and internal consistency of item selection, we conducted a confirmatory factor analysis with the 2017 data. To evaluate model fit, we primarily considered the comparative fit index (CFI) as the most informative for large sample sizes [30]. Secondary model fit statistics included χ2, the root mean square error of approximation, the standardized root mean square residual and item's standardized expected parameter change (SEPC; indicating the magnitude of change if relaxing the constraint of variance). We compared a model where items were not allowed to covary with models where, within a theme, any pair of items with a modification index (MI) >100 was allowed to covary.

As some items only applied to certain patient groups (i.e. related to needling, blood tests and transport), we undertook a series of sensitivity analyses to ensure that group selection did not influence the instrument's internal consistency. For this, we repeated the confirmatory factor analysis while excluding items not applying to all participants.

Confirming internal consistency in a new dataset. We conducted a principal component analysis using the 2018 data to confirm the stability of the final version's internal consistency [30]. We used Cronbach's α as a global statistic of internal consistency, further investigating individual item statistics if α was <0.7.

Phase 3: development of the Kidney PREM Short Form

We developed a shorter version of the Kidney PREM to facilitate quicker and more frequent measurement to inform local service improvements. Items for the short form were selected from items in the final version of the Kidney PREM. To identify representative items in each theme in the final version we selected those with significant covariance based on the MI and SEPC model fit statistics. We then assessed the fit of five models with different representative questions per theme, comparing them with the fit of the model resulting from the phase 2 confirmatory factor analysis. Using this information, the expert group selected the model that represented an optimal balance between model fit and face validity.

RESULTS

Participants in the annual data collections

Table 1 shows the characteristics of participants in the three annual data collections, who together contributed 32 959 responses. Coverage of centres was 39/52 in England in 2016, 56/57 in England and Wales in 2017 and all 71 UK centres in 2018. The majority of participants were 56–74 years of age (centre range 38.4–43.9%), with good representation of those ≥75 years (centre range 25.9–28.8%); male (centre range 52.1–53.6%); and white (centre range 74.2–74.6%), with 15.3–16.5% being of minority ethnic heritage. In 2017 and 2018, <1% of valid responses were received through translated questionnaires. Most participants (centre range 76.4–81.5%) were on some form of KRT; only 6.0% were not in 2016, increasing to 15.2% (2017) and 12.0% (2018).

Table 1.

Participants' characteristics across phases

	Annual data collections
Characteristic	2016	2017	2018	Welsh data collection[a]
Total, n (%)	8162	11 027	13 770	296
Renal centres, n	39[b]	56[c]	71[d]	5
Age (years)
≤30	325 (4.0)	353 (3.2)	391 (2.8)	6 (2)
31–55	1910 (23.4)	2797 (25.4)	3095 (22.5)	41 (14.5)
56–74	3134 (38.4)	4731 (42.9)	6042 (43.9)	122 (43.1)
≥75	2110 (25.9)	2902 (26.3)	3970 (28.8)	114 (38.5)
Missing	683 (8.4)	244 (2.2)	272 (2.0)	13 (4.4)
Gender, n (%)
Male	4250 (52.1)	5907 (53.6)	7295 (53.0)	179 (60.5)
Female	3652 (44.7)	4031 (36.6)	4891 (35.5)	91 (30.7)
Rather not say	0 (0.0)	39 (0.4)	57 (0.4)	0 (0.0)
Missing	260 (3.2)	1050 (9.5)	1527 (11.1)	19 (6.4)
Ethnicity, n (%)
Asian	–[e]	1048 (9.5)	1275 (9.3)	–[e]
Black	–[e]	774 (7.0)	829 (6.0)	–[e]
White	–[e]	8184 (74.2)	10 267 (74.6)	–[e]
Other	–[e]	265 (2.4)	355 (2.6)	–[e]
Rather not say	–[e]	155 (1.4)	202 (1.5)	–[e]
Missing	–[e]	601 (5.5)	842 (6.1)	–[e]
Current treatment, n (%)
PD	738 (9)	808 (7.3)	982 (7.1)	19 (6.4)
HD	4433 (54.3)	6194 (56.3)	8834 (64.2)	244 (82.4)
Transplant	1070 (13.1)	1545 (14.0)	1399 (10.2)	1 (0.3)
CKD not on KRT	493 (6.0)	1671 (15.2)	1659 (12.0)	4 (1.3)
Missing	1428 (17.6)	809 (7.3)	896 (6.5)	28 (9.5)
HD location, n (%)
At home	226 (5.1)	276 (4.5)	299 (3.4)	3 (1.2)
In centre	3353 (75.6)[f]	2671 (43.1)	3843 (43.5)	179 (73.4)
Satellite	–[f]	3036 (49.0)	4412 (49.9)	32 (13.1)
Missing	854 (19.3)	211 (3.4)	280 (3.2)	30 (12.3)

Data collection for the test-retest exercise conducted in phase 1.

The total number of renal centres in England was 52.

The total number of renal centres in England and Wales was 57.

The total number of renal centres in England, Scotland, Wales and Northern Ireland was 71.

eNo data collected for this characteristic.

A distinction between in-centre and in-satellite patients was not made in 2016.

Preliminary versions of the Kidney PREM (phase 1)

Item response profile, factor structure and test–retest reliability. The item response profile of the preliminary version 1 showed that most items (35/42) had means of at least 3 on a scale of 1–4 with SDs <1. This implied that participants tended to use the upper two (of four) response options, which constrained response variance. Only items 34 and 37 had median scores of 2. Both were framed negatively, perhaps indicating that their direction confused participants. A total of 16 items had a ‘not applicable’ option. Around 10% of participants selected this option for 11 of these items. In most cases this suggested that items were relevant only for some patient groups (e.g. transport). In others, it may have indicated that participants did not know how to respond. Negatively framed items, those with high ‘not applicable’ responses and those with >10% missing, were selected for further exploration in the cognitive interviews. Supplementary data, Table S3 contains the response profiles and fit statistics for all items.

Exploratory factor analysis suggested one underlying factor (i.e. dimension) reflecting the construct of ‘patient experience’ with high internal consistency (Cronbach's = α, 0.904), encompassing 13 themes. Three additional factors had eigenvalues >1, but these were >10 points smaller than the eigenvalue for the first factor (Supplementary data, Table S4).

Table 1 shows the characteristics of the 296 test–retest participants. The mean total scores across all 42 items were 139.4 (SD 19.3) and 140.7 (SD 20.1) for the test and retest, respectively (r = 0.79), and 8.8 (SD 1.4) and 9.0 (SD 1.3) (r = 0.72) for the overall experience item. This implied high test–retest reliability. Reliability remained high in participants reporting a significant change or event between the test and retest compared with those who did not (Supplementary data, Tables S2 and S5) and where correlation was lower, the question (1e) related more closely to patient experience than to an event unrelated to experience (Supplementary data, Table S6).

Content validity and changes from Kidney PREM preliminary version 1 to 2. From four centres, 12 patients took part in cognitive interviews. Ages ranged from 21 to >75 years. Three were of minority ethnicity. Seven were on in-centre or home HD and five on other types of KRT or attending low clearance clinics. Supplementary data, Table S7 presents detailed findings from the cognitive interviews.

Table 2 summarizes the main changes to the Kidney PREM preliminary version 1 resulting from phase 1. These included response scales increased from 4 to 7 points with labels only for extreme response options (addressing constrained response variance), ‘not applicable’ and ‘don't know’ added as response options for all items, negative items (34 and 37) were reverse scored so they aligned with other items (higher scores implying better experience), HD vascular access cannulation added as a theme, 1 item removed and 8 added and 24 items revised to improve clarity. Some items with suboptimal response profiles and fit statistics (e.g. transport) were retained because patients considered them essential to the experience. This resulted in the Kidney PREM preliminary version 2 (Supplementary data, Table S8), consisting of 50 items across 13 themes.

Table 2.

Summary of changes from Kidney PREM preliminary version 1 to 2, informed by findings from phase 1

	Preliminary version 1			Preliminary version 2
Theme	Theme number	Number of items[a]	Reasons for item removal/rewording	Theme number	Number of items[b]
Changes to themes and items
Access to the renal team	2	3	Items do not correlate with other items (r = 0.48–0.54). Cognitive interviews: Items 6 and 7 should be would you feel able to… (6) …contact the unit from home, and (7) …make an additional appointment, rather than (6) encouraged to… and (7) allowed to….	1	3
Support	3	5	Items do not correlate with other items (r = 0.42–0.56). Cognitive interviews: Use ‘issue’ rather than ‘problem’ in items 9 and 10; re-order the items to consider medical, then practical, then other issues; change items on sources of support to reflect where patients get support from; remove item on networks.	2	4
Communication	4	4	Items do not correlate with other items (r = 0.35–0.53). Cognitive interviews: Add item on communication between you and your renal team; add an item on communication from your GP to your renal team; some items (15 and 16) are not applicable to all.	3	6
Patient information	5	4	Cognitive interviews: Misunderstanding of items 19 and 10 about format and timing of information. Change items to focus on patient needs; questions on format and timing is not applicable to all.	4	4
Fluid intake and diet	6	2	Lower reliability in the test–retest analysis. Cognitive interviews: Not relevant to all, particularly low-clearance and transplanted patients. Double question (clear/consistent). Substantial change, from two items: (21) Does the renal team give you the advice you want on your diet and fluid intake? (22) Is that advice clear and consistent? to six items: (18) Does the renal team give you the advice you want on your fluid intake? (19) Is that advice clear? (20) Is that advice consistent? Repeated for diet (21–23).	5	6
Needling			New theme. Cognitive interviews: Significance of needling to the dialysis experience raised by in-centre HD patients. Added item from CAHPS (How often do the renal team insert your needles with as little pain as possible?)	6	1
Tests	7	3	Cognitive interviews: Tests do not always need to be explained (items 23 and 25); for those using PatientView (i.e. the national kidney patient portal), item 24 is not always applicable. All three items are not applicable to some patients; change the focus to whether patients understand tests and receive results on time.	7	3
Sharing decisions	8	4	High levels of missing response [mean 702 per item (629–771)]. Cognitive interviews: Some misunderstanding of ‘goals’ and ‘encourage’ in items 28 (support you with setting and reviewing the goals that are important to you) and 29 (encourage you to take a more active role in managing your own kidney care). Change items 28 and 29 to ‘talk to you’ about goals and taking an active role.	8	4
Privacy and dignity	9	2	Cognitive interviews: Consider plain English working. Removed ‘sufficient’ from item 30.	9	2
Scheduling and planning	10	3	Items do not correlate with other items (r = 0.09–0.53); patients respond to items 32 and 34 very differently compared with other items. Lower reliability in the test-retest analysis. Cognitive interviews: Item 32 (convenience of blood tests) varies according to whether bloods are taken by the GP, phlebotomy or whilst on dialysis. Item 34 includes a response scale reversal (Is time wasted? ‘never’ is the positive option). Several changes: introduced a filter to the blood test item (only applies to non-dialysis patients). Item on wasted time reversed to be positive; ‘consultation’ focus removed.	10	3
How the renal team treats you	1	4	Items do not correlate well (r = 0.56–0.61) and overlap with other items. Cognitive interviews: Items have a ‘feel good’ factor. Moved from being the 1st to 11th theme (starting with a general item about the team may have introduced positive response bias). Some additional changes to clarify meaning. Added emotional support item.	11	5
Transport	11	3	Higher proportions of ‘not applicable’ and missing responses for items 35–37 (51%, 34% and 42%; and 667, 792 and 833). The theme responses have a consistently different pattern to the rest of the instrument. Lower reliability in the test retest. Changed to emphasize focus on hospital transport only, added a filter.	12	3
Environment	12	5	The theme responses have a consistently different pattern to the rest of the instrument. Greater missing responses in items on accessibility, parking and waiting areas (items 38, 41 and 42; 643, 1323 and 637).	13	5
Your overall experience	13	1	High proportion of responses for 8/9/10 of 10 for overall experience (85.6%). Cognitive interviews: Patients always justified their high scores. The response scale (‘very poor’ to ‘excellent’) was changed to provide more extreme end points and to provide a more consistent description (‘worst it can’ be to ‘best it can be’).	14	1
Changes to other aspects
Introductory text			Cognitive interviews: Common for patients to miss the introductory text. Make the introductory text more prominent.
Missing responses			Cognitive interviews: There are a number of items where ‘not applicable’ or ‘don't know’ is a genuine option. See ‘scale options’ below.
Scale options			Cognitive interviews: Positive answers tend to be ‘yes’ with the decision then being ‘usually/always’. Negative answers tend to be assigned ‘never’ without consideration of ‘seldom’. The scale is 4 point with never–always or poor–excellent descriptors, and a 10-point scale on the overall experience item. ‘Not applicable’ is an option for 12 items; ‘don't know’ an option for four items. Consider a 7 point scale as preferable to 4 point, with ‘don't know’ and ‘not applicable’ as standard.

GP, general practitioner.

The total number of items in the initial version was 43.

The total number of items in the preliminary version was 50.

Kidney PREM final version (phase 2)

Items response profile and factor structure. The response profile and fit statistics of most items in the preliminary version indicated no major issues. As in phase 1, some had high proportions of ‘not applicable’ or missing responses (e.g. transport and parking) because they did not apply to all patient groups. This affected their fit statistics. All had factor loadings ≥0.4 (Supplementary data, Table S9), were considered important by the expert group and were retained to preserve face validity.

Exploratory factor analysis confirmed the single ‘patient experience’ dimension, which again had high internal consistency (Cronbach's α = 0.977) and accounted for 78% of the observed variance.

In seven themes (e.g. diet, fluid, patient information, communication, etc.), there was a high correlation between items (Supplementary data, Table S10). Overall, we removed 11 and reworded 2 items (Table 3). This led to the Kidney PREM final version, consisting of 38 items in 13 themes and 1 overall experience question. The instrument is freely available at https://ukkidney.org/sites/renal.org/files/KQuIP/PREM%20paper_11.20.pdf, with further information on analysis and use at https://ukkidney.org/kidney-patient-reported-experience-measure. A copy of the Kidney PREM is also included in Supplementary data, Table S2.

Table 3.

Summary of changes from preliminary version 2 to the Kidney PREM final version, informed by findings from phase 2

Theme	Items preliminary version 2	Items removed/reworded	Reasons for removing/rewording items	Items final version
Access to team	1–3	None	N/A	1–3
Support	4–7	Question 5 removed	Items 4, 5 and 6 correlated highly. Item 5 was considered to be the least characteristic of the theme.	4–6
Communication	8–13	Question 10 removed	Items 8 and 9 and 10 and 11 correlated highly. Items 10 and 11 were judged to be indistinguishable. Item 10 was selected for removal.	7–11
Patient information	14–17	Questions 16 and 17 removed	All items correlated highly. All items had high discrimination. Items 16 and 17 had highest correlation and discrimination, and were judged to contribute least to the theme.	12–13
Fluid and diet	18–23	Questions 18 and 20 and 21 and 23 removed; questions 19 and 22 reworded	Items were very highly correlated within the Fluid and Diet themes, but less so between themes, indicating that one item in each theme would be sufficient. Item 19 and 22 were retained but reworded.	14–15
Needling	24	None	N/A	16
Tests	25–27	None	N/A	17–19
Sharing decisions	28–31	Question 30 removed	All items correlated highly, with the highest correlation between item 29 and 30 (r = 0.92). Item 30 was removed.	20–22
Privacy and dignity	32–33	None	N/A	23–24
Scheduling and planning	34–36	None	N/A	25–27
How the renal team treats you	37–41	Questions 37 and 38 removed	Items 37, 38 and 39 were very highly correlated. Items 37 and 38 had the highest discrimination. To preserve face validity, item 39 was retained. Items 37 and 38 were removed.	28–30
Transport	42–44	None	This theme had the poorest statistics. However, as the expert panel considered transport a particularly important issue, these items were retained to preserve face validity.	31–33
Environment	45–49	None	N/A	34–38
Overall experience	50	None	N/A	39

N/A, not applicable.

Confirmation of internal consistency. Confirmatory factor analysis of the final version showed a moderate model fit if items were not allowed to covary (CFI 0.753). Relaxing this constraint improved model fit substantially (CFI >0.9). Sensitivity analysis confirmed that model fit was not affected by the removal of items or themes that did not apply to all patient groups, with CFIs all remaining >0.9 if items were allowed to covary (Supplementary data, Table S11).

Table 4 displays response profiles and fit statistics of all items in the final version resulting from the principle component analysis conducted using the 2018 data. The analysis confirmed high internal consistency (Cronbach's α = 0.94) with no need for further change.

Table 4.

Response profile and fit statistics of items in the Kidney PREM final version (phase 2, 2018 data collection, N = 13 559)

		Item response profile					Item fit statistics
Themes and items[a]	n b	Mean (SD)[c]	Median[c]	D/K (%)[d]	N/A (%)[d]	Missing (%)[e]	Factor loading	Uniqueness	Item-test correlation	Item-rest correlation	α if removed[f]
Access
1 Does team answer question	13 336	6.2 (1.5)	7	197 (1.5)	166 (1.2)	223 (1.6)	0.662	0.561	0.642	0.609	0.940
2 Comfortable to contact unit	13 345	6.3 (1.5)	7	252 (1.9)	149 (1.1)	214 (1.6)	0.536	0.713	0.556	0.518	0.940
3 Ask for additional appointments	13 341	5.9 (2.1)	7	852 (6.4)	135 (1.0)	218 (1.6)	0.501	0.749	0.527	0.489	0.941
Support
4 Help with medical issues	13 282	6.0 (1.9)	7	573 (4.3)	490 (3.7)	277 (2.0)	0.616	0.621	0.635	0.601	0.940
5 Help with practical issues	13 085	5.6 (2.3)	7	817 (6.2)	685 (5.2)	474 (3.5)	0.495	0.755	0.541	0.503	0.941
6 Help accessing support	13 028	4.7 (3.3)	6	2748 (21.1)	2065 (15.9)	531 (3.9)	0.440	0.806	0.459	0.417	0.941
Communication
7 You and team	13 358	6.3 (1.3)	7	87 (0.7)	46 (0.3)	201 (1.5)	0.726	0.473	0.724	0.697	0.939
8 Members of team	13 258	6.2 (1.6)	7	333 (2.5)	67 (0.5)	301 (2.2)	0.667	0.555	0.645	0.613	0.940
9 GP and team	13 242	4.5 (2.7)	5	2433 (18.4)	169 (1.3)	317 (2.3)	0.444	0.803	0.450	0.407	0.941
10 Team and other medical	13 250	4.9 (2.7)	6	2377 (17.9)	313 (2.4)	309 (2.3)	0.505	0.745	0.501	0.461	0.941
11 Team and non-medical	13 189	5.0 (3.6)	6	2390 (18.1)	3956 (30.0)	370 (2.7)	0.403	0.838	0.406	0.363	0.942
Patient information
12 Explain easy	13 297	6.3 (1.3)	7	105 (0.8)	59 (0.4)	262 (1.9)	0.754	0.432	0.740	0.716	0.939
13 Enough information	13 240	6.2 (1.6)	7	212 (1.6)	108 (0.8)	319 (2.4)	0.760	0.422	0.743	0.718	0.939
Fluid
14 Clear advice	13 283	6.2 (1.8)	7	231 (1.7)	816 (6.1)	276 (2.0)	0.592	0.650	0.583	0.548	0.940
Diet
15 Clear advice	13 280	6.0 (1.9)	7	228 (1.7)	657 (4.9)	279 (2.1)	0.591	0.651	0.585	0.550	0.940
Needling
16 Insert needles with little pain[g]	9972	6.5 (2.2)	7	208 (2.1)	2649 (26.6)	3587 (26.5)	0.342	0.883	0.345	0.300	0.942
Tests
17 Understand reasons for tests	13 036	6.1 (1.7)	7	442 (3.4)	146 (1.1)	523 (3.9)	0.594	0.648	0.578	0.543	0.940
18 Acceptable time to get results back	13 022	5.8 (2.0)	7	715 (5.5)	217 (1.7)	537 (4.0)	0.588	0.654	0.579	0.545	0.940
19 Understand the results	12 986	5.7 (2.0)	7	610 (4.7)	199 (1.5)	573 (4.2)	0.597	0.643	0.573	0.537	0.940
Sharing decisions
20 Talk with you about treatment and goals	13 078	5.3 (2.3)	6	450 (3.4)	538 (4.1)	481 (3.5)	0.687	0.528	0.678	0.649	0.939
21 Enable you to participate in decisions	13 085	5.5 (2.2)	6	593 (4.5)	439 (3.4)	474 (3.5)	0.699	0.512	0.686	0.658	0.939
22 Talk with you about self-management	13 051	5.3 (2.5)	6	1112 (8.5)	998 (7.6)	508 (3.7)	0.631	0.602	0.613	0.580	0.940
Privacy and dignity
23 Are you given privacy	13 218	6.2 (1.6)	7	113 (0.9)	135 (1.0)	341 (2.5)	0.675	0.544	0.646	0.615	0.940
24 Is dignity respected	13 193	6.5 (1.3)	7	138 (1.0)	141 (1.1)	366 (2.7)	0.639	0.591	0.615	0.582	0.940
Scheduling and planning
25 Change appointment times	13 142	5.5 (2.4)	7	1489 (11.3)	393 (3.0)	417 (3.1)	0.399	0.841	0.424	0.381	0.942
26 Is time used well	13 072	6.1 (1.7)	7	353 (2.7)	203 (1.6)	487 (3.6)	0.632	0.600	0.622	0.589	0.940
27 Blood tests convenient[h]	7899	6.5 (1.8)	7	202 (2.6)	743 (9.4)	5660 (41.7)	0.387	0.851	0.447	0.406	0.941
How the renal team treat you
28 Take you seriously	12 955	6.4 (1.4)	7	125 (1.0)	82 (0.6)	604 (4.5)	0.683	0.533	0.667	0.637	0.939
29 Show caring	12 964	6.5 (1.1)	7	61 (0.5)	75 (0.6)	595 (4.4)	0.699	0.512	0.677	0.648	0.939
30 Ask about feelings	12 873	5.3 (2.4)	6	393 (3.1)	608 (4.7)	686 (5.1)	0.586	0.657	0.596	0.562	0.940
Transport[i]
31 Suitable vehicle	8312	6.80 (1.8)	7	71 (0.9)	1971 (23.7)	5247 (38.7)	0.387	0.850	0.426	0.386	0.941
32 Time to travel acceptable	8149	6.3 (2.1)	7	57 (0.7)	1493 (18.3)	5410 (39.9)	0.432	0.813	0.461	0.422	0.941
33 Can leave within 30 min	8148	5.4 (2.76)	6	110 (1.4)	1536 (18.9)	5411 (39.9)	0.381	0.855	0.425	0.385	0.941
Environment
34 Accessibility	12 960	6.5 (1.3)	7	74 (0.6)	596 (4.6)	599 (4.4)	0.495	0.755	0.468	0.427	0.941
35 Comfort	12 965	6.1 (1.3)	7	25 (0.2)	125 (1.0)	594 (4.4)	0.615	0.622	0.605	0.571	0.940
36 Cleanliness	12 994	6.5 (1.0)	7	27 (0.2)	96 (0.7)	565 (4.2)	0.594	0.647	0.576	0.541	0.940
37 Waiting area	13 008	6.2 (1.4)	7	57 (0.4)	187 (1.4)	551 (4.1)	0.570	0.676	0.555	0.519	0.940
38 Parking	12 787	5.8 (2.7)	7	366 (2.9)	3000 (23.5)	772 (5.7)	0.239	0.943	0.207	0.158	0.943
Overall experience
39 Overall experience of the service	13 252	6.3 (1.04)		–[j]	–[j]	320 (2.4)	–[k]	–[k]	–[k]	–[k]	–[k]

D/K, don't know.

For a full description of each item, see Kidney PREM final version: https://ukkidney.org/sites/renal.org/files/KQuIP/PREM%20paper_11.20.pdf.

The number of analysable responses for each item, which included the number of N/A and D/K responses.

Response scale was 1 (never)–7 (always).

The percentage of N/A and D/K responses is given as the percentage of analysable responses for that item.

Missing responses are those for which no response of any kind was recorded. The percentage of missing responses is presented as the percentage of total returned responses (N = 13 559).

Cronbach's α for the patient experience dimension was 0.904. Values in this column indicate the instrument's Cronbach's α if removing the item.

Question 16 is applicable only for people receiving HD.

Question 27 is not applicable for people receiving in-centre/in-satellite haemodialysis.

Transport theme is applicable only for those using hospital transport.

No recorded data.

Item fit statistic does not apply to this item.

Kidney PREM Short Form (phase 3)

We assessed four alternative versions of the Kidney PREM Short Form. All had an excellent model fit with CFIs >0.9 (Supplementary data, Table S12). We selected the model with the strongest face validity in consultation with the expert group (Table 5).

Table 5.

Items selected for the Kidney PREM Short Form

Item number Kidney PREM
Short form	Final version	Theme	Item description
1	1	Access to the renal team	Does the renal team take time to answer your questions about your kidney disease and treatment?
2	5	Support	Does the renal team help you to get the support you want with any other concerns of anxieties resulting from your kidney disease or treatment?
3	7	Communication	Do you think there is good communication between you and your renal team?
4	12	Patient information	Does the renal team explain things to you in a way that is easy for you to understand?
5	14	Fluid	Does the renal team give you clear advice on your fluid intake?
6	15	Diet	Does the renal team give you clear advice on your diet?
7[a]	16	Needling	How often does the renal team insert your needles with as little pain as possible?
8	17	Tests	Do you understand the reasons for your tests?
9	21	Sharing decisions about your care	Does the renal team enable you to participate in decisions about your kidney care as much as you want?
10	24	Privacy and dignity	Is your dignity respected during visits and clinical examinations?
11	26	Scheduling and planning	Do you feel your time is used well at your appointments relating to your kidneys?
12	28	How the renal team treats you	Thinking about how the renal team treats you, do they take you seriously?
13[a]	32	Transport	Is the time it takes to travel between your home and the renal unit acceptable to you?
14	35	Environment	When you attend the renal unit, how would you grade comfort?
15	39	Overall experience	How well would you grade your overall experience of the service provided by your renal unit on a scale from 1 (worst it can be) to 7 (best it can be)?

This item only applies to people on in-centre/satellite unit HD.

DISCUSSION

Summary of findings

We developed the Kidney PREM using a phased approach involving 32 959 responses across three annual data collections, informed by cognitive interviews and a multidisciplinary expert group to ensure content and face validity. The instrument is suitable for assessing the experiences of people with CKD within secondary care renal services, regardless of disease stage and treatment modality. It consists of 38 items, contributing to a single underlying dimension reflecting the construct of ‘patient experience’ with high internal consistency. Items are grouped into 13 themes and rated on a 7-point scale, with higher scores indicating better experience. The Kidney PREM Short Form consists of 15 items, across the same 13 themes, allowing quicker and more frequent measurement to inform local service improvement.

Relation to other studies

The Kidney PREM differs substantially from other established PREM measures used in the renal context [13, 15, 17] despite both CAHPS [13] and the CQ Index [15] informing its development. These instruments were developed in different healthcare systems, have different target populations and specify involvement with different professional staff groups. The Kidney PREM refers to the whole spectrum of experience of CKD patients, regardless of modality, in the context of the whole renal team, across themes including communication, information provision, scheduling and tests. The PREM has a single underlying dimension, with all items pertaining to the construct of ‘patient experience’. This is in contrast with the CAHPS and CQ Index, which divide patient experience across 3 and 10 dimensions, respectively. Hence the Kidney PREM provides an integral measure of patient experience compared with the more fragmented versions offered by the CAHPS and CQ Index.

Strengths and limitations

The Kidney PREM was validated using almost 33 000 responses across all UK renal centres, making it the largest study internationally in terms of participating patients and centres. It enables assessing experiences of all patients using renal services in secondary care, capturing the broadest possible range of experience within one instrument. Service users have contributed to its development from the start.

It has not been validated for non-English speakers, a limitation most pertinent for South Asian patients and perhaps for those of Eastern European and Somali origin. Although our data had good representation of people of Black and other minority ethnicities, the PREM's cross-cultural validity remains to be assessed. The absence of information on non-responders is a further limitation, leaving patient-level response rates unclear. In addition, underrepresented groups, including those with low literacy levels and/or poor health, may be poorly represented or even missing altogether. Exploring non-response patterns will be important to address escalating health inequalities.

Some treatment modalities were underrepresented, with a significantly higher proportion of people on HD responding than expected based on UKRR data. As data are only recorded for people receiving KRT, we cannot determine the true patient population composition. However, Kidney PREM collection for 2020 [31] was entirely online; this resulted in a proportional increase in participation by people with a kidney transplant, albeit still underrepresented. All treatment groups had a sufficiently large number of responses such that differences between modalities could be explored during almost all phases of Kidney PREM development. However, differences by treatment modality appeared marginal, especially in compared with differences between domains and treating centres. This strongly suggests that the Kidney PREM measures patients’ experiences of the service rather than the characteristics of the patient. The test–retest reliability statistics were similar among patients who did or did not report a significant event between tests, largely irrespective of event type. This provided further evidence that the Kidney PREM measures experiences of the service rather than people's experiences of their condition or circumstances.

The Kidney PREM is prone, similar to other scales of this kind, to a positive response bias, often due to the need to project a socially acceptable face or a social desirability bias [32]. We addressed this phenomenon to our best ability by modifying the response anchors to discourage extreme response options and eliminating items where positive response bias was extreme, indicating limited discrimination. Although most Kidney PREM responses on the 7-point scale still ended up in the range of 5–7, the mean scores for treatment centres and domains did vary, and in some cases were <5. There was also a small but significant number of patients who provided negative responses in the range of 1–3. This gives confidence that the ceiling effect is at least partly accounted for. Data collected during the coronavirus disease 2019 pandemic gave additional confidence that the scale is sensitive to change [31], providing evidence that scale scores are capable of discrimination.

Implications for practice

The Kidney PREM was developed as a driver for change. It is supported by UK professional and patient organisations and deployed by the renal community to inform person-centred care [33]. Kidney PREM data are available to units as a national report with analysis at the centre level, with local data available to unit clinical directors (via an interactive portal since 2019). Analysis by theme and patient characteristics can facilitate initiatives relevant to local priorities.

Kidney PREM data have also been used to compare patient experiences across UK renal centres. While our study has shown some evidence of positive response bias (i.e. clustering of responses towards the scale's higher end), subsequent analyses have demonstrated sufficient response variance to reveal significant differences between centres and across some patient characteristics [34]. Of the 13 Kidney PREM themes, transport, vascular access needling and shared decision making have been consistently identified as areas where patient experience varies unwarrantedly within and between centres. These are areas where patient experience in the UK could be improved.

Although it was developed in the UK, the Kidney PREM is suitable, with limited adaptation, for use in other countries with similar health delivery systems. Kidney PREM themes were selected to represent key service delivery indicators with relevance beyond local and national contexts. The instrument can be adapted without substantially affecting its measurement properties as long as key meaning is retained, e.g. by aligning wording of items to local terminology or by removing items not applicable to a particular healthcare context. Our sensitivity analysis showed a limited impact on internal consistency of such changes. Using the Kidney PREM may be more challenging in settings where healthcare delivery has a different cultural context and where access to healthcare is limited.

Future research

The Kidney PREM Short Form is currently used to inform local service improvement initiatives. To be useful in research, future studies should confirm its internal consistency in an external dataset. This would also allow assessment of its convergent validity by comparing short form scores with Kidney PREM data collected as part of the annual national audit.

To be useful in guiding kidney service improvement, the Kidney PREM (full and short form) must be responsive to change. Analysis has shown evidence that the Kidney PREM is able to detect differences between renal centres [34]. Results from 2020 [31], the year of the global pandemic, provide further evidence of change between years and between groups of patients. However, further research is needed to establish the link between changes in service delivery and changes in Kidney PREM scores.

CONCLUSION

We developed and validated a stable instrument to robustly measure patient experience in people receiving secondary care for CKD, regardless of disease stage and treatment modality. The full instrument has been adopted by the UKKA to collect reliable patient experience data as part of the national audit of renal services. Kidney PREM data can be used to guide local and national initiatives to improve patients’ experiences of renal services in the UK and elsewhere. Ultimately this will contribute to improving CKD care and outcomes.

Click here for additional data file.