Role of practices and Clinical Commissioning Groups in measures of patient experience: analysis of routine data.

Introduction

Recent years have seen an increased focus on measuring how people experience health services to ensure that care and treatment is of the highest quality and safety.1 2 Better patient care experiences are associated with better adherence, clinical outcomes and patient safety, and with lower healthcare utilisation.3 4 In England, the national GP Patient Survey (GPPS) measures patients’ experience of primary care.5 As part of an NHS initiative to improve patient experience and facilitate patient-centred care, GPPS scores are currently reported at the level of general practices, Clinical Commissioning Groups (CCGs) and nationally.

English CCGs are clinically led NHS bodies that commission local health care services. There were 209 CCGs in 2016, though there are now fewer, on account of recent practice-mergers. Although GPPS scores are reported for CCGs, little is known about the influence CCGs have on patient experience scores. We have previously shown that patient experience scores vary considerably between GPs within a practice, and that measures reported at practice level can mask this variation.6 The quality of care or patient experience delivered at any one practice may be influenced by drivers from higher organisations such as CCGs. Recently, CCGs have been shown to influence the rate and accuracy of ‘fast-track’ cancer referrals from English primary care.7 Here, we explore whether there was systematic variation in the quality of patient experience between CCGs compared with variability between practices themselves.

Methods

Data were analysed from 836 172 GPPS respondents in 2015/2016 (response rate 38.9%). The GPPS survey was conducted in two waves (July–September 2015 and January–March 2016). Full details of the survey development and methodology are published elsewhere.8

Patient experience measures

We focused on seven patient experience measures which have been commonly used in previous research, and which reflect key aspects of patient experience.6 9–13 Six were based on single survey items: (1) access (difficulty making an appointment), (2) continuity of care (how often it is possible to see a preferred GP), (3) helpfulness of receptionists, (4) overall experience, (5) out-of-hours speed of advice and (6) out-of-hours overall experience. For the seventh measure, a composite GP communication score was calculated from the mean rating across five linked communication items among patients providing three or more informative responses. Responses to all items were rescaled linearly from 0 to 100 (most favourable), following previous research.6 9–11

Statistical analysis

For each outcome measure, a three-level mixed-effect linear regression model was fitted (patients nested within practices nested within CCGs). Patient age, gender, ethnicity, deprivation and presence of a long-term condition were included as fixed effects. Each model estimated three variance components: residual, practice and CCG. We focused on variances associated with CCGs and practices, and express them as percentages of their sum. From this, we can quantify the percentage of the variation in practice GPPS scores attributable to either practices or CCGs. Where the percentage associated with CCGs is high, the implication is that much of the differences in practice scores is being driven by factors aligned with CCGs. Where the percentage associated with CCGs is low, this implies that practice-level factors are dominant in predicting practice scores.

As some variability between practices and CCGs may reflect structural differences between organisations, rather than organisational policies, we ran an additional set of models adjusting for region (14 large regions covering England), rurality (rural/urban based on practice location) and the interaction between them. Comparison of the variance associated with CCGs between models with and without rurality provides some insight into the extent to which differences between CCGs reflect the geography covered by CCGs and regional differences in staffing (eg, recruitment).

Analyses were performed using R V.3.4.4.

Results

Table 1 shows the estimated variance components. Before adjusting for region and rurality, the variation in practice scores for the out-of-hours items was mostly associated with the CCG (57% for out-of-hours speed of advice and 56% for out-of-hours overall experience), an area for which CCGs have statutory responsibility. For all other items, practice score variation was mainly associated with the practice, though GP communication and overall experience had notable CCG contributions (17% and 14% of variance associated with CCGs, respectively). A considerable proportion of between-CCG variance can be explained by region and rurality. This was largest for GP communication (43%), helpfulness of receptionists and overall experience (40% for both), and smallest for continuity of care (4%).

Table 1

Variance associated with CCGs and practices for the seven GPPS outcome measures estimated with the hierarchical models

Outcome	Percentage of practice score variance associated with*		Percentage of between CCG variance associated with region and rurality†
	CCG	Practice
Access	11.8	88.2	30.3
Continuity of care	4.9	95.2	4.3
Overall experience	14.2	85.8	40.4
Helpful receptionists	8.1	91.9	40.4
GP communication	16.7	83.3	42.6
Out-of-hours speed of access	56.5	43.5	36.0
Out-of-hours overall experience	57.1	42.9	38.3

*Estimated from a model without region and rurality.

†Estimated from a model including region and rurality. The effects of rurality are inconsistent in direction and magnitude across both region and patient experience items; similar inconsistencies exist by region as permitted by the interaction term in the model.

CCG, Clinical Commissioning Group; GPPS, GP Patient Survey.

Discussion

Our findings suggest that CCGs may have a greater influence than individual practices on patient experience scores relating to aspects of care that are under the control of CCGs, such as out-of-hours care. Although CCGs were observed to contribute to other aspects of patient experience, the contribution from practices was much greater—in agreement with expectations, given that these are generally under the direct control of practices. Similar patterns of variation have previously been seen for US health plans.14 Various mechanisms exist through which CCGs may exert influence; for example, local enhanced services may have focused efforts on improving access as well as introduction of the extended hours directed enhanced service. Furthermore, at the time these data were collected, important transitional changes were taking place in the commissioning of primary care services, with transfer of responsibility progressing from centralised (NHS England) to local (CCG) commissioning, and thus with a potential local influence on patient experience. Certainly, some of the variation associated with CCGs can be explained by macro-scale structural differences such as larger region and rurality, and the larger-scale approach adopted, for example, at regional rather than at practice level in relation to configuring the delivery of out-of-hours GP services. The contribution of region and rurality is largest for measures related directly to staffing and overall experience, potentially reflecting geographical variation in the availability of high-quality staff, which has led to schemes aimed at attracting GPs to underserved areas.15

This study is not without limitations. The GPPS response rate is 39%. Although this is typical of such surveys,16 research suggests that non-response does not meaningfully affect relative performance after accounting for case mix.16 We also note that our analysis addresses the data which are publicly reported—any biases in our analysis will also be present in those publicly reported measures. Also, we have only accounted for macro-scale structural influences using large-scale proxy measures for region and rurality. There may be other factors of influence which we have not accounted for, for example, the quality of secondary care or access to particular services.

To improve patient experience, or indeed any aspect of quality, it is essential to recognise and understand which parts of the health system are responsible for, and influence, different domains of this experience.17 Just because a measure of patient experience is reported at a certain level of organisational structure does not mean that that level is the most relevant when considering impact on the actual experience of patients. This study builds on previous work and suggests that individual GPs, practices and CCGs (and newly emerging Primary Care Networks) all have a role to play, and that the magnitude of that role may be very different for different aspects of patient experience. As such, quality improvement efforts may be targeted at all levels of the system, tailored to the scope for improvement that exists. In this study, practices were shown to be associated with substantial variation in out-of-hours care experience, and on this account, improvement efforts at practice level rather than CCG level may be warranted.