Characteristics of service users and provider organisations associated with experience of out of hours general practitioner care in England: population based cross sectional postal questionnaire survey.

OBJECTIVE: To investigate the experience of users of out of hours general practitioner services in England, UK.
DESIGN: Population based cross sectional postal questionnaire survey.
SETTING: General Practice Patient Survey 2012-13. MAIN OUTCOME MEASURES: Potential associations between sociodemographic factors (including ethnicity and ability to take time away from work during working hours to attend a healthcare consultation) and provider organisation type (not for profit, NHS, or commercial) and service users' experience of out of hours care (timeliness, confidence and trust in the out of hours clinician, and overall experience of the service), rated on a scale of 0-100. Which sociodemographic/provider characteristics were associated with service users' experience, the extent to which any observed differences could be because of clustering of service users of a particular sociodemographic group within poorer scoring providers, and the extent to which observed differences in experience varied across types of provider.
RESULTS: The overall response rate was 35%; 971,232/2,750,000 patients returned surveys. Data from 902,170 individual service users were mapped through their registered practice to one of 86 providers of out of hours GP care with known organisation type. Commercial providers of out of hours GP care were associated with poorer reports of overall experience of care, with a mean difference of -3.13 (95% confidence interval -4.96 to -1.30) compared with not for profit providers. Asian service users reported lower scores for all three experience outcomes than white service users (mean difference for overall experience of care -3.62, -4.36 to -2.89), as did service users who were unable to take time away from work compared with service users who did not work (mean difference for overall experience of care -4.73, -5.29 to -4.17).
CONCLUSIONS: Commercial providers of out of hours GP care were associated with poorer experience of care. Targeted interventions aimed at improving experience for patients from ethnic minorities and patients who are unable to take time away from work might be warranted. © Warren et al 2015.

Introduction

Provision of out of hours general practitioner (GP) care in England encompasses the time between 6 30 pm and 8 am on weekdays and throughout weekends and bank holidays.1 Over the past 10 years, there have been organisational changes in the provision of out of hours GP care, coinciding with the publication of National Quality Requirements,2 which were subsequently updated.3 Out of hours GP services are targeted on health problems that cannot wait until the next working day and are not intended as an alternative route to healthcare for non-urgent problems for those patients who are unable to attend their practice during working hours either because of their personal circumstances, such as work commitments, or because of inability to access in hours GP services at a convenient time. There is evidence, however, that some patients might use out of hours services for complaints not deemed urgent by GPs.4 5 A patient contacting an out of hours GP service would usually initially speak to a trained call handler, who, though not always clinically qualified, can advise on self care or recommend further management.

In the United Kingdom, a range of different types of organisations provide out of hours GP services (“providers”), including NHS providers, not for profit providers (for example, social enterprises), and commercial healthcare providers.6 This contrasts with the position in some other European countries. In Denmark, for example, out of hours primary care is provided by local GPs, who provide telephone triage on a rota basis.7 8 In the Netherlands care is provided through large scale GP cooperatives,9 and patients receive telephone triage from a physician assistant or nurse, with GP supervision.8 In the UK, different providers can cover the same geographical area with regard to call handling and provision of clinical care, especially since the phased introduction of the NHS 111 service, completed in February 2014.10 Whereas there is a paucity of evidence regarding the clinical performance of different types of out of hours provider, recent data indicated that the average cost per case was virtually the same across organisational type, with commercial providers seeing the fewest cases per head.10 In recent years, there has been concern regarding variation in quality of out of hours GP care in England.6 The Care Quality Commission (CQC) developed a rating system for providers,11 with the aim of ensuring that they all satisfy a minimum benchmark. It has recently conducted inspections of out of hours GP services, finding most service provision to be of high quality, but with some areas where improvements could be made.12

The English General Practice Patient Survey (GPPS13) invites service users (aged 18 or over, who can be patients themselves or people contacting an out of hours GP service on behalf of someone else) to report their use of primary healthcare services and to evaluate a wide range of aspects of their experience of such services.14 We have previously reported on sociodemographic variation in the experience of in hours primary care services.15 Evidence regarding the basis of variation in service users’ evaluations of out of hours GP services, however, is also required, as the organisations providing such services vary substantially, as do the sociodemographic characteristics of their service users. The CQC is also using the GPPS to monitor service users’ experience of out of hours GP care. Focus on the reasons underlying variation in service users’ experiences of care has the potential to inform policy and service developments aimed at improving experience for all patients, as well as those groups of service users currently reporting less positive experience of out of hours GP care.

We investigated potential variations in experience of service users across individual level sociodemographic factors to establish the extent to which any variation in experience was related to “clustering” of service users reporting poorer experience within providers reporting poorer performance overall. We also determined whether the type of provider organisation was associated with service user experience, evaluated the extent to which provider type explained variation between providers in service user experience, and established how difference in provider type could potentially influence the centile rating of an equivalent (with regard to sociodemographic composition) NHS or commercial provider relative to not for profit provider organisations. We considered three aspects of service user experience: timeliness of provision of care, confidence and trust in the out of hours clinician, and overall experience of out of hours GP care.

Methods

Patient questionnaires

We analysed data collected by the English GPPS (from July to September 2012 and January to March 2013), which achieved an overall response rate of 35% (971 232 patients returned a questionnaire out of 2 750 000 patients invited to complete it).13 The questionnaire13 included four evaluative questions on out of hours provision, three of which were considered here: timeliness of receiving care (“about right,” “took too long,” or “don’t know/doesn’t apply”); level of confidence and trust in the out of hours clinician (“yes, definitely,” “yes, to some extent,” “no, not at all,” or “don’t know/can’t say”); and overall experience of the out of hours GP service (5 point Likert scale from “very good” to “very poor”). These questions were asked only of service users who had attempted to contact an out of hours GP service within the preceding six months and were related to aspects of care delivered directly by the provider. The fourth evaluative question captured data regarding service users’ experience of contacting the service by telephone. We did not analyse these data because of difficulties we encountered in mapping individual GP practices to the organisation providing telephone contact. Furthermore, the questionnaire survey took place at a time when the system for out of hours telephone care was in a transition phase, coinciding with the phased national introduction of the NHS 111 service. We were thus not able to ascertain for individual practices and service users whether they would be contacting an out of hours provider through the NHS 111 helpline or through the previous method of using a telephone number to directly contact an out of hours GP provider.

Service user characteristics

We included six sociodemographic variables in the analyses; five of these were taken from the questionnaire responses: sex (male as reference); ethnic group (white as reference, based on five categories derived from Office for National Statistics categorisations16); age in eight categories (18-24 as reference), parent status (non-parent as reference); and whether the service user was able to take time away from work to attend his/her practice during working hours (the reference category was service users who reported that they were not in paid work: “not applicable”). Service users who were doing paid work were divided into two categories based on whether they could or could not take time away from work during working hours. The sixth sociodemographic variable, deprivation (in fifths, based on national quintiles; least deprived as reference), was determined from the respondent’s residential postcode, mapped to the corresponding lower super output area Index of Multiple Deprivation 2010.17

Practice and out of hours GP service provider issues

Each service user was mapped to a specific provider that was responsible for providing clinical care on behalf of the service user’s practice during the six month period before we sent the questionnaire. We were able to map 934 931 service users (96% of an original dataset of 971 232 service users from 7886 practices) to a provider. We identified 91 providers for provision of clinical care; 86 of these had an identifiable type of provider organisation (not for profit (reference), NHS, or commercial). We determined each provider’s organisational type by information from the provider’s website, other web based sources, or by direct contact with the provider.

Statistical methods

Sociodemographic data are described for all service users who reported attempting to contact an out of hours GP provider in the previous six months (for themselves or on behalf of another person). To facilitate comparison between measures on different scales, we linearly rescaled outcomes from 0-100.15 On this scale, a difference of <3 points is considered small in magnitude with regard to practical importance.18 Responses of “don’t know/doesn’t apply” were considered as missing and excluded from analyses, as were questions that were not answered. We assumed that responses of service users mapped to the same provider would be “clustered” by provider but not by practice, as clinical care provision by the provider was in no way mediated by the service user’s practice, hence we used random effects for providers rather than practices. For analyses of sociodemographic associations with service user evaluations, we included service users only if they could be mapped to an out of hours provider with known organisational type, had provided data on all sociodemographic variables included within the model, and provided a response for the question being investigated.

We used three statistical models in the analyses. The first (model A) comprised a fixed effect multivariable linear regression model including individual sociodemographic factors as covariates and generated mean differences in outcome scores for comparator sociodemographic groups compared with reference categories, without adjustment for differences in outcome across providers. The second (model B) was a mixed effects model that extended model A by incorporating a random intercept for provider, thus adjusting for differences in outcome between providers, and estimated the mean difference between comparator and reference group in outcome scores within providers. A comparison between models A and B informed the extent to which any overall difference between service users of specific sociodemographic groups was caused by clustering of those service users within providers that had low scores overall.15

Model C extended model B by adding provider type as a provider level covariate. Thus, we estimated the effects of provider type, with adjustment for the sociodemographic characteristics of individual service users, for each outcome. Comparison of the variance between providers derived from models B and C informed the degree of this variation attributable to provider type. For each aspect of care, and for each provider type, we derived 95% central ranges for the difference between individual provider mean scores and the overall mean score for not for profit providers, around the overall mean difference in scores between that provider type and not for profit providers, using the standard deviation between providers from model C.15 The effect of provider type was expressed in terms of the standardised mean difference (derived by dividing the mean difference between type of comparator provider and not for profit providers by the standard deviation between providers derived from model C, analogous to an effect size such as Cohen’s d). We thus determined the strength of association of provider type with the overall score and the centile that would be achieved by a provider type (compared with not for profit) with an equivalent sociodemographic profile, by approximating the observed distribution of the adjusted scores from 86 providers to a standard normal distribution.19 Analyses were performed with Stata MP v.12.

Patient involvement

Service users were not involved in the question formulation or design of these secondary analyses of GPPS data. We were provided with anonymised data by Ipsos MORI, and therefore we cannot directly disseminate results to questionnaire respondents.

Results

Table 1 shows the sociodemographic characteristics of 106 513 service users (from 7492 practices) who reported that they had attempted to contact an out of hours provider on their own behalf or for someone else in the preceding six months and were mapped to a practice with known provider organisation type. Service users’ overall evaluations of out of hours GP services were generally positive. Table 2 shows the raw frequencies for responses to all three evaluative questions (for all service users mapped to a provider with known organisation type). While 71% (73 983/103 523) of participants reported a “very good” or “fairly good” overall experience, 31% (31 966/104 145) reported that they felt it took too long to receive care from the out of hours GP service.

Table 1

 Sociodemographic characteristics of service users who had attempted to contact out of hours GP provider (on their own behalf or for someone else) within past six months and whose practice was mapped to provider of known type of organisation. Figures are numbers (percentage) of respondents

Sociodemographic characteristics*	No of people
Sex:
Men	38 553 (36.6)
Women	66 879 (63.4)
Total	105 432
Age (years):
18-24	4850 (4.6)
25-34	14 745 (14.0)
35-44	20 066 (19.0)
45-54	18 699 (17.7)
55-64	16 760 (15.9)
65-74	14 704 (13.9)
75-84	11 201 (10.6)
≥85	4509 (4.3)
Total	105 534
Ethnic group:
White	90 034 (85.5)
Mixed/multiple ethnic groups	860 (0.8)
Asian/Asian British	7985 (7.6)
Black/African/Caribbean/black British	2471 (2.4)
Other ethnic group	3934 (3.7)
Total	105 284
Deprivation fifth:
1 (least deprived)	19 537 (18.4)
2	20 672 (19.4)
3	21 633 (20.3)
4	21 486 (20.2)
5 (most deprived)	23 028 (21.7)
Total	106 356
Parent/guardian of children aged under 16:
No	61 276 (62.8)
Yes	36 277 (37.2)
Total	97 553
Can you take time away from work to see GP during your typical working hours?†:
Not relevant†	51 027 (51.3; NA)
Yes	31 298 (31.5; 64.7)
No	17 057 (17.2; 35.3)
Total	99 382
Total relevant	48 355

GP=general practitioner; NA=not applicable.

*Service users mapped to 86 out of hours GP providers through 7492 practices. 106 513 service users attempted to contact an out of hours GP provider in the six months preceding completion of the General Practice Patient Survey; sociodemographic characteristics are reported where available.

†First percentage is percentage of total service users who provided data; second percentage is percentage of total service users for whom this question was relevant (working service users).

‡Service users reported not doing paid work (for example, retired, unemployed, full time student).

Table 2

Timeliness of care, confidence and trust in out of hours clinician, and overall experience of care: raw scores. Figures are numbers (percentage) of respondents

Question	Response frequency*
How do you feel about how quickly you received care from out of hours GP service?
It was about right	65 298 (62.7)
It took too long	31 966 (30.7)
Don’t know/doesn’t apply	6881 (6.6)
Total	104 145
Did you have confidence and trust in out of hours clinician you saw or spoke to?
Yes, definitely	42 264 (40.7)
Yes, to some extent	42 938 (41.3)
No, not at all	12 222 (11.8)
Don’t know/can’t say	6490 (6.3)
Total	103 914
Overall, how would you describe your experience of out of hours GP services?
Very good	33 662 (32.5)
Fairly good	40 321 (39.0)
Neither good nor poor	15 638 (15.1)
Fairly poor	8140 (7.9)
Very poor	5762 (5.6)
Total	103 523

GP=general practitioner.

*Includes all service users mapped to out of hours GP provider with known organisation type even if complete demographic data not available.

Of the 86 providers with confirmed organisational type, 44 were not for profit organisations, with 21 providers in each of the NHS and commercial sectors. There was evidence that type of provider organisation was associated with all three outcomes (global P<0.001 for trust and confidence in the attending clinician and for overall experience of care; global P=0.013 for timeliness of care). There was no evidence of differences between NHS organisations and not for profit organisations with regard to any of the outcomes. There was, however, evidence that commercial providers scored lower than not for profit organisations for all three individual outcomes (table 3). The magnitude of these differences was about 3 points (model C) for all outcomes.

Table 3

 Associations between type of provider and timeliness of out of hours GP care, confidence and trust in out of hours clinician, and overall experience of out of hours GP care

Provider type*	Mean difference† (95% CI)	P value‡	Standardised mean difference	Equivalent centiles compared with not for profit providers on 25th, 50th, and 75th centiles§
Timeliness of out of hours GP care
Model C¶** (86 providers; 83 176 service users; SD between providers 5.19:
NHS	1.28 (−1.61 to 4.17)	0.013	0.25	34, 60, 82
Commercial	−3.52 (−6.40 to −0.64)		−0.68	9, 25, 50
Confidence and trust in out of hours clinician
Model C¶** (86 providers; 83 316 service users; SD between providers 3.14):
NHS	1.00 (−0.79 to 2.79)	<0.001	0.32	36, 63, 84
Commercial	−3.25 (−5.03 to −1.46)		−1.04	4, 15, 36
Overall experience of out of hours GP care
Model C¶** (86 providers 88 423 service users; SD between providers 3.33):
NHS	1.07 (−0.77 to 2.90)	<0.001	0.32	36, 63, 84
Commercial	−3.13 (−4.96 to −1.30)		−0.94	5, 17, 39

GP=general practitioner.

*Models included 44 not for profit providers, 21 NHS providers, and 21 commercial providers; not for-profit providers are reference group for all analyses.

†All outcomes linearly rescaled from 0 to 100.

‡P value refers to global effect of covariate across all groups v reference group.

§Estimated using standard normal distribution to derive centiles for NHS and commercial providers that would be equivalent to not for profit providers on the 25th, 50th, and 75th centiles (assuming the same sociodemographic profile).

¶Models also adjusted for age, sex, ethnic group, deprivation fifth, parent status, and ability to take time away from work during work hours.

**Random effect on out of hours GP provider organisation.

A comparison of the variance between providers derived from model B and that derived from model C indicated that, for timeliness of care, 11.3% of the variability of model B was attributable to type of provider (tables 3 and 4 ). Equivalent proportions for confidence and trust in the out of hours clinician, and for overall experience of care were 20.9%% (tables 3 and 5) and 18.6% (tables 3 and 6 ), respectively. Across the three provider organisation types, the 95% central ranges for the difference between individual provider mean scores and the overall mean score for not for profit providers overlapped considerably for all three outcome measures, but the 95% central range for commercial providers was less positive than those for not for profit and NHS providers (figure). The standardised mean difference for commercial provider type compared with not for profit was −0.68 SD for timeliness of care, −1.04 SD for confidence and trust in the out of hours clinician, and −0.94 SD for overall experience of out of hours GP care (table 3), indicating a moderate (timeliness) or large (confidence and trust, and overall experience) effect size attributable to commercial provider type.

Table 4

Timeliness of care from out of hours GP services: linear regression modelling

Sociodemographic covariate	Model A* (83 176 service users), overall difference			Model B*† (86 providers; 83 176 service users); difference within out of hours providers; SD between providers 5.51		Percentage of overall difference (if negative) attributable to clustering of sociodemographic group in lower scoring providers
	Mean difference‡ (95% CI)	P value§		Mean difference‡ (95% CI)	P value§
Ethnic group¶:
Mixed	−4.78 (−8.34 to −1.23)	<0.001		−3.45 (−6.99 to 0.09)	<0.001	28
Asian	−13.27 (−14.51 to −12.03)			−11.08 (−12.37 to −9.79)		17
Black	−7.64 (−9.86 to −5.42)			−5.67 (−7.92 to −3.42)		26
Other	−8.44 (−10.24 to −6.64)			−6.57 (−8.40 to −4.75)		22
Able to take time away from work during typical working hours**:
Yes	3.45 (2.62 to 4.27)	<0.001		3.48 (2.65 to 4.30)	<0.001	Not applicable
No	−6.58 (−7.56 to −5.61)			−6.48 (−7.45 to −5.51)		2

GP=general practitioner.

*Models also adjusted for age, sex, deprivation fifth, and parent status.

†Random effect on out of hours GP provider organisation.

‡All outcomes linearly rescaled from 0 to 100.

§P value refers to global effect of covariate across all categories v reference category.

¶Mixed: mixed/multiple ethnic groups; Asian: Asian/Asian British; black: black/African/Caribbean/black British; other: other ethnic group; reference group: white.

**Reference group (not relevant) includes service users who reported one of following options in response to question “Which of these best describes what you are doing at present?”: full time education; unemployed; permanently sick or disabled; fully retired from work; looking after home; doing something else.

Table 5

 Confidence and trust in out of hours clinician: linear regression modelling

Sociodemographic covariate	Model A* (83 316 service users), overall difference			Model B*† (86 providers; 83 316 service users); difference within out of hours providers; SD between providers 3.53		Percentage of overall difference (if negative) attributable to clustering of sociodemographic group in lower scoring providers
	Mean difference‡ (95% CI)	P value§		Mean difference‡ (95% CI)	P value§
Ethnic group¶:
Mixed	−3.02 (−5.58 to −0.46)	<0.001		−1.72 (−4.27 to 0.84)	<0.001	43
Asian	−5.95 (−6.85 to −5.05)			−3.92 (−4.86 to −2.99)		34
Black	−2.62 (−4.22 to −1.02)			−0.33 (−1.95 to 1.29)		88
Other	−1.18 (−2.48 to 0.13)			0.87 (−0.46 to 2.19)		>100
Able to take time away from work during typical working hours**:
Yes	2.24 (1.64 to 2.84)	<0.001		2.23 (1.63 to 2.82)	<0.001	Not applicable
No	−5.35 (−6.05 to −4.64)			−5.27 (−5.97 to −4.57)		1

GP=general practitioner.

*Models also adjusted for age, sex, deprivation fifth, and parent status.

†Random effect on out of hours GP provider organisation.

‡All outcomes linearly rescaled from 0 to 100.

§P value refers to global effect of covariate across all categories v reference category.

¶Mixed: mixed/multiple ethnic groups; Asian: Asian/Asian British; black: black/African/Caribbean/black British; other: other ethnic group; reference group: white.

**Reference group (not relevant) includes service users who reported one of following options in response to question “Which of these best describes what you are doing at present?”: full time education; unemployed; permanently sick or disabled; fully retired from work; looking after home; doing something else.

Table 6

 Overall experience of out of hours GP services: linear regression modelling

Sociodemographic covariate	Model A* (88 423 service users), overall difference			Model B*† (86 providers; 88 423 service users), difference within out of hours providers, SD between providers 3.69)		Percentage of overall difference (if negative) attributable to clustering of sociodemographic group in lower scoring providers
	Ethnic group¶:
Mixed	−3.44 (−5.47 to−1.41)	<0.001		−2.01 (−4.03 to 0.01)	<0.001	42
Asian	−5.61 (−6.32 to −4.90)			−3.62 (−4.36 to −2.89)		35
Black	−2.14 (−3.40 to −0.89)			0.13 (−1.14 to 1.40)		>100
Other	−0.75 (−1.78 to 0.27)			1.29 (0.25 to 2.32)		>100
Able to take time away from work during typical working hours**:
Yes	1.30 (0.82 to 1.78)	<0.001		1.29 (0.81 to 1.76)	<0.001	Not applicable
No	−4.79 (−5.36 to −4.23)			−4.73 (−5.29 to −4.17)		1

GP=general practitioner.

*Models also adjusted for age, sex, deprivation fifth, and parent status.

†Random effect on out of hours GP provider organisation.

‡All outcomes linearly rescaled from 0 to 100.

§P value refers to global effect of covariate across all categories v reference category.

¶Mixed: mixed/multiple ethnic groups; Asian: Asian/Asian British; black: black/African/Caribbean/black British; other: other ethnic group; reference group: white.

**Reference group (not relevant) includes service users who reported one of following options in response to question “Which of these best describes what you are doing at present?”: full time education; unemployed; permanently sick or disabled; fully retired from work; looking after .home; doing something else.

Timeliness of out of hours care, confidence and trust in out of hours clinician, and overall experience of out of hours GP services, by provider organisation type. Central lines (within bar) represents difference in overall mean scores between each provider type (not for profit, NHS, commercial) and not for profit providers (reference). Bars represent 95% central ranges for difference between individual provider mean scores and overall mean score for not for profit providers. All outcomes analysed on 0-100 scale

To illustrate this further, we estimated the centiles for commercial and NHS providers that would be equivalent to those for not for profit providers at the 25th, 50th, and 75th centiles, based on the standard normal distribution (table 3). There were substantial differences in centiles across the provider types. For example, with regard to overall experience of care, a not for profit provider on the 50th centile would equate to the 17th centile if provider type was changed to commercial.

There was evidence that service users of mixed ethnicity and Asian ethnicity reported poorer care for all three evaluative questions than white respondents; a more variable pattern of care was evident for service users of black ethnicity and other ethnicity (tables 4-6). The magnitude of the differences, however, varied across the three questions. For example, in model B, with adjustment for service provider, the mean difference between Asian and white service users was −11.08 (95% confidence interval −12.37 to −9.79) with regard to timeliness of care but only −3.92 (−4.86 to −2.99) with regard to trust and confidence in the out of hours clinician and −3.62 (−4.36 to −2.89) for overall experience of care. In general, the mean differences in scores between white service users and service users from the mixed, black, and other ethnic groups tended to be of lower magnitude that those between Asian and white service users.

A comparison of models A and B indicated that with regard to timeliness, only 17% of the mean difference in scores between Asian and white service users derived from model A (−13.27, 95% confidence interval −14.51 to −12.03; table 4) was because of clustering of Asian service users within providers that scored lower overall (versus 28%, 26%, and 22% in mixed, black and other ethnicity services users, respectively). For overall experience of care, 35% of the mean difference between Asian and white service users (−5.61, −6.32 to −4.90; model A, table 6) was attributable to clustering of Asian service users within a lower scoring provider. Much or all of the mean differences in score between black/other ethnicity service users compared with white service users, with regard to confidence and trust in the out of hours clinician and overall experience of care, could be attributed to clustering within poorer scoring providers; however, in these cases the mean differences were of small magnitude (model A, tables 5and 6), accounting in part for this observation.

Service users who were unable to take time away from work to attend their practice reported lower mean scores across all three outcomes than service users for whom this was not an issue, whereas service users who were able to take time away from work reported higher mean scores (tables 4and 6).

Other individual level sociodemographic characteristics (sex, age, deprivation fifth, and parent status) were also associated with the three outcomes measures (deprivation was associated only with trust and confidence in the out of hours clinician and with overall experience of care), but we did not further explore the effects of these characteristics either because of the small magnitude of mean differences compared with the relevant reference category or because of more positive scores in the comparator category (assumed to be potentially more disadvantaged) than in the reference category.

Discussion

Summary

In this study, commercial provider organisations were associated with poorer reports of out of hours GP care across all three outcome measures compared with not for profit organisations, such that a not for profit provider on the 50th centile would equate to the 25th centile or lower if provider type was changed to commercial.

Additionally, service users from ethnic minority groups tended to report less favourable care than white service users. This finding was in part attributable to clustering of ethnic minority service users in out of hours providers with lower overall scores; however, clustering in this manner was not itself the major driver of lower scores for individual ethnic groups. Inability to attend the practice because of work commitments was also significantly associated with lower scores across all three aspects of care.

Comparison with previous literature

The poorer scoring associated with commercial providers is in keeping with observations on patients’ experience from the United States, which indicated that for profit hospitals were associated with worse patient experience than non-profit hospitals.20 21 Those patient surveys, however, were based on inpatient care and not on provision of out of hours GP care, hence comparability with this setting could be limited. The reasons for lower scores for commercial versus not for profit organisations in these analyses are unclear. The lower scores could reflect a genuine poorer experience of care provided by commercial providers or the willingness of commercial providers to provide care in areas deemed less attractive to the NHS or not for profit organisations, despite the fact that these differences persist after adjustment for individual level sociodemographic variables. Finally, a further consideration could be that service users’ perceptions of different types of provider influence their reported experience of care, although we believe that service users are mostly unaware of who provides their out of hours GP care or the type of organisation. In areas where there has been media attention related to a local out of hours GP provider, however, service users might have greater awareness of the identity and nature of their provider.

Previous analyses of GPPS data with regard to in hours care have indicated that ethnic minority patients reported generally lower experience scores15 and that patients of different ethnic backgrounds might differ with regard to drivers of satisfaction.22 Early research into patient satisfaction with out of hours GP care has indicated that white patients reported greater overall satisfaction with their care compared with patients of other ethnicities.23 In our analyses, Asian service users reported lower mean scores than white service users for all three experience outcomes; the greatest negative difference between Asian and white service users was observed with regard to timeliness of care. Similar differences for timeliness of care were seen for service users of other ethnic groups but of a lesser magnitude, suggesting that service users from ethnic minorities and Asian service users in particular, place substantial value on the timeliness of out of hours care. The ability of an out of hours GP service to meet the expectations of its users has been presented as a strong driver of satisfaction with care.24 While analysis of GPPS data cannot shed any light on service users’ expectations in this context, it remains possible that variations in expectations of care might have some bearing on differences between ethnic groups in experience of out of hours GP care—for example, because of differences in cultural attitudes regarding an acceptable waiting time. There is some ecological evidence to indicate that poor timeliness of care (objectively measured) is associated with reduced satisfaction with care received.1 Hence, if patients from ethnic minority backgrounds perceive the timeliness of care to be poor, they might tend to report worse experiences of care.

Service users who were unable to take time away from work to attend their practice reported poorer experience on all three outcomes compared with users not in paid work. In contrast, individuals who reported being able to take time off work for health related reasons reported somewhat better experiences of care. These work related effects were not related to clustering of individuals within lower scoring providers. Whether these effects are related to case mix effects (difference in type and/or severity of complaint, possibly resulting from delayed care because of the service user’s inability to attend his/her practice during regular hours) or to other reasons cannot be determined from our data. It remains possible that expectations of service users might vary with work status; hence, out of hours providers who do not provide routine “non-urgent” care might be less able to meet the expectations of service users who cannot take time to attend their practice during regular hours, thus resulting in poorer scores.

Strengths and limitations

The English GPPS is a large survey. For most practices we were able to map the practice to a specified out of hours GP provider and also to determine the organisational type of the provider. Such an approach has not previously been reported in the UK literature on patient experience of out of hours GP care. This approach facilitated evaluation, in a series of sophisticated models based on a large sample of service users, of the association between provider organisation type and service user sociodemographic characteristics and service users’ reports of care received.

Several limitations were evident in respect of the data available from the GPPS. No data were available regarding the nature of the service user’s complaint, the time or date of the contact, or how the contact was managed. Current (on completion of the survey) health status data were collected as part of the GPPS but could not be related to the out of hours data as the precise date of the out of hours contact was not known. We consider, however, that the lack of objective data regarding an individual service user’s out of hours contact does not diminish the usefulness or validity of our results as they are based on service users’ reports of their care. Although data on ethnicity were collected, the GPPS did not ask about service users’ English language ability, nor about educational attainment, both of which might be related to experience of care.25

Service users who were unable to take time away from work to attend their practice within working hours reported worse experiences for all three outcome measures, a finding comparable with those reported in a recent trial26 of telephone triage in primary care.27 We do not, however, have any data on the specific types of work that might be associated with inability of service users to attend their practice during working hours. Also, we would ideally want to have data available regarding the degree of urgency of the contact, as out of hours providers might simply signpost service users with non-urgent complaints to in hours services, which are difficult or impossible to access for these service users.

Service users were invited to provide feedback on their experiences of out of hours care in the preceding six months. This interval is typical for patient surveys and is believed to result in less recall bias than longer intervals, although there is evidence that patients might find it difficult to restrict their responses to the six month reference period.28 The potential for recall bias cannot be dismissed with regard to recall of the contact occurring, whether it occurred within the appropriate time period, and in reporting experience of the contact. For example, there is evidence that older patients might not accurately report use of health service resources over a relatively short time frame.29

Though we acknowledge the limitations presented by a response rate for the GPPS of 35%, no evidence of adverse association between response rate and non-response bias has been found for the GPPS. Previous research has indicated that the use of rigorous probability sampling methods (as adopted in the GPPS) is consistent with only a weak association between non-response rates and non-response bias.19 30 31 An analysis of GPPS data collected in 2009 indicated that practice level patient response rate was only weakly associated with variation in practice level score for questions related to patient access,32 and an analysis of data on out of hours care in the Netherlands suggested that non-response bias was small with respect to overall satisfaction with out of hours care.9 Also, we have no objective data regarding whether an individual questionnaire respondent did, in fact, attempt to contact the out of hours GP service within the relevant time period, therefore we were unable to ascertain the extent of individual item “missingness” with regard to the out of hours questions within the overall questionnaire. Limited granularity in response options regarding whether the service user was able to take time away from work, and timeliness of care, restricted our ability to interpret these data.

Despite these limitations, policy makers have proposed changes in service provision on the basis of findings from the GPPS, and survey derived metrics are adopted within the key national metrics of UK healthcare provision.33 The National Audit Office and CQC are using GPPS out of hours items to monitor the performance of such services. Therefore, we believe the findings presented here are relevant and important to service users (patients, parents, and carers), healthcare professionals and commissioners, policy makers, out of hours providers, and individuals and organisations planning, funding, or undertaking related research.

Future research

The observation that commercial providers were associated with poorer experience of out of hours GP care is a new finding, which warrants further exploration. While some insights might be gained from an understanding of patient sociodemographic differences at the level of the provider, such data are not presently available. In addition, other relevant factors, such as user awareness of the provider type, and others yet to be identified, might also be important in the interpretation of service users’ reports of their experience of care. Future research, possibly involving qualitative approaches, is required to investigate reasons for generally lower scores from service users from ethnic minority backgrounds and to investigate reasons why service users who were unable to take time from work to attend their practice during regular hours reported poorer scores across all three evaluative questions. Finally, as for in hours GP care,15 investigation of the extent to which any discrepancy between sociodemographic groups in respect of reports of care might be attributable to the clustering of servicer users belonging to sociodemographic groups reporting relatively lower scores within providers with lower overall scores would help inform the development and targeting of an intervention aimed at improving service users’ experience of out of hours GP care.

In the US, variations in patient experience of inpatient hospital care have been observed across different types of provider organisation, such as for profit or non-profit, with poorer experiences reported in for profit hospitals

Patient experience in the UK varies across sociodemographic groups for in hours GP care, but there is less evidence regarding sociodemographic variations in experience of out of hours GP care

No evaluation of patient experience based on type of out of hours GP provider organisation has yet been conducted in England

Commercial out of hours GP providers were associated with poorer reports of service users’ experience of out of hours GP care than not for profit and NHS providers

Service users who were unable to take time away from work to attend their practice during working hours reported poorer experience of care than non-working service users

Service users from ethnic minority backgrounds reported poorer experience of care, which is in part due to clustering of such service users within poorer scoring providers overall