Effect of a national urgent care telephone triage service on population perceptions of urgent care provision: controlled before and after study.

OBJECTIVE: To measure the effect of an urgent care telephone service NHS 111 on population perceptions of urgent care.
DESIGN: Controlled before and after population survey, using quota sampling to identify 2000 respondents reflective of the age/sex profile of the general population.
SETTING: England. 4 areas where NHS 111 was introduced, and 3 control areas where NHS 111 had yet to be introduced. PARTICIPANTS: 28 071 members of the general population, including 2237 recent users of urgent care. INTERVENTION: NHS 111 offers advice to members of the general population seeking urgent care, recommending the best service to use or self-management. Policymakers introduced NHS 111 to improve access to urgent care. OUTCOMES MEASURES: The primary outcome was change in satisfaction with recent urgent care use 9 months after the launch of NHS 111. Secondary outcomes were change in satisfaction with urgent care generally and with the national health service.
RESULTS: The overall response rate was 28% (28 071/100 408). 8% (2237/28 071) had used urgent care in the previous 3 months. Of the 652 recent users of urgent care in the NHS 111 intervention areas, 9% (60/652) reported calling NHS 111 in the 'after' period. There was no evidence that the introduction of NHS 111 was associated with a changed perception of recent urgent care. For example, the percentage rating their experience as excellent remained at 43% (OR 0.97, 95% CI 0.69 to 1.37). Similarly, there was no change in population perceptions of urgent care generally (1.06, 95% CI 0.95 to 1.17) or the NHS (0.94, 95% CI 0.85 to 1.05) following the introduction of NHS 111.
CONCLUSIONS: A new telephone triage service did not improve perceptions of urgent care or the health service. This could be explained by the small amount of NHS 111 activity in a large emergency and urgent care system. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

This is a large population survey undertaken to measure whether a new way of managing urgent care changed people's perceptions of urgent care.

The controlled before and after design is robust for measuring change.

It uses an innovative way of identifying recent users of urgent care and their views of the whole episode of care they experienced.

The telephone survey has a low response rate, although it is similar in all areas in both time periods.

It is possible that the lack of change was due to the insensitivity of the questionnaire to identify change, although it was developed and validated for measuring change in user perceptions of emergency and urgent care systems.

Introduction

Telephone triage plays a significant role in the delivery of healthcare internationally.1–6 In 2011, a new telephone triage service called NHS 111 was introduced in some parts of England.7 The service offers advice to members of the general population seeking urgent care. It is aimed at people having healthcare episodes where the situation is not life threatening. The service employs non-clinical call handlers to direct callers to the most appropriate service or offers self-management advice. Call handlers use computerised software to triage calls, with the option of referral to onsite clinicians to make triage decisions. To promote ease of access, the telephone number is easy to remember, and the service is available at all times.

Policymakers introduced the new service to improve access to urgent care, increase efficiency by directing people to the right place first time, increase satisfaction with urgent care and the National Health Service (NHS) generally, and in the longer term reduce unnecessary calls to the emergency ambulance service.7 England had a pre-existing nationally available nurse-led 24-hour telephone helpline called NHS Direct which also provided out of hours call handling for some general practices. NHS 111 was introduced to eventually replace this national telephone service, and this finally occurred across England in 2014. Policymakers intended that NHS 111 be an improvement over NHS Direct by offering easier and quicker access to health advice for urgent healthcare through an easier to remember number, shorter assessment times and shorter waiting times because most callers would receive advice from a non-clinical operator rather than waiting for advice from a nurse.

NHS 111 started in four areas in 2010, with roll out occurring in different areas over subsequent years. Evaluation of NHS 111 in the first 4 areas showed that although it was generally acceptable to users,8 over half the population were aware of it and 1 in 10 had called it at some point in the 9 months postlaunch,9 it did not have the expected effect of reducing use of emergency services by directing callers to urgent care services, general practice and self-care.10 Indeed, it increased the use of emergency ambulance services by a small amount. As part of a larger evaluation of NHS 111 in the first four areas,11 we undertook a controlled before and after population survey to measure its effect on perceptions of urgent care provision and the health service generally.

Methods

Design

We undertook a controlled before and after population survey in each of the four NHS 111 areas, and three control areas, prior to the launch of NHS 111 and exactly 12 months later. The population survey identified recent users of urgent care and sought their views about their most recent episode of care, as well as identifying population satisfaction with urgent care and with the NHS.

Setting

NHS 111 was established in four geographical areas, defined by primary care trusts, in England covering a total population of 1.8 million: Durham and Darlington; Nottingham; Luton and Lincolnshire. In all four areas, the service could be accessed directly by dialling ‘111’ or indirectly when patients called their general practice out of hours and were automatically rerouted to NHS 111. Over 353 000 calls were answered by NHS 111 in these four areas in the first year, with over 80% of these calls triaged. Control areas were selected to match the pilot areas in terms of population demographics, health profile and health service use.11 NHS 111 was not operational in the control areas during the study period. Three suitable control areas were selected because one area offered the best match for two NHS 111 areas: North of Tyne; Leicester; and Norfolk. During the study period, NHS 111 operated in parallel to NHS Direct, that is, intervention and control groups had access to both telephone-accessed services.

Sampling

Survey methodology which had been previously tested and validated was used.12 A market research company was engaged to undertake a telephone survey of the general population in each NHS 111 and control area before NHS 111 was launched and exactly 12 months later. The age and sex profile of each area population formed the sampling frame for quota sampling. The market research company undertook random digit dialling with 1 attempt to contact a landline telephone number, aiming to identify 2000 respondents in each of the 7 areas in each year, representative of the age/sex profile of the population. Standard market research procedures were followed to identify an adult to speak to within a household who was aged 16 and over. An adult or a child in the household was selected as the focus of the interview in line with meeting the quota sample. The surveys were undertaken in 2010, ∼1 month prior to the planned launch of NHS 111 in each area and again in 2011.

A sample size of 2000 in each area in each time period was chosen based on previous use of the questionnaire to identify recent users of the urgent care system.13 We estimated that 15% of respondents would be recent users, giving us 1200 urgent care users in the NHS 111 areas each year and 900 in the control areas. A key outcome is overall satisfaction with recent use of the urgent care system. Our previous survey identified 39% of urgent care users reporting their recent use as ‘excellent’.14 A sample size of 1065 in each time period would give us 80% power to detect a difference of 6 percentage points at the 5% level before and after the introduction of NHS 111.

Questionnaire

The validated Urgent Care System Questionnaire administered in the survey was developed based on qualitative research with users of the emergency and urgent care system and is designed to capture recent experience of the system.15 All participants were asked a screening question about whether they had sought help for an urgent health problem in the previous 3 months, sociodemographic questions, awareness and use of NHS 111 and satisfaction with urgent care and the NHS. If they had sought help urgently from health services in the previous 3 months, they were asked to complete the remainder of the questionnaire in relation to their most recent urgent health problem which includes questions on how people accessed the emergency and urgent care system, the number of services contacted in the episode of care, the services used in the episode and three domains of satisfaction: entry into the emergency and urgent care system, progress though the system and convenience of the system.14 Each of the three domains has a maximum score of 5; changes of 0.3 or more are associated with a step change in satisfaction.14

Analysis

We compared the change in perceptions in the combined four NHS 111 areas with the combined three control areas, before and after the introduction of NHS 111. We adjusted any comparisons for age group, sex and ethnicity. For the three domains of urgent care satisfaction, we undertook a linear regression with ‘entry’, ‘progress’ and ‘convenience’ as the outcome variable, adjusted for age group (16–44, 45–64, 65+), sex, ethnicity and area. We tested the interaction between pilot/control and before/after. For the categorical satisfaction variables, we undertook a logistic regression dichotomised at the ‘very satisfied’ or ‘excellent’ category and adjusted for the same variables as above. Variables were dichotomised at the highest rating versus all other ratings because previous research has shown that patients selecting this category have no complaints about the service they are offering an opinion about.16 Results were similar whether area was treated as a fixed or random effect. We report the analysis with area as a fixed effect. We used PASW V.18 for these analyses.

Results

Response rates

The overall response rate was 28% (28 071/100 408). The response rate was calculated by including all calls resulting in a completed questionnaire in the numerator, and removing from the denominator all calls where there was no one in the household who matched the remaining quota, or where the telephone number was unobtainable or engaged. Response rates were similar between the seven areas, ranging between 27% and 30%. The age/sex profiles of respondents were similar in 2010 and 2011 for each area as expected, given the use of quota sampling. The aggregated profiles for age, sex and ethnic group are given in table 1.

Table 1

Demographic profile of respondents

	Pilots (4 areas)		Controls (3 areas)
	Before% (n)	After% (n)	Before% (n)	After% (n)
Age
0–4	6 (481)	6 (452)	5 (323)	5 (325)
5–19	20 (1606)	19 (1527)	20 (1187)	20 (1147)
20–44	33 (2673)	34 (2733)	31 (1870)	32 (1891)
45–64	25 (1993)	25 (2029)	26 (1599)	26 (1591)
65+	16 (1256)	16 (1269)	18 (1061)	17 (1058)
Sex
Female	52 (4163)	51 (4085)	53 (2285)	52 (3101)
Male	48 (3846)	49 (3925)	47 (2832)	48 (2911)
Ethnicity
White	86 (6915)	87 (6993)	86 (5207)	86 (5160)
Other	14 (1094)	13 (1017)	14 (833)	14 (852)
Total (n)	8009	8010	6040	6012

Use of emergency and urgent care system

The proportion of the population seeking help for an urgent health problem in the previous 3 months was 8% (2237/28 071), varying between 6% and 11% in the different NHS 111 and control areas. This identified ∼150 recent urgent care users in each area.

Use of NHS 111 among system users

In the ‘after’ population survey in NHS 11 areas, 9% (60/652) of recent users of urgent care reported using NHS 111, varying between areas (table 2). This variation was likely to be related to the different service models in use because some models auto-routed calls from GP out of hours to NHS 111 and users were not necessarily aware that they had used NHS 111. The majority of NHS 111 users called the service as a first point of contact (table 2). That is, NHS 111 was their entry into the urgent care system for their reported episode of care.

Table 2

Use of NHS 111 among recent users of the emergency and urgent care system

	Area 1% (n)	Area 2% (n)	Area 3% (n)	Area 4% (n)	All% (n)
Any contact with NHS 111	13 (27)	2 (3)	15 (21)	6 (9)	9 (60)
First service contacted NHS 111	11 (22)	2 (3)	11 (16)	5 (8)	8 (49)
N=100% recent urgent care users	205	155	141	151	652

User satisfaction with urgent care

There was no evidence of a change in perceptions of urgent care in the NHS 111 areas compared with controls for entry, convenience or progress through the emergency and urgent care system, or for overall assessment of their experience (table 3). Approximately 43% of recent urgent care users assessed their experience as excellent in all areas at all times.

Table 3

Satisfaction with recent use of the emergency and urgent care system

	Pilots		Controls		Change in pilots compared with controls, adjusted. 95% CI	p Value
	Before	After	Before	After
Mean score ‘Entry’*	4.21	4.15	4.14	4.19	−0.05 (−0.13 to 0.03)	0.116
Mean score ‘Progress' *	4.04	3.98	4.01	4.02	−0.03 (−0.10 to 0.04)	0.327
Mean score ‘Convenience’*	3.84	3.81	3.85	3.80	−0.05 (−0.14 to 0.03)	0.605
% (n) Satisfaction overall
Excellent	43 (270)	42 (276)	43 (219)	43 (199)	0.97 (0.69 to 1.37)†	0.875
Very good	28 (171)	30 (198)	27 (137)	31 (141)
Good	17 (107)	15 (95)	16 (82)	14 (64)
Fair	6 (36)	7 (47)	6 (31)	7 (30)
Poor or very poor	6 (37)	6 (36)	7 (36)	5 (25)
N=total	621	652	505	459

*Items scored 1–5, summed and mean calculated.

†OR for % excellent.

Population satisfaction with urgent care and the NHS

There was no change in the percentage of the population reporting using urgent care in the previous 3 months, population perception of urgent care or population perception of the NHS in general in NHS 111 areas compared with control areas (table 4).

Table 4

Population use of, and satisfaction with, urgent care and the wider NHS

	Pilots		Controls		Adjusted OR (95% CI)	p Value
	Before% (n)	After% (n)	Before% (n)	After% (n)
Seeking care urgently in previous 3 months	8 (621)	8 (652)	8 (505)	8 (459)	1.15 (0.96 to 1.37)*	0.120
The way in which the NHS runs when you need to seek help URGENTLY
Very satisfied	30 (2423)	32 (2529)	33 (1993)	33 (1989)	1.06 (0.95 to 1.17)†	0.307
Quite satisfied	43 (3428)	43 (3487)	43 (2627)	43 (2566)
Neither	17 (1383)	16 (1277)	15 (879)	16 (960)
Quite dissatisfied	7 (535)	6 (499)	6 (378)	6 (339)
Very dissatisfied	3 (240)	3 (218)	3 (163)	3 (158)
The way in which the NHS runs in general
Very satisfied	29 (2337)	28 (2251)	31 (1897)	31 (1880)	0.94 (0.85 to 1.05)†	0.272
Quite satisfied	50 (4004)	51 (4126)	51 (3064)	50 (2986)
Neither	12 (976)	11 (860)	10 (636)	11 (646)
Quite dissatisfied	6 (455)	6 (520)	5 (302)	6 (358)
Very dissatisfied	3 (237)	3 (253)	2 (141)	2 (142)
N=total	8009	8010	6040	6012

*OR for % seeking care.

†OR for ‘very satisfied’ versus all other satisfaction categories.

Discussion

Introducing a new telephone triage service NHS 111 into a population appeared to have no effect on perceptions of recently used urgent care, urgent care provision generally or the wider health service. This fits with other findings of our wider study where analysis of routine data on the use of emergency ambulance services, emergency departments and urgent care centres showed a small increase for the first service and no change for the latter two services.10 It also fits with a study of the effect of the introduction of a nurse-led telephone triage service on the use of emergency and urgent care services, which also found no change on the use of services in the wider healthcare system.17 It is challenging putting this work into perspective using a wider evidence base because, even though some countries have introduced national and province-wide telephone triage services, we have not been able to find evidence of the effect of these services on population views of urgent care. Even when researchers measure the effect of wider interventions in primary and urgent care, they focus on the use of urgent care,18 or equity,19 or demand20 rather than population perceptions.

A possible explanation for the lack of change in perceptions of urgent care is that awareness levels of this new service were low in the population. This is unlikely to explain the findings because overall population awareness of NHS 111 in the four areas during the study period was high at 59%.9 Another possible explanation is that the ‘dose’ of NHS 111 represented a small amount of activity within the emergency and urgent care system. The dose was a minimum of 1 in 10 recent users of urgent care reporting using NHS 111.

Strengths and limitations

Obtaining the experiences and views of recent users of emergency and urgent care is a challenge because people can use a wide range of services, and often use two or more services for the same episode of healthcare are seeking.13 A major strength of this research was the use of a validated methodology and questionnaire to identify recent users and seek their views. Limitations include the low response rate, but this is not unusual for surveys of urgent care. For example, a recent survey of users of general practice out of hours had a response rate of 35%.21 This low response rate may have introduced non-response bias, and the sampling may have excluded some types of people (such as those who do not possess a telephone landline). However, this is unlikely to have affected the results because the same response rate was obtained in NHS 111 areas and control areas in the before and after periods. It is possible that the lack of change seen here was due to the insensitivity of the questionnaire to identify change, although it was developed and validated for measuring change in user perceptions of emergency and urgent care systems. The survey of recent users had lower statistical power than expected due to smaller percentages of recent users identified compared with our earlier survey using this questionnaire. However, there was no sign of any change over time in satisfaction with recent use of urgent care, so this lack of power did not explain our null findings. The lack of change may also be due to the small amount of NHS 111 activity in the system, although we believe we have reported the ‘minimum’ activity rather than actual activity. The lack of change observed in our study may be due to a lack of impact NHS 111 had on the emergency and urgent care system during the evaluation period. During our evaluation, NHS 111 continued to operate as an alternative service to NHS Direct. NHS Direct has since ceased to operate. While both telephone services offered different provision, it is possible that some callers to NHS Direct have now shifted to NHS 111. As a result, there may have been a substantial increase in call volumes to NHS 111. Any increases may affect the characteristics of the population using the service, and the ability of the service to maintain its principles in terms of immediate access to advice without waiting, which may then impact on satisfaction levels.

Our surveys were of pilots for NHS 111. When the service was rolled out nationally, it was provided by a wider range of service providers operating in different areas of England. Only some of these service providers were represented in the pilots. The findings may be transferable to other countries with similar emergency and urgent care configurations.

Implications

Telephone accessed healthcare is a modern addition to health services internationally and likely to be more popular with policymakers in the future as countries struggle to manage demand for emergency and urgent care. A key objective of a new telephone triage service NHS 111 was to improve perceptions of urgent care. One year after its launch, there was no evidence that NHS 111 had improved population perceptions of urgent care and this could have been because the dose of NHS 111 was small in a large urgent care system.