Has the NHS 111 urgent care telephone service been a success? Case study and secondary data analysis in England.

OBJECTIVES: To explore the success of the introduction of the National Health Service (NHS) 111 urgent care service and describe service activity in the period 2014-2016.
DESIGN: Comparative mixed method case study of five NHS 111 service providers and analysis of national level routine data on activity and service use. SETTINGS AND DATA: Our primary research involved five NHS 111 sites in England. We conducted 356 hours of non-participant observation in NHS 111 call centres and the urgent care centres and, linked to these observations, held 6 focus group interviews with 47 call advisors, clinical and managerial staff. This primary research is augmented by a secondary analysis of routine data about the 44 NHS 111 sites in England contained in the NHS 111 Minimum Data Set made available by NHS England.
RESULTS: Opinions vary depending on the criteria used to judge the success of NHS 111. The service has been rolled out across 44 sites. The 111 phone number is operational and the service has replaced its predecessor NHS Direct. This new service has led to changes in who does the work of managing urgent care demand, achieving significant labour substitution. Judged against internal performance criteria, the service appears not to meet some targets such as call answering times, but it has seen a steady increase in use over time. Patients appear largely satisfied with NHS 111, but the view from some stakeholders is more mixed. The impact of NHS 111 on other health services is difficult to assess and cost-effectiveness has not been established.
CONCLUSION: The new urgent care service NHS 111 has been brought into use but its success against some key criteria has not been comprehensively proven. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

Unique primary study of five NHS 111 provides rich detail about service provision and wider impact, notably on Urgent Care Centres.

Complementary analysis of routine data on activity provides a higher level, national view of the service.

Together these analyses can be used to assess the success of this new model of urgent care provision.

The case study data are qualitative and not statistically generalisable, but they provide useful insights about the service.

The quantitative analysis was reliant on publicly available data and is thus limited in terms of what is collected/reported, and there are substantial missing data notably on resource use and costs. However, these data provide a consistent time series and it is unlikely that the trends we have observed are data collection or classification artefact.

Introduction

National Health Service (NHS) 111 provides telephone triage for urgent care in England. The service is free to use and is available 24 hours a day, 365 days a year across England. It is positioned at the heart of the policy vision for integrated care set out in the Five-Year Forward View (2014) and the more recent General Practice Forward View (2016).1 2

Depending on who you ask, NHS 111 is ‘a high-quality service in the face of high demand’,3 a serial killer of babies4 5 or the source of an increasing and inappropriate burden on over stretched NHS ambulance and emergency care services.6 7 These wildly contrasting views suggest that evaluating the success or otherwise of this service depends on the criteria used. Nonetheless, given the centrality of NHS 111 as the ‘front door’ for the newly proposed Clinical Hubs (which will provide clinical advice and support to patients and professionals outside hospital settings), it seems timely to consider what we have learnt about NHS 111.

NHS 111 was officially launched in February 2014 following piloting in four sites in England in 2010 (evaluated by researchers at the University of Sheffield.8 9) and the establishment of further ‘first wave’ services in the intervening years. Its predecessor, NHS Direct, previously available on a 0845 telephone number, was discontinued in March 2014, and there are extant plans to extend NHS 111 to Wales. Badged as the service ‘for when it is less urgent than 999’, NHS 111 is incorporated in a policy vision for ‘a functionally integrated 24/7 urgent care service that is the ‘front door’ of the NHS and which provides the public with access to both treatment and clinical advice’ (page 4,10) that has emerged following the Urgent and Emergency Care Review led by Sir Bruce Keogh.11

NHS 111 uses a computer decision support system (CDSS) called NHS Pathways to manage calls. Pathways is an algorithm built on an extensive library of current, regularly updated, clinical expertise combined with a real-time directory of services (DoS) available for patients who need to be seen. This software is unusual in the UK health system in that it was designed and developed by and continues to be owned and licensed by the NHS. Most other CDSS are developed and owned by private corporations. The Pathways CDSS is also licensed for, and used in UK NHS ambulance services, several of whom provide NHS 111 alongside their 999 emergency services. NHS 111 calls are answered by non-clinical staff who are supported by nurses, paramedics and general practitioners (GPs) (often one or two clinicians are available on each shift). The call handlers use the Pathways CDSS to assess accounts of symptoms, prioritise care needs and direct callers to services or self-care. Calls conclude with a ‘disposition’ which can range from sending an ambulance, arranging a home visit, booking an urgent primary care consultation or advising actions the caller can take to address their health problem.

In this paper we draw on our detailed case study of five NHS 111 sites and subsequent secondary analysis of routine data made available by NHS England to explore the success or otherwise of this service against key criteria of deployment (spread), meeting performance targets, satisfaction, impact on other services and cost-effectiveness.

Methods

We conducted a comparative case study of five English NHS 111 call centres between 2011 and 2013.12 This comprised 356 hours of non-participant observation of NHS 111 call centres and linked out-of-hours services, 6 focus groups with a total of 47 staff and key stakeholders and a staff survey (the latter is not reported here). This study followed on from an earlier study investigating the deployment of NHS Pathways software in 999 and out-of-hours call handling sites.13 Our continued interest in seeing how this service fared after our study completed led us to undertake a quantitative secondary analysis. To do this we obtained the NHS 111 Minimum Data Set (MDS) made publicly available by NHS England14 which comprises call data from the 44 NHS 111 sites in England for the period 2010–2016. We provide a descriptive analysis focused on the period February 2014–July 2016 (the latest date for which data were available) in order to examine the period beyond our case study work.

Qualitative data were analysed together using a broadly inductive method, moving from familiarisation and independent coding to team discussion and recoding and the development of emerging themes. Charting techniques were used to facilitate cross-case comparisons and to enable further discussion in the team about interpretations. Secondary analysis of the NHS MDS was conducted using descriptive and time-series functions in R.15

Results

Successful deployment?

In England, NHS 111 has expanded from the original pilots in 2010 in County Durham and Darlington, Nottingham, Lincolnshire and Luton and now operates in 44 sites across the country. So the overall picture is one of a successful roll-out of this service.

NHS 111 initially received backing from a number of key stakeholders including the British Medical Association and the Royal Colleges of Medicine (although these organisations have also periodically voiced concerns), and successfully navigated initial turbulence and technical difficulties associated with the pilot phase.16 Our research indicated that considerable effort was expended by central government policy-makers and key local stakeholders locally and nationally to smooth the deployment of the software and launch the new service, but it is now an established part of the urgent care service landscape. The local effort in terms of additional staff and additional work is captured in the focus group exchange below:

111 Manager 1: It was…it very quickly leapt from being, you know, a very short-term thing to a very long-term thing. I have a deputy that works with me, to help run the team, and I was told she was going to be seconded for  months. And 18 months later, she’s still here (laughter). So it sort of started off as this small thing, then all of a sudden, whoa, it was something much, much bigger

Interviewer: And what have been the effects of that, do you think?

UCC Nurse 1: Stress. At all levels, I think. From, you know…

111 Nurse 2: Fear… […]

111 Nurse 1: Everybody’s wearing three or four different hats and doing extra work over and above what they used to do.

111 Site 4, focus group 2

Analysis of the MDS shows that the service currently takes over 1 million calls per month (see table 1). Time-series decomposition (using the R stl() function) indicates an increasing trend in the number of calls initiated, from just over 1 million per month at launch to approximately 1¼ million by the middle of 2016 (see online supplementary figure 1).

Table 1

Call volumes, transfers/use of call back and performance against standards

	2013–2014 Q4	2014–2015 Q1	2014–2015 Q2	2014–2015 Q3	2014–2015 Q4	2015–2016 Q1	2015–2016 Q2	2015–2016 Q3	2015–2016 Q4	2016–2017 Q1	2016–2017 Q2
Observations (months) per quarter	2	3	3	3	3	3	3	3	3	3	1
Call volumes
Calls initiated*	9 71 718	1 059 104	9 58 185	1 165 760	1 110 541	1 100 726	1 010 607	1 202 030	1 366 497	1 207 808	1 238 972
Calls answered*	9 18 986	9 98 508	9 14 948	1 076 362	1 055 134	1 050 668	9 66 416	1 114 758	1 204 628	1 111 211	1 141 770
Calls per thousand of population served*	NA	19.5	17.7	21.5	20.3	20.2	18.5	22.0	24.8	NA	NA
Percentage of transfers to clinical advisors and use of call back
Calls transferred to a clinical advisor	20.2	19.9	20.7	20.1	21.3	21.1	21.5	20.6	18.9	19.7	19.6
Calls ‘warm’ transfer to a clinical advisor†	11.8	11.4	12.7	10.6	9.9	10.0	9.8	9.1	6.8	7.2	7.0
Calls resulting in a call back	8.4	8.5	8.1	9.4	11.4	11.1	11.7	11.6	12.1	12.5	12.6
Calls resulting in a call back within 10 min	4.0	4.3	4.2	4.5	5.1	5.0	5.1	4.8	4.4	5.0	4.8
Performance against standards
Percent calls answered within 60 s	94.8	93.6	95.5	87.5	92.5	93.9	93.1	88.9	77.3	88.6	88.1
Calls abandoned - over 30 s wait* (%)	11 212 (1.2)	12 834 (1.2)	8219 (0.9)	39 775 (3.4)	17 736 (1.6)	16 496 (1.5)	14 705 (1.5)	32 321 (2.7)	83 474 (6.1)	28 583 (2.4)	30 790 (2.5)

Quarters are defined according to the financial year (consistent with reporting in the MDS). Q1 = April–June; Q2 = July–September; Q3 = October–December; Q4 = January–March.

*Absolute numbers are the average per month for quarter (ie, number of calls in quarter divided by number of eligible months in the quarter). Proportions are calculated across the whole quarter. All values calculated by the authors.

†A ‘warm’ transfer is defined as situations where the call adviser determines that the call should be transferred to a clinical advisor, the call adviser speaks to the clinician and then transfers the call without any call back.

Does the service meet its own performance targets?

Service standard 3.22 in the NHS 111 Commissioning Standards,17 states that ‘[p]atient calls … should be resolved in a single contact’. Two items reported in the MDS indicate that this is not always achieved and that performance against this criterion may be declining over time. Table 1 shows that, while the proportion of calls transferred to a clinical advisor have remained relatively constant at around 20% of all calls answered, the proportion of these calls that are transferred directly to a clinical advisor (referred to as warm transfers) has declined over time (from 58% to 36%). At the same time, the overall proportion of calls resulting in a separate call back phone call has increased from 8.4% to 12.6%. Care should be exercised in interpreting these measures, as the proportion of call backs is calculated on the total number of calls answered, not on the smaller number of calls designated as ‘transferred to a clinical advisor’.

The MDS Providers Version specification18 lists two items indicating conformance with National Quality Standard 8 for Delivery of Out-Of-Hours Services (to minimise the number of abandoned calls and to ensure calls are answered within 60 s). Table 1 indicates that both these standards have been missed in at least one period since the official launch. The benchmark that 95% of calls should be answered within 60 s (adopted in MDS statistical reports) has rarely been met, while the maximum of 5% of calls being abandoned by the caller was breached during Quarter 4 of 2015/16. The service appears to be operating under particular pressure from Quarter 3 2015/16 forward—in terms of all unanswered calls, answering calls within 60 s and abandoned calls (see online supplementary figure 2 for full details).

Our observational work suggested that staff worked hard to try to meet performance targets but that this was not always possible:

The dispatcher is controlling the cars for home visits. She has to decide how urgent the call is to decide who to send where, also keeping an eye on target times according to category. If it gets close to the target and the patient hasn’t been seen she gets the clinical lead to give them a ‘comfort call’. If it becomes clear that they can’t hit targets she will look again at the urgent calls to see if it looks like any can be regraded. In fact, she says, she reviews all ‘urgents’ that come direct from call handlers and asks the clinician to take another look. I ask ‘Do they ever run out of cars?’ ‘Yes, all the time. We have to renegotiate. We use the clinical lead a lot’.

111 Site 4, Observation

Satisfaction

Patients and callers seem largely satisfied with the service. The evaluation of the pilot sites8 reported that overall satisfaction was very good, with 73% (1255/1726) of survey respondents reporting that they were very satisfied with the new service. A more recent review led by the Royal College of Paediatrics and Child Health which looked at 111 services in North West London found that 84% of people calling the helpline ‘got what they needed’, whereas 80% said they would call NHS 111 again if they had the same problem.19 Analysis of the MDS indicates that 87% of patients are very or fairly satisfied with the service (see table 2).

Table 2

Caller experience

Caller experience	April–September 2014 (n=11 572)	October 2014–March 2015 (n=13 138)	April–September 2015 (n=12 185)	October 2015–March 2016 (n=10 950)	All surveys (n=47 845)
Very satisfied with 111 experience	68.8%	68.0%	70.3%	64.6%	68.0%
Fairly satisfied with 111 experience	18.1%	19.7%	19.1%	19.7%	19.2%
Neither satisfied nor dissatisfied with 111 experience	3.8%	4.3%	4.1%	4.8%	4.3%
Dissatisfied with 111 experience*	6.1%	6.1%	5.3%	6.9%	6.1%
No view on satisfaction with 111 experience	3.2%	1.9%	1.2%	4.0%	2.5%
No response	1.4%	0.1%	1.2%	6.2%	2.1%

Percentages for responses to satisfaction questions were calculated by the authors using the sum of valid responses as denominator (rows 1 to 5 sum to 100%).

No response was calculated by the authors, subtracting the sum of recorded responses' satisfaction questions from number responding to the survey (as reported in the MDS).

*The MDS collapses the ‘fairly dissatisfied’ and ‘very dissatisfied’ categories into one.

MDS, minimum data set; n, number of people responding to the survey as reported in the MDS.

Healthcare professionals appear to have a more mixed response to questions about satisfaction with NHS 111. Early on the service was called ‘dreadful’ by GP leaders calling for a review of the structure of the service6 but recently a more optimistic view has emerged in some quarters.20 Anderson and Roland’s observational analysis7 suggested that the service would benefit from having more experienced GPs involved in triage decisions, and elsewhere it has been suggested that only one in four NHS 111 referrals to general practice are clinically appropriate,21 and this may be the cause of some dissatisfaction among primary care professionals. In 2014, an NHS England Quality and Safety Report22 concluded that there were high levels of variation in clinical governance arrangements around the quality of the DoS and the quality of communication between out-of-hours services and NHS 111, all of which might contribute to dissatisfaction among service providers. Online and print media coverage, perhaps not unsurprisingly focuses on ‘bad news’ and patient and public dissatisfaction (see, eg, refs 23–26) but our focus groups suggested that service providers felt that callers were satisfied with the new service:

Manager: the older generation … they wouldn't phone 999 because, ‘oh well, you're very busy, and I don't think I need an ambulance’, whereas the 111 has now given them that option that actually, ‘I can ring that number, because it's there to help me, because it's not going to tie up an ambulance’ [Group nodding and sounds of agreement] So, you know, it's opened that door, so, and we are seeing that.

Site 3, focus group.

Call handler 1: People know that they can go to work and then just ring 111 when they come in on a night-time, and they know they’re going to be seen… or at least have their symptoms triaged …. So I think it’s successful in that extent,

Site 1, focus group.

Impact on other services?

One criteria of the success of NHS 111 might be whether it reduced the demand for other services. However, the impact on other health services is notoriously difficult to assess. Clearly NHS 111 has replaced NHS Direct, which was officially discontinued in March 2014. The pilot evaluation showed that NHS 111 did not have a statistically significant impact on emergency ambulance calls, but there was a statistically significant increase in emergency ambulance incidentattendances.9 Analysis of the MDS shows that, to date, an average of 11.3% of eligible calls resulted in an emergency ambulance dispatch and that 8.1% of callers were recommended to attend accident and emergency (see table 3 for quarterly data).

Table 3

Calls resulting in use of emergency services (ambulance calls or recommendation to attend A&E)

	2013–2014 Q4	2014–2015 Q1	2014–2015 Q2	2014–2015 Q3	2014–2015 Q4	2015–2016 Q1	2015–2016 Q2	2015–2016 Q3	2015–2016 Q4	2016–2017 Q1	2016–2017 Q2
Percentage of calls recommended to emergency services
Emergency ambulance dispositions	10.8	10.6	11.1	11.1	10.6	10.2	11.4	11.6	11.7	12.1	12.5
Recommended to A&E	7.4	7.6	8.2	7.2	7.7	8.3	8.9	8.0	7.9	8.8	9.0
Percentage of calls recommended to access non-emergency care
Recommended to primary care	63.3	63.2	61.0	62.7	62.5	62.4	60.7	62.0	62.3	60.9	60.0
Recommended to another service	3.3	3.7	3.7	3.5	3.7	3.6	3.7	3.7	3.8	3.6	3.7
Recommended to self care	15.1	14.9	15.9	15.5	15.6	15.5	15.4	14.7	14.3	14.5	14.7

A&E, accident and emergency.

Our case study data suggested that staff believe that the NHS 111 service may have increased the number of ‘non-urgent calls’ (compared with previous out-of-hours primary care services), including those from people seeking medical advice and reassurance:

Interviewer: In terms of daytime demand for 111, have you seen changes since it started?

111 Call handler 1: It’s doubled.

111 Clinician: we have certainly had a huge increase in demand …A lot more requests now for health information are coming through, and general advice.

Interviewer: where do those health information calls go? Who answers them?

111 Clinician: The majority we sort ourselves, within [111]. Some are referred to other areas, more expertise, some back to their own GPs… In the out of hours period [the UCC] assists us… but the majority are sourced in-house.

111 Call handler 2: Sometimes it’s just a telephone number … very simple…

111 Call handler 3: Quite often… it's a dental help line number, … my son's got chicken pox, can he go to school, … through to some really quite complex stuff that we've been getting.

Site 3, Focus group.

In addition, the referral of patients from NHS 111 to other services sometimes put pressure on working relationships between services:

Clinician 2 says that one of the big problems is ‘sending all the home visits through as a 2 hour priority’. …He says that many of these calls do not need visiting within 2 hours. Other problems include sending cases to the UCC that really require Emergency Department visit. …He says that there is some tension with [111] about triaging and what the UCC perceive as incorrect dispositions. He says that [111] will ‘never admit they are wrong’.

UCC site 1, Observation.

Is NHS 111 cost-effective?

There is a significant lack of robust data about resource use and costing of this service, both in terms of initiation and ongoing running costs. The development costs for the CDSS software are not publicly available and it has been outside the scope of our studies to obtain cost data for individual NHS 111 services which are separately commissioned by Clinical Commissioning Groups. The use of some existing infrastructure and estate (call centre accommodation, telephony, computing and training facilities and human resource management capability) from NHS Direct and or Ambulance Service sites compounds the difficulties in calculating initiation costs. The Sheffield evaluation team was able to estimate costs for the four pilot sites as £201 000 per month, with a total impact for the NHS of £307 000 per month.8 This averaged as £12.26 per call, however the report authors urged caution around these estimates due to variability in services across the four sites. A significant barrier to assessing cost-effectiveness is the lack of data on measurable health outcomes and the difficulties in capturing and disaggregating activity data (eg, walk-in centre, urgent care centre and emergency department use) and lack of data on other key costs (eg, in-hours general practice dedicated to urgent care). The increasing involvement of private sector providers and competitive contracting processes has introduced commercial sensitivities that are likely to prevent full economic costing of these services in the future.

Discussion

Summary of our findings

The UK NHS 111 service has fundamentally changed the way in which urgent care is organised and delivered. This service is now predominantly provided by non-clinical call handling staff using a specially designed decision-support software, supported by clinical advisors. The provision of this service has thus moved into the domain formerly occupied by nurses in NHS Direct, and before that by medical doctors who traditionally triaged and managed out-of-hours urgent care services. This represents a significant labour substitution and deployment of digital technology in the NHS. The service has been successfully rolled out across 44 sites. It has seen a steady increase in demand over time with significant peaks in winter months. Although the service aspires to ensure that patients receive the ‘right advice in the right place, first time’ it seems that an increasing percentage of calls require a call back suggesting that callers are not getting a one-stop service. The service appears not to meet some targets such as call answering times, but patients appear largely satisfied. This satisfaction is not consistently evident among other stakeholders, notably general practice and news media which offer more negative accounts. The impact of NHS 111 on other health services is difficult to assess and cost-effectiveness has not been established. While labour substitution is often driven by a desire to reduce costs, it should not be assumed that employing ‘cheaper’ non-clinical staff to assess calls has reduced overall costs to the NHS, if for example, considerably more of them are needed to deliver the same service as the clinical staff they replace.

Strengths and limitations

The primary study reported here was built on a previous project which examined the CDSS used by NHS 111 in different emergency and urgent healthcare service settings. It provided rich detail about the five case studies and captured the diversity of organisational models in different NHS 111 services at different points in the deployment cycle (some were early adopters/pilots). We were only able to explore the impact of NHS 111 on Urgent Care Centres and recognise that a wider network of health services are involved in the provision of urgent care, which we were not able to explore. It was beyond the scope of our primary case study work to directly explore patient experiences, so we are only able to report staff perceptions of satisfaction and service use. The analysis of the MDS is limited to the published, aggregate, data and is subject to decisions made by the NHS 111 Programme Team on reporting categories. The analysis is further limited by substantial missing data under some headings—for example, the resource use and cost (contract value) items typically have >50% observations missing. However, the MDS represents a consistent time series over the period and it is unlikely that the trends we have observed are data collection or classification artefact.

Implications

A core theme of the Hollywood movie ‘Field of Dreams’ is ‘build it and people will come’. (In the film this refers to a baseball pitch constructed in the Iowan cornfields.) NHS 111 appears to have pulled off a similar feat. It had over 15 million users in 2016 and an apparently upward trend in demand. For the most part, the patients who use the service appear satisfied. If roll out, use and patient satisfaction are markers of success, then NHS 111 can be deemed highly successful. However, our analysis of five case studies and routine activity data from 44 sites suggest that a more nuanced assessment can be made. We lack strong evidence of the impact of NHS 111 on health outcomes or on the wider network of services such as emergency care—although our research has provided some evidence about negative views expressed by some staff in Urgent Care Centres. The lack of cost-effectiveness analyses is more significant. Although it is often assumed that labour substitution saves money, this is not always the case.27

NHS 111 is currently being championed as the single point of access to urgent care, alongside the new policy vision for clinical hubs which will expand the clinical input to urgent care (the model is that NHS 111 will transfer more callers to nurse practitioners and GPs).28 This appears very similar to the operating model of NHS Direct, the service NHS 111 replaced. Moreover, approximately 50% of the current callers to NHS 111 are directed to primary care where they consult a doctor or nurse. A cynic might ask if this is success or merely the reinvention of an old wheel?