Culture and behaviour in the English National Health Service: overview of lessons from a large multimethod study.

BACKGROUND: Problems of quality and safety persist in health systems worldwide. We conducted a large research programme to examine culture and behaviour in the English National Health Service (NHS).
METHODS: Mixed-methods study involving collection and triangulation of data from multiple sources, including interviews, surveys, ethnographic case studies, board minutes and publicly available datasets. We narratively synthesised data across the studies to produce a holistic picture and in this paper present a high-level summary.
RESULTS: We found an almost universal desire to provide the best quality of care. We identified many 'bright spots' of excellent caring and practice and high-quality innovation across the NHS, but also considerable inconsistency. Consistent achievement of high-quality care was challenged by unclear goals, overlapping priorities that distracted attention, and compliance-oriented bureaucratised management. The institutional and regulatory environment was populated by multiple external bodies serving different but overlapping functions. Some organisations found it difficult to obtain valid insights into the quality of the care they provided. Poor organisational and information systems sometimes left staff struggling to deliver care effectively and disempowered them from initiating improvement. Good staff support and management were also highly variable, though they were fundamental to culture and were directly related to patient experience, safety and quality of care.
CONCLUSIONS: Our results highlight the importance of clear, challenging goals for high-quality care. Organisations need to put the patient at the centre of all they do, get smart intelligence, focus on improving organisational systems, and nurture caring cultures by ensuring that staff feel valued, respected, engaged and supported.

Introduction

A commitment to delivering high-quality, safe healthcare has been a policy goal of governments worldwide for more than a decade, but progress in delivering on these aspirations has been modest:1 patients everywhere continue to suffer avoidable harm and substandard care.2 3 England's National Health Service (NHS) has not been immune to these problems. Despite some encouraging evidence of improvement in quality and safety,4 5 large and inexplicable variations in quality of care are evident across multiple domains and sectors of healthcare, from primary through to community and secondary care.6 7 England has also seen a number of high-profile scandals involving egregious failings in the quality and safety of individual providers. These include the case of Mid Staffordshire NHS Foundation Trust,8 the subject of a recently published public inquiry by Sir Robert Francis into how catastrophic failings in the quality and safety of care went undetected and uncorrected.9

Francis identified the causes of organisational degradation at Mid Staffordshire as systemic; he saw the underlying faults as institutional and cultural in character. He found significant weaknesses in NHS systems for oversight, accountability and influence for patient safety and quality of care. Central to his analysis was evidence of a large-scale failure of control and leadership at multiple levels, from what social scientists term the ‘blunt end’ of the system where decisions, policies, rules, regulations, resources and incentives are generated,10 through to the ‘sharp end’, often known as the ‘frontline’, where care is provided to patients. The distinction between the blunt end and the sharp end is of course a heuristic one; many within healthcare organisations function in hybrid positions as managers and practitioners. Nonetheless, it is useful to recognise how the blunt end, by shaping the environment where care is delivered, may create the ‘latent conditions’11 that increase the risks of failure at the sharp end, but may equally generate organisational contexts that are conducive to providing high-quality care. Such contexts include culture: Francis blamed an ‘insidious negative culture involving a tolerance of poor standards and a disengagement from managerial and leadership responsibilities’.8 Culture is, of course, a term that is widely used but notoriously escapes consensual definition.12 Many definitions of culture (including Schein's13 influential approach) nonetheless have in common an emphasis on the shared basic assumptions, norms, and values and repeated behaviours of particular groups into which new members are socialised, to the extent that culture becomes ‘the way things are done around here’.

The findings of the Francis inquiry are depressingly familiar. England is not alone in experiencing organisational crises in healthcare; examples of failures in healthcare systems have occurred as far apart as New Zealand, the USA and the Netherlands. Several demonstrate precisely the same features as Mid Staffordshire, including long incubation periods during which warning signs were discounted, poor management systems, failure to respond to patient concerns, cultures of secrecy and protectionism, fragmentation of knowledge about problems and responsibility for addressing them, and cultures of denial of uncomfortable information.14 An important question thus concerns the extent to which the features of the Mid Staffordshire case might be symptoms of more widespread pathologies, given that other organisations in the NHS are exposed to the same institutional and regulatory environment. In this article, we offer lessons from a large multimethod research programme on culture and behaviour related to quality and safety in the NHS.

The research programme covered a critical period between 2010, following the initial inquiry into Mid Staffordshire15 and the White Paper on the NHS,16 and 2012, when the Health and Social Care Act was passed. The programme involved a large number of substudies using different methods to seek evidence from staff and patients throughout the English NHS, from large subsamples of NHS organisations, strategic level stakeholders, teams, and patient and carer organisations, and from detailed case studies. It was thus able to provide graduated levels of focus and multiple lenses. Each of the individual substudies will be reported separately, but there is considerable value in bringing the learning from them together holistically. In this article, we provide a synthesis across the studies to draw out high-level learning about culture and behaviour in NHS organisations; what influences culture and behaviour; and what needs to change to give effect to the vision of a safe, compassionate service in which patients and their families could have trust and confidence.

Methods

We conducted a large, mixed-method research programme involving seven separate substudies (table 1). The programme received ethical approval from an NHS Research Ethics Committee. In summary, primary data were drawn from:

Table 1

Summary of elements of the research programme

Study element	Participants and scheduling	Setting	Focus of research	Analytic approach
1. Stakeholder interviews	107 semi-structured telephone interviews with those closely involved in quality and safety	Acute trusts, ambulance trusts, mental health trusts, community trusts, foundation trusts, primary care trusts, strategic health authorities, general practices and healthcare commissioning organisations	Understanding of vision of high-quality and safe care; what is required to make it happen; theories of change; plans to implement quality and safety improvement, enhance leadership and promote staff engagement; views on what quality improvement means, how it could best be secured, and obstacles	Analysis based on constant comparative methodUse of QSR NVivo 8 software
2. Ethnographic case studies: observations and interviews	Comparative case studies across seven purposively chosen cases650 h of observation; 197 semi-structured interviews with executive and board-level staff and frontline staff	Four hospital trusts; a quality improvement collaborative; a large-scale quality improvement programme involving dozens of organisations; one primary care provider involving a chain of practices	Assessing culture and behaviour in relation to quality, staff engagement with quality, leadership for quality, quality improvement, practical actions for promoting cultures of high-quality care	Analysis based on constant comparative methodCoding within and across cases, systematically searching for where clusters of codes formed a patternCombining data from interviews across cases and stakeholders to form a single dataset
3a. Patient and public involvement: survey	715 survey responsesCross-sectional	Patient participation groups	The survey consisted of 14 statements about patient experience. Open text box provided for each statement	Quantitative analysis—largely descriptiveOpen-ended responses subject to content analysis to derive themes inductively
3b. Patient and public involvement: focus groups and interviews	Two focus groups and 10 interviews	Patient and carer organisations	Interpret the findings of the surveyAssessing views on obstacles to delivering improved quality and safety and greater accountability in the NHS	Qualitative analysis of key themes
4a. NHS staff and patient surveys: patient satisfaction survey data	165 acute trusts—data from 2007, 2009, 2011	Acute trusts	Patient satisfaction came from the National Acute Inpatient Survey, using the data on patients’ overall ratings of care	Descriptive statistics and paired sample t tests
4b. NHS staff and patient surveys: national staff survey data	309 NHS trusts from 2007, 2009, 2011 national staff survey	Primary care, ambulance, acute care and mental health trusts	Staff engagement, organisational climate, job satisfaction, manager support, job design, errors and reporting, work pressure, bullying, harassment and abuse, team working, training, appraisal, stress	Descriptive statistics and paired sample t tests
4c. NHS staff and patient surveys: outcome measures	2005–2009	Primary care, ambulance, acute care and mental health trusts	Patient mortality (acute sector only) (hospital standardised mortality ratio); quality of services and use of resources (Annual Health Check ratings by Healthcare Commission between 2005/2006 and 2008/2009); infection rates (MRSA) per 10000 bed days; staff absenteeism; staff turnover	Detailed correlation analysis between staff survey and inpatient survey; multiple and multilevel regression analysis, using HR practice variables to predict engagement; regression and ordinal logistic regression analysis to predict patient satisfaction, patient mortality, staff absenteeism, staff turnover, infection rates, and Annual Health Check ratings, controlling for trust type, size and location; latent growth curve modelling to predict outcomes
5. Clinical teams functioning, effectiveness and innovation	621 teams (4604 responses)Aston Team Performance InventoryCross-sectional data with data on team changes collected from 388 teams (1299 individuals) 3 months laterTeam performance data from team leaders/external raters	51 trusts (13 acute, 17 mental health, 10 ambulance and 11 primary care trusts)	Team functioning:task design, team effort and skills, organisational support, resources, objectives, participation, creativity, conflict, reflexivity, task focus, leadership, satisfaction, attachment, effectiveness, inter-team relationships, innovationLeaders’/external raters’ evaluations of effectivenessInnovations introduced by teamsSources of frustration and resilience	Descriptive analysis, ANOVA, regression and relative importance analysisAnalysis and ratings from domain relevant expertsOpen-ended responses subject to content analysis to derive the themes
6a. Objectives and team working of trust boards	34 boards (306 individuals) Administered processes section of Aston Team Performance InventoryDetails of board objectives	Primary care, ambulance, secondary care and mental health trusts	Team processes and content: objectives, participation, reflexivity, task focus (lack of team) conflict, creativity and innovationClarity and challenge of board objectives	Descriptive analysis, regression and relative importance analysisAnalysis and ratings from domain relevant experts
6b. Trust board innovation	71 NHS trust boards793 sets of minutesMinutes from 18 months of board meetings	Primary care, ambulance, secondary care and mental health trusts	Innovations introduced by boards and domain of focus (e.g. productivity, targets, organisational effectiveness, quality, safety, patient complaints, clinical effectiveness)	Analysis and ratings from domain relevant experts
6c Quality and safety in trust boards	Detailed analysis of minutes for eight boards	Primary care, ambulance, secondary care and mental health trusts	Board discussions of quality and safety	Ethnographic content analysis and summative analysis

ANOVA, analysis of variance; HR, human resources; MRSA, methicillin-resistant Staphylococcus aureus; NHS, National Health Service.

107 interviews with key, senior level stakeholders from across the NHS and beyond;

197 interviews from the ‘blunt end’ (executive and board level) of NHS primary care and acute organisations through to the ‘sharp end’ (frontline clinicians) where staff care for patients;

over 650 h of ethnographic observation in hospital wards, primary care practices, and accident and emergency units;

715 survey responses from patient and carer organisations;

two focus groups and 10 interviews with patient and carer organisations;

team process and performance data from 621 clinical teams, drawn from the acute, ambulance, mental health, primary care and community trust sectors;

793 sets of minutes from the meetings of 71 NHS trust boards from multiple sectors over an 18-month period, including detailed analysis of eight boards’ minutes.

We did not use a formal protocol for integrating the findings across these studies,17 instead deploying a more interpretive, narrative approach.18 We engaged in extensive discussions as a team, and identified points of convergence and updated our analytic categories as we came closer to agreement. Given the size of our datasets, we are able to provide only very limited primary data in support of our analysis in this article; our focus is on high-level messages. Further details of the methods and the data are available in a longer report.19

Results

Our synthesis of the findings across the substudies allowed many insights into the challenges of realising a vision of reliably safe, high-quality care across the NHS, and important learning about how improvement can best be secured.

Goal setting

Virtually all those we interviewed (over 300 in total) were firmly committed to the ideal of a safe, high-quality health service for patients and to good patient experience. Many identified the values of compassion and care as at the heart of the mission for their organisations, and as their most deeply felt personal professional commitment. Our interviews, observations, surveys and documentary analysis were united in suggesting that, for organisations to succeed in delivering high-quality, safe care, they needed to have a clearly articulated vision, including explicit goals for quality and safety and a strategy for achieving them. Interviews and observations repeatedly emphasised the importance of clear goals in establishing and signposting priorities for improvement, motivating staff and ensuring resources were appropriately directed. Our survey evidence from the national patient satisfaction and national staff surveys (NSS) showed that patient satisfaction was highest in trusts that had clear goals at every level. Consistent with the findings of the Francis inquiry, boards of organisations were identified in interviews as particularly influential in setting the overall direction and demonstrating the commitment and organisational priority given to quality and safety.

But converting laudable aspirations for high-quality, safe and compassionate care into clear goals appeared challenging. Clarity about goals, how they could be achieved, and leadership for delivery were highly variable. Our questionnaire surveys of board members showed that they rarely stated clear board objectives that were challenging and measurable. The 621 frontline clinical teams we studied were generally even less clear about their objectives.

A major challenge in creating unifying visions for patient safety and quality and setting clear objectives was the range, diversity and complexity of external expectations and requirements that NHS organisations faced. Board and executive teams described an institutional and regulatory environment that was populated by external agencies and actors who served different but overlapping functions. They reported that targets, standards, incentives and measures seemed to crowd in from multiple external sources; that the same information was required many times in different formats; and that answering to so many masters and producing data for so many external audiences was costly and distracting. The proliferation of externally set priorities and the number of different agencies and actors created what we termed ‘priority thickets’—dense patches of overlapping or disjointed goals that commanded very substantial attention and resources, but did not necessarily provide clear direction or facilitate the development of clear goals, internally coherent visions or strategies linked to local priorities.

Faced with so many competing demands, some organisations tended to revert to a highly bureaucratised form of management, characterised by proliferation of rules, procedures and forms corresponding to externally imposed demands. Many of these seemed to be motivated mostly by a need to make displays of compliance,20 rather than by genuine efforts to make systems safer or of better quality. Much of this activity could be characterised as defensive and reactive. It was a source of frustration throughout organisations; frontline teams complained of ‘blanket’ policies which were seen as ‘very prescriptive and not concentrated on clinical work’.

We also found considerable variability in how far organisations succeed in making their aspirations for high-quality care real: what we termed ‘bright spots’ and ‘dark spots’ were both evident, even within the same organisations. Bright spots included teams and individuals who demonstrated caring, compassion, cooperation and civility, and a commitment to learning and innovation. Direct observations found that in many settings patients were often treated with kindness and respect, systems functioned well, and staff were busy but knew what they were doing and why. Compliance with many standards of good practice, such as hygiene and equipment counting, was observed to be very good in many cases.

Though much care was of such high quality as to be inspiring, substandard care or ‘dark spots’ were also evident. Dark spots were found where staff were challenged to provide quality care, were harried or distracted, or were preoccupied with bureaucracy. Across our interviews, surveys and observations we found evidence of staff and patient concern about variability in quality of care, and a lack of confidence that care would be reliably good. Interviews and surveys with patient and carer groups suggested that patients and their carers were often concerned about quality and safety. Vulnerable patients, including older patients, young patients or those who lacked the ability to ‘speak up’, were reported to be at risk of being left to ‘fend for themselves’ or ‘being forgotten’. Our observations in clinical areas and our interviews confirmed that inconsistency was a feature of many settings. For example, many staff spoke to patients politely and with kindness, but some others were brusque, impatient or discourteous. Some senior clinical nursing staff highlighted their concern at what they saw as a tendency towards task-focused rather than person-centred care.

Further evidence of the challenges of realising a vision of consistently high-quality safe care came from our analysis of the NHS Staff Survey and inpatient survey data over the period 2007–2011 (tables 2 and 3). This suggested that there had been improvements in scores relating to quality and safety reported by patients and staff nationally between 2007 and 2009, but subsequently these improvements stalled or went into reverse. Some of the plateauing may reflect a natural maximum level being reached. For example, the percentage of staff receiving health and safety training increased from 2007 to 2009, and appears to remain relatively constant in 2011. Likewise, levels of job satisfaction increased from 2007 to a moderately high level in 2009, and then stayed approximately the same in 2011. However, measures on the staff survey relating to error and incident reporting, blame cultures and improvements following incidents, where there was headroom for improvement, appeared to have shown only very modest gains. The number of staff working paid extra hours has decreased consistently, but since 2009 the number working unpaid extra hours has increased sharply. The percentage of staff receiving training in infection control related issues increased from 2007 to 2009, but fell in 2011.

Table 2

Changes in the National Staff Survey 2007–2011 (NHS trusts in England)

	2007	2009	2011	Change 2007–2009	Change 2009–2011	Standard deviation (2007)
‘I have adequate materials, supplies and equipment to do my work’	3.22	3.36	3.38	+0.14**	+0.02*	0.18
'There are enough staff at this Trust for me to do my job properly’	2.61	2.78	2.72	+0.17**	−0.06**	0.17
‘I do not have time to carry out all my work’	3.30	3.26	3.25	−0.04**	−0.01	0.21

Results are based on the 309 NHS trusts in England with data from all 3 years shown. p values are based on paired sample t tests. Responses were on a 1–5 scale with 5 indicating greater agreement with statements; an increase of 0.10 is equivalent to 10% of respondents moving up one category of response, for example, from ‘neither agree nor disagree’ to ‘agree’.

*p<0.05; **p<0.01.

NHS, National Health Service.

Table 3

Changes in the National Staff Survey and Acute Inpatient Survey 2007–2011 (NHS trusts in England)

	2007	2009	2011	Change 2007–2009	Change 2009–2011	Standard deviation (2007)
Staff survey
‘My trust encourages us to report errors, near misses or incidents’	3.84	3.93	3.94	+0.09**	+0.01*	0.11
‘My trust treats reports of errors, near misses or incidents confidentially’	3.55	3.63	3.66	+0.08**	+0.03**	0.10
‘My trust blames or punishes people who are involved in errors, near misses or incidents’	2.75	2.67	2.68	−0.08**	+0.01	0.12
‘When errors, near misses or incidents are reported, my trust takes action to ensure that they do not happen again’	3.47	3.54	3.57	+0.07**	+0.03**	0.13
Acute inpatient survey
In your opinion, how clean was the hospital room or ward that you were in?	3.45	3.60	3.63	+0.15**	+0.03**	0.13
As far as you know, did doctors wash or clean their hands between touching patients? (% of positive responses)	53	59	57	+6**	−2*	6

Staff survey results are based on the 309 NHS trusts in England with data from all 3 years shown. p values are based on paired sample t tests. Staff survey responses were on a 1–5 scale with 5 indicating greater agreement with statements; an increase of 0.10 is equivalent to 10% of respondents moving up one category of response, for example, from ‘neither agree nor disagree’ to ‘agree’. Inpatient survey results are based on the 157 NHS acute trusts in England with data from all 3 years shown. Inpatient survey responses were on a 1–4 scale, with 4 indicating greater agreement with statements.

*p<0.05; **p<0.01.

NHS, National Health Service.

Variability in intelligence

A major challenge to achieving goals relating to quality and safety was that of ensuring that high-quality intelligence was available to organisations, teams, and individuals about how well they were doing and where the deficits and risks in organisational systems lay. NHS organisations that we studied were putting considerable time, effort and resources into data collection and monitoring systems. They typically used a combination of routinely collected data, specific data collection initiatives, and sporadic sources such as spot checks and audits. To a varying extent they also drew on feedback provided by clinical staff and patients as a means of assessing trends. However, the degree to which data collection efforts translated into actionable knowledge, and then into effective organisational responses, differed markedly between organisations.

Some behaviours in relation to data gathering might be described as ‘problem-sensing’; other, less positive behaviours were ‘comfort-seeking’. Problem-sensing involved actively seeking out weaknesses in organisational systems, and it made use of multiple sources of data—not just mandated measures, but also softer intelligence. Soft intelligence could be gathered in many ways, including active listening to patients and staff; informal, unannounced visits to clinical areas; and techniques such as ‘mystery shoppers’, shadowing of staff, and swapping roles for a short period. While sometimes discomfiting, this less routinely gathered knowledge enabled fresh, more penetrating insights to complement quantitative data. Senior teams displaying problem-sensing behaviours tended to be cautious about being self-congratulatory; perhaps more importantly, when they did uncover problems, they often used strategies that went beyond merely sanctioning staff at the sharp end, making more holistic efforts to strengthen their organisations and teams.

Comfort-seeking behaviours are defined here as being focused on external impression management and seeking reassurance that all was well; consequently, what was available to organisations was data, but not intelligence. Serious blind spots could arise when organisations used a very limited range of methods for gathering data, were preoccupied with demonstrating compliance with external expectations, failed to listen to negative signals from staff or lacked knowledge of the real issues at the frontline. Comfort-seeking tended to demonstrate preoccupation with positive news and results from staff, and could lead to concerns and critical comments being dismissed as ‘whining’ or disruptive behaviour. When comfort-seeking was the predominant behaviour, data collection activities were prone to being treated by sharp-end staff as wearisome and fruitless accountability exercises. Some staff reported that they felt the main purpose of much data collection was to allow individuals to be blamed if something did go wrong, not to make the system safer.

Variability in systems

Interviews, observations and surveys showed that when staff had access to appropriate resources, perceived that staffing levels were adequate with the right skill mix, and had systems that functioned effectively, they felt that they could complete their work successfully, could explore new ways of improving quality and could develop reflective practices. This reinforced their levels of motivation and morale in a virtuous circle. But deficits in systems often obstructed and frustrated well-motivated staff in their mission to provide good care for patients. Our analysis of questionnaire reports from 621 clinical teams showed that many staff felt unable to achieve their goals for patients because of organisational factors outside their control. Observations showed that staff wasted time working with poorly designed IT systems, negotiating clinical pathways with obstructions and gaps, and battling with multiple professional groups and subsystems (e.g. pharmacy, microbiology and imaging, and many others) that did not operate in integrated ways. We also found evidence of problematic handovers between shifts, departments and teams, team conflict and a diffusion of responsibility relating to particular patients. Patient and carer groups reported discontinuities in care between institutional boundaries and even within single organisations. These ‘responsibility cordons’ left patients variously ill informed, distressed and disappointed, and sometimes in danger.

Staff at the sharp end were very often aware of systems problems but felt powerless to bring about change. Changes within organisations, uncertainty about priorities, poor systems, heavy workloads and staff shortages were all blamed for staff feeling they lacked support, further reducing their motivation and morale. Given that many systems required significant improvement, it was disappointing that we found a clear trend of decreasing levels of board innovation, especially in relation to quality and safety.

We defined innovation as the intentional introduction of processes and procedures, new to the unit of adoption (team or organisation) and designed to significantly benefit the unit of adoption, staff, patients or the wider public. An analysis of board minutes from 71 NHS Trusts covering an 18-month period between January 2010 and June 2011 identified a total of 144 innovations that were implemented in organisations, representing an average of only 1–3 per organisation. More than half were focused on increasing productivity (73), with very few related to safety (14). The largest number of innovations (62) was identified between January and June 2010, followed by 56 innovations between July and December 2010. Only 26 innovations were identified in data covering the time between January and June 2011. Separately, analysis of 4976 responses to open-ended questions in our survey of 486 clinical teams identified 183 innovations over a 6-month period. This also suggested relatively low rates of innovation among frontline teams, though many of the solutions they did devise were ingenious and resourceful. The largest number of frontline staff innovations was focused on enhancing quality of patient care; fewer aimed at improving administrative effectiveness, and the smallest number concerned staff wellbeing.

Many organisations were using specific quality improvement methods to achieve change, including Plan–Do–Study–Act cycles, Collaboratives, Lean, Six Sigma, and Productive Ward (an NHS programme to support ward teams in reviewing the processes and environment used to provide patient care21). Some organisations also used wider techniques to improve quality, including organisation-level campaigns. Great enthusiasm for these approaches was often reported by those leading improvement efforts, but we also sometimes observed a tendency towards uncritical or indiscriminate use, and some evidence of ‘magical thinking’ (‘this initiative will solve many problems easily and quickly’). Frontline staff who had to implement these initiatives were often not consulted or adequately informed about their purpose and implementation, and sometimes initiatives were abandoned or forgotten after a short period of intense activity. In some cases, there was insufficient acknowledgement of the effort, expertise and investment required to make such approaches work, and substantial problems with quality of data collection and interpretation.

Culture and behaviour

Leadership was important for setting mission, direction and tone. Our observations, interviews and surveys all emphasised the importance of high-quality management in ensuring positive, innovative and caring cultures at the sharp end of care. Some senior teams encouraged and enabled frontline teams to address challenges and to innovate, but recognised that, along with demanding personal accountability from staff, they also needed to fix systems problems that prevented staff from functioning well. A strong focus by executive and board teams on their own role in identifying and addressing systems problems was powerful in supporting cultural change that delivered benefits for patients, and our observations and interviews identified many examples of impressive gains being made by the sharp and blunt ends working together around unifying goals.

Nevertheless, an important consequence of the failure to clarify goals, to gather appropriate intelligence or to address systems deficits was the existence of frequent misalignments between the ways the blunt end and the sharp end of organisations conceived of quality and safety problems and their solutions. For sharp-end staff, threats to safety and quality were identified as weaknesses in systems, failures of reliability, suboptimal staffing, inadequate resources and poor leadership. Lack of support, appreciation and respect, and not being consulted and listened to were seen as endemic problems by staff in some organisations. In contrast, some senior managers—particularly those engaged in comfort seeking—tended to see frontline staff behaviour and culture as the cause of quality problems. In consequence, those at the blunt and those at the sharp end often did not agree on the causes of variation in quality and safety and, therefore, on how they should be addressed.

We also found substantial variation in the quality of management. Our analyses of NSS data showed that hospital standardised mortality ratios were inversely associated with positive and supportive organisational climates. Higher levels of staff engagement and health and wellbeing were associated with lower levels of mortality, as were staff reporting support from line managers, well structured appraisals (e.g. agreeing objectives, ensuring the individual feels valued, respected and supported), and opportunities to influence and contribute to improvements at work. NSS data also showed that staff perceptions of the supportiveness of their immediate managers, the extent of staff positive feeling, staff satisfaction and staff commitment were associated with other important outcomes, including patient satisfaction. In places where staff reported high work pressure, patients on the national surveys also reported too few nurses, insufficient support, and problems with information, privacy and respect. In trusts with poor staff health and wellbeing, high injury rates, and a high level of staff intention to quit their jobs, patients reported that they were generally less satisfied, and Care Quality Commission ratings described poorer care and poorer use of resources. These findings were consistent across trust types (primary care, ambulance, mental health, community and acute).

Also concerning was evidence that though team-working seemed well established and widespread on the surface, there was a surprising lack of clarity about team purpose, objectives, membership, leadership and performance among many teams. Our survey of 621 clinical teams demonstrated that team inputs and team processes were significantly associated with the effective provision of good-quality care, but senior managers were sometimes unable to identify teams and team leaders. When team leaders were identified, they were often confused about who their team members were. Team members themselves had low agreement about who was part of their team. Factors associated with successful teams were the effort and skills of team members, resources made available and good processes. Clarity and agreement about team objectives were key to clinical team effectiveness, along with a participative approach to decision-making that engaged all team members. Teams who regularly took time out to reflect on their objectives, how they were going about achieving these and how their performance needed to change were particularly likely to be more effective and innovative.

Discussion

This large mixed-method study identified many ‘bright spots’ of excellent caring and practice and high-quality innovation across the NHS, but also considerable inconsistency. Though Mid Staffordshire may have been one particularly ‘dark spot’ in the NHS, organisations throughout the NHS are likely to have at least some shadows: there was little confidence that care could be relied upon to be good at all times in all parts of organisations, and we found evidence of structural and cultural threats to quality and safety. Our analysis points to how things may improve. First, clear and explicit goals that are coherent from ward to Whitehall are essential. Second, organisations need intelligence: they need to know how well they are really doing as an organisation, and where they need to improve. This means actively seeking uncomfortable and challenging information from patients and staff, rather than relying solely on formal data collection against narrow performance indicators that may not give a fully rounded picture of quality of care. Third, organisations must constantly review, strengthen and improve their systems. System improvement and strengthening may be needed at many different levels, from smoothing clinical pathways to improving communication, teamwork, assuring that clinical areas are adequately staffed with the right kinds of skills, and ensuring personal development, equipment standardisation, and training. Organisations also need to focus on developing cultures that are person centred—not just task focused—by valuing and building on the excellent care and commitment delivered by many staff throughout the NHS. This involves modelling and reinforcing values and behaviours that underpin high-quality care, patient safety and positive patient experience from the blunt end to the sharp end of the whole system.

Our work involving a very large number of organisations confirms that achieving quality and safety in NHS organisations requires a robust strategy and unifying vision. National leadership sets the tone, signals importance, legitimises, and creates accountability mechanisms. Yet the Francis public inquiry showed that a major problem for Mid Staffordshire was the large number of different agencies and bodies with a say in the NHS. This contributed to fragmentation, multiple competing pressures, ambiguity and diffusion of responsibility. Our work similarly demonstrates that proliferation of external agencies and expectations creates conflicts, distraction and confusion from the blunt to the sharp end of organisations about where resources and attention should be directed. Where incentives and external expectations conflict, compete or fail to cohere, the ability of organisations to set themselves clear, internally valued goals for achieving their aspirations is weakened.22

In a distributed and complex system such as the NHS, failures are least likely when the goals are clear and uniting, and when appropriate, sensitively designed regimes of control and support are found at every level: from policymaking, through the layers of formal regulatory systems, the institutional environment, individual organisations, teams and practitioners, through to patients’ experiences.23 Coherence of national direction is therefore essential to avoid dispersing responsibility and accountability, and creating confusing messages and signals.24 As new bodies move forward, including NHS England and the Clinical Commissioning Groups, it is important that they avoid creating further competing priorities, and instead ensure focus and coherence.

National leadership needs to be matched by high-quality leadership across multiple organisational levels underpinned by clear, patient-focused goals and objectives. The role of organisational boards in securing the quality and safety of health services has become an increasing focus of academic and policy interest,25 26 not least because of evidence of the link between leadership from the top and the priority and resources given to quality25 and clinician engagement.27 But we found worrying evidence of NHS trust boards failing to set clear goals for themselves as boards and for their organisations. Goals do need to be set, and they should be limited in number (to identify priorities while avoiding the creation of priority thickets) and known not only by all board members but (if appropriate) more widely within their trusts. Goals should be shaped by the need to promote quality and safety, to ensure sound financial performance, and to value dignity and respect for patients. They should provide a framework for objectives at all levels of trusts, from senior management to clinical teams at the sharp end. They should be framed to encourage innovation at all levels, and quality and safety of patient care must be overriding. Not all innovations need to be grand and over-arching: fixing (apparently) small problems may result in major gains.28

The Francis public inquiry showed that discounting of warning signs of deterioration was a key feature of board and executive behaviour at Mid Staffordshire. Those in senior positions appear to have developed ‘blindsight’—a way of not seeing what was going wrong. Our work confirms the importance of high-quality intelligence (not just data) and making that intelligence actionable. But we found sobering evidence that NHS organisations are not always smart with intelligence, and need to gear more towards problem-sensing rather than comfort-seeking. At the national level, care needs to be taken to ensure that the number of measures organisations are expected to report externally is well managed,29 and that measures are aligned with local priorities, avoid imposing excessive burdens, generate accurate intelligence,30 and most of all, are useful for informing improvement locally. Thus the right intelligence needs to be gathered, interpreted correctly and fed back clearly to staff at the sharp end of care, so that they consolidate and improve their performance.31

Organisations need to be especially alert to the possibility of blind spots where they are unaware of problems. They should use multiple strategies to generate intelligence and undertake self-assessment32 of local culture and behaviours—not just rely on mandated measures—and use a range of techniques for hearing the patient's voice and the voice and insights of those at the sharp end of care. Consistent with Francis’ findings, good management is as important as good leadership in our analysis: the wellbeing of staff is closely linked to the wellbeing of patients, and staff engagement is a key predictor of a wide range of outcomes in NHS trusts. Achieving high levels of engagement is only possible in cultures that are generally positive, when staff feel valued, respected and supported, and when relationships are good between managers, staff, teams and departments and across institutional boundaries. Staff experience frustration and conflict when asked to work in systems that do not effectively serve them, or the patients they care for; these system defects include staff shortages or inappropriate skill mix to address the needs of specific clinical areas. Our analysis suggests that improving culture, behaviour and systems requires system improvement and better communication between the blunt end and sharp end. This needs to be sustained, intense, mutually respectful and focused on achieving a shared understanding of quality problems and joint working to put them right. Trusts can develop these cultures by using specific strategies (box 1), while recognising the complexities of trying purposefully to engineer culture.12

Senior leaders should:

Continually reinforce an inspiring vision of the work of their organisations

Promote staff health and wellbeing

Listen to staff and encourage them to be involved in decision making, problem solving and innovation at all levels

Provide staff with helpful feedback on how they are doing and celebrate good performance

Take effective, supportive action to address system problems and other challenges when improvement is needed

Develop and model excellent teamwork

Make sure that staff feel safe, supported, respected and valued at work.35

This article has some limitations. In the available space, we have not been able to provide full details of methods or data from this unusually large research programme. Instead we have sought to provide an overview of the key findings of the component studies. Our synthesis of findings was interpretive and narrative, and did not use a formal protocol. Others might reach somewhat dissimilar conclusions or interpretations of our data.33 However, we believe that our careful scrutiny of the data, extensive discussions, and detailed analysis of themes have enabled us to produce a powerful, robust and rich picture. Future work should assess the generalisability of these findings in other contexts, including the other countries of the UK.

Conclusions

This very large-scale research programme suggests that there is room for improvement in the quality and safety of care offered by the NHS, and that this improvement can build on the progress already made. Trusts need continually to refresh, reinforce and model an inspiring vision that keeps the patient at the centre. It is essential to commit to an ethic of learning and honesty,34 to work continually to improve organisational systems, and to nurture the core values of compassion, patient dignity and patient safety through high-quality leadership. This implies equal attention to systems, cultures and behaviours: setting coherent and challenging goals and monitoring progress towards them; empowering staff to provide high-quality care and providing them with the means to achieve this through routine practice and innovation; and exemplifying and encouraging sound behaviours.