Integrated inspection of services for people with learning disabilities in Scotland: the way forward?

PURPOSE: The article summarises the process and the results of the first, integrated inspection of managed care services for people with learning disabilities in Scotland. The multi-agency model used was developed to be congruent with the existing performance inspection models, used by single agency inspection. The inspection activities and main outcomes are described, and suggestions are made for improvements. CONTEXT OF CASE: In 2006 an inspection model was devised to assess the quality of health, social services and education services for people with learning disabilities in one geographical area of Scotland, as a precursor to a programme of inspections nationally. The first joint, integrated inspection of all services for people with learning disabilities in Scotland took place in June 2006, and the report was published in March 2007. This was the first multi-agency inspection of its kind in the UK, and the first to involve carers and people with learning disabilities on the inspection team. DATA SOURCES: A number of data sources were used to check existing practice against agreed Quality Outcome indicators. Primary sources of data were social work records, health records, education records, staff surveys, carer surveys, interviews with staff, family carers and people with learning disabilities, and self evaluations completed by the services being inspected. Eleven different domains, each with sub-indicators were investigated. CASE DESCRIPTION: This paper summarises the process of an integrated, multi-agency inspection, how the inspection activities were conducted and the main findings of this inspection. Practical improvements to the process are suggested, and these may be of use to other services and inspectorates. CONCLUSIONS AND DISCUSSION: The integrated inspection was a qualified success. Most major objectives were achieved. The sharing of data amongst inspection agencies, establishing the level of commitment to integrated inspection and conducting multiple, integrated inspections nationally in a reasonable timescale are the main barriers remaining. The data were collected in an innovative way during this inspection, to make the analysis directly relevant to services, by providing domain specific and area specific details about how well local needs are being met. The lessons from this integrated inspection may be of interest to other practitioners in the UK and beyond, both in terms of process and outcomes.

Introduction

Background

This is a report on the results of the first ever integrated inspection of services for people with learning disabilities in Scotland [1]. It was carried out in June 2006 by a multi-agency, integrated team, including family carers and people with learning disabilities, and will be a pilot for subsequent inspections to be done for the 32 Local Authority areas (education and social work) and 15 Health Board areas (primary and community health services) across Scotland. Scotland has health, social work and education services that are different from, and independent of those in the rest of the UK. In England health services are inspected by the Healthcare Commission [2]; the social care services by the Commission for Social Care Inspection [3].

Prior to June 2006, managed care services for people with learning disabilities in Scotland were inspected by up to nine different regulatory organisations in Scotland: Social Work Inspection Agency (SWIA), The Commission for the Regulation of Care (CC), National Health Service Quality Improvement Scotland (NHS QIS), Mental Welfare Commission (MWC), Audit Scotland, Communities Scotland, Her Majesty's Inspectorate for Education (HMIE), Her Majesty's Inspectorate of Prisons (HMIP) and Her Majesty's Inspectorate of Constabulary (HMIC). All of these organisations were consulted in planning and carrying out the first integrated inspection, together with representatives from family carers and people with learning disabilities, from Carers Scotland and People First Scotland.

The model used in the inspection had been developed between February–April 2006 [4], based on the potential advantages and disadvantages of adopting such a comprehensive approach to inspection; both in terms of the range of services being inspected and the make up of the inspection team.

The methods of systematic data collection and analysis used in this inspection may be of interest to other practitioners, looking for ways of gaining a comprehensive assessment of the quality of services being delivered in their area.

Rationale

The purpose of inspecting services for people with learning disabilities in Scotland, and in the UK generally1 is threefold: to try to protect against neglect and abuse, to ensure effective performance management, and to improve services [5-9].

The quality of services was assessed using a new model of joint inspection, including a set of outcome indicators, applicable to multiple service settings, e.g. hospitals, community services, day services or family homes.

A major challenge for the process of integrated inspection is reviewing services which are not co-terminus, i.e. health and local authority boundaries may differ, and an integrated inspection will (typically) look at a single health authority area and all of the local authorities which are either within that geographical area, or have part of their services within that defined area.

The barriers to a truly integrated, multi-agency inspection are discussed in this article. It is argued that the time and resources needed to overcome these barriers should to be factored into the planning process, and should not be underestimated.

Method

In June 2006, a model of joint inspection was used to inspect services for people with learning disabilities in Ayrshire, Scotland (Figure 1). This model combined three existing approaches [4]:

Figure 1

Area inspected—Ayrshire, Scotland.

a Performance Inspection model used by Social Work Inspection Agency

a National Care Standards model used by the Care Commission

a Quality Indicators model used by NHS Quality Improvement Scotland

A multi-agency team carried out the inspection, using a set of agreed objectives (see Table 5). The inspection team had representatives from the following organisations:

Table 5

Achievement of general objectives for integrated inspection

Pre-inspection general objectives	Achieved/partly achieved/not achieved/undetermined
All layers of services are inspected	Achieved. The methodology allowed contact with all layers of services, through interviews, meetings and service self-evaluation
People with learning disabilities and carers are part of the inspection team	Achieved. There were two people with learning disabilities on the team
Information about the inspection and the reports is in in easy read language and other formats	Achieved. Information about the inspection was available in different formats and the inspection report has an accessible summary and alternative formats available on request
What people say in the inspection is private and confidential	Achieved. All interviews were confidential and the report does not identify individuals
Inspections look at how people are protected and empowered	Achieved. There was a specific adult protection focus in the reading of case files and interviewing of individuals
Services use a self assessment tool to evaluate the quality of their service	Achieved. A self-assessment tool was developed from the original inspection model and this was used in preparation for the fieldwork phase of inspection
People are involved in developing the outcomes that inspections look at	Achieved. There was a pre-inspection conference to look at outcomes, as well as a comprehensive consultation phase, with all stakeholders, pre-inspection
There is meaningful user and carer involvement	Partly achieved. There was a systematic approach to involving a range of users and carers before and during inspection. Whether this was meaningful will be determined by post inspection evaluation
Inspections are outcome based and lead to action	Partly achieved. The inspection was outcome based. It is too early to say whether it has led to action
There is support available to services to help prepare for inspection	Partly achieved. Support was available. Whether there was enough support will be determined by post inspection evaluation
There is no duplication and no over inspecting	Partly achieved. It is too early to say whether services already inspected will be inspected again in the near future. Post inspection evaluation will give some feedback on whether services felt over inspected
All inspectors are fully trained and inspections follow a consistent methodology	Partly achieved. There was a training event for inspectors. Not all inspectors attended this, and the different inspection agencies involved used slightly different methodology during inspection
The NHS and local authorities work closely together	Partly achieved. Developing partnership working was reported as ‘good’ for all three local authority areas in the inspection report. There was some variation in the level on multi-agency working in the final report
Service users know who to contact about the inspection and this is simple—e.g. a phone number	Partly achieved. Contact numbers were circulated before and during inspection. Post inspection evaluation will give some feedback on whether this was successful in informing people
Legal issues about sharing information are sorted out before inspections	Not achieved. Inspectors could not scrutinise health records. A self-audit system by the NHS being inspected was agreed as an alternative. Although this is the first time any inspection has obtained aggregate data from the scrutiny of individual adults' health records it is not direct, independent inspection of the records. Also, the number of education records accessed was not statistically significant
Services are inspected on a regular basis	Not achieved. No timetable has yet been set for inspection of other services for people with learning disabilities in Scotland
Inspectors respond quickly and give feedback within a reasonable timescale	Not achieved. The inspection took place in June 2006. The inspection report was published in March 2007. While inspectors did respond quickly during the inspection, there was a very long gap between inspection and publication of the report
Inspection agencies work better together	Undetermined. To be determined by comparison between this inspection and the next one
There is commitment to Joint Inspection	Undetermined. To be determined by post inspection evaluation
People have independent advocacy support, and time to prepare for meetings	Undetermined. To be determined by post inspection evaluation
If people raise concerns something will happen to change or make things better	Undetermined. To be determined by post inspection evaluation

People First Scotland (Independent self-advocacy organisation of people with learning disabilities in Scotland)

Quality Action Group (Information, advice and training organisation, run by people with learning disabilities)

Carers Scotland (Campaigning, policy and information organisation for carers in Scotland)

PAMIS (Profound and Multiple Impairment Service-voluntary organisation working with people with profound and multiple learning disabilities, their family carers and professionals who support them)

Social Work Inspection Agency (lead agency for inspections of social work services)

NHS Quality Improvement Scotland (lead agency on reviewing and improving quality of care and treatment delivered by the health service)

HM Inspectorate of Education (lead agency for inspection of all education services in Scotland)

HM Inspectorate of Constabulary for Scotland (lead agency for examining and improving the efficiency of the Police Service in Scotland)

Care Commission (lead agency for regulation of care services in Scotland, through inspection)

There were a total of four people with learning disabilities and two family carers on the inspection team.

The area being inspected was Ayrshire, Scotland (Figure 1). Ayrshire has an estimated population of 367,010 [10], spread over 3338 square kilometres of mixed countryside and urban areas, and including two islands. There are 8219 adults with learning disabilities and an estimated 431 children with learning disabilities in Ayrshire [7].

Administratively, public services in Ayrshire are delivered by three local authorities; North, South and East Ayrshire Council, and one NHS Board, Ayrshire and Arran Health Board.

North Ayrshire has a population of 135,830, East Ayrshire 119, 400 and South Ayrshire 111,780.

The fieldwork for the inspection was done over a period of two weeks and prior to this a number of meetings were held with senior management and all other stakeholders from the health authority and three local authorities to agree criteria for the inspection and the inspection activities to collect data.

The inspection team looked at 11 key outcome indicators, each with a number of Quality Outcome statements [1, 4] (Table 1).

Table 1

Quality outcome indicators (number of Quality outcome statements)

1. Enabling and sustaining independence (1)

2. Promoting inclusion (6)

3. Meeting healthcare needs (13)

4. Safety and protection (2)

5. Record keeping and communication (5)

6. Meeting staff needs (1)

7. Developing partnership working (2)

8. Leadership and direction (2)

9. Financial resource and information management (1)

10. Meeting lifelong learning needs (2)

11. Capacity for improvement (3)

Inspection activities

To collect data to check against each of the Quality Outcome statements, a variety of pre-inspection and fieldwork inspection activities were undertaken, including staff, carer and stakeholder surveys, inspection of social work, health and education records, and interviews with people with learning disabilities. These are summarised in Tables 2a and 2b in the Results section.

Table 2a

Inspection activities (Pre-fieldwork)

Inspection activity	Number of responses/sample size	Response rate (%)	Confidence intervala
Staff survey	329/607	49	3.7
Carers survey	240/1036	23	5.6
Stakeholders and partners survey	31b
Interviews with people with learning disabilities	92/100c	92
Interviews and meetings with staff, carers	149d

This was an overall rating, encompassing service users of education, social work and health services and combinations of all three. The confidence interval is based on a 95% confidence level for this population. For example, 53% of all carers who responded agreed they were satisfied with the services they receive. This means that for the population of carers of people with learning disabilities, between 47.4 and 58.6 are satisfied with the services they receive, i.e. plus or minus 5.6.

The small number of responses here are not statistically significant, and the information was used only indicatively.

Interviewing was done prior to the fieldwork inspection by staff from the Scottish Consortium for Learning Disabilities (National Organisation set up to implement the recommendations of major national review of services in Scotland) [11]. The people interviewed were a sample randomly chosen from case files categorized as adult protection, ‘at risk’, age and services transitions, complex disabilities, and ‘other’. The interviews were structured on a specific set of questions [4] representing quality indicators about: You, Your Home, Choices and Being In Control, Feeling Included, Work, Your Health, Money, Services, Changes in Your Life, Your Job. For some people with complex needs these were ‘proxy’ interviews with appropriate carers.

One hundred and forty-nine individual and group interviews took place during the inspection. A full list of these is given in Appendix 4 of the main inspection report.

Table 2b

Inspection activities (Fieldwork)

Inspection activity
Social work files read	246 (82 from each council in North, South and East Ayrshire)e
Health files read	44f
Education records read	g

The social work files were a stratified random sample, chosen to include children with learning disabilities, young people in transition, people with complex disabilities and high support needs, people with autistic spectrum disorder, adults with learning disabilities subject to the adult protection procedure, adults with learning disabilities who have had concerns expressed about them being abused, neglected or exploited. The sample size was statistically significant at local authority level for population.

NHS Ayrshire & Arran's clinical effectiveness unit did the file scrutiny and the inspection team analysed the results. For legal reasons the inspection team could not scrutinise health records directly. A self-audit system was agreed. This is the first time any inspection has obtained aggregate data from the scrutiny of individual adults' health records. The file types were categorised as Nursing (13), Psychiatry (7), Psychology (7), Occupational Therapy (7), Speech and language (4), Physiotherapy (5), Music therapy (1). The small number of files here are not statistically significant, and the information was used only indicatively, e.g. to indicate areas of good or poor practice.

HMIe inspectors scrutinized a small, but statistically insignificant number of education records.

The data from these activities were combined with the self-assessment data supplied by authorities being inspected, and observations and follow-up interviews by inspectors. This ‘triangulation’ method is summarised in Figure 2. Inspectors met at the end of each day of the 2 week inspection to review the data collected. They discussed inspection activities related to each of the outcome indicators and identified any inconsistencies in different sources of information. Further investigation was then undertaken as necessary, to verify the quality of the service being provided.

Figure 2

The collection and verification of outcomes by triangulation.

Results

Self evaluation questionnaires

In addition to the information collected (Tables 2a and 2b), self-evaluation questionnaires were completed by each of the three partnerships (North, South and East Ayrshire councils in partnership with NHS Ayrshire and Arran in each case). Each partnership evaluated their services against the 11 outcome indicators using an agreed rating scale (Table 3), and these self evaluations were then compared with inspectors own evaluation, which was based on all the data collected from sources described above. The joint inspection report assessed the closeness of the match between the self-evaluation of outcomes and the information collected during inspection.

Table 3

Rating scale for service evaluation

Level	Description	Definition
6	Excellent	Outcome was achieved in full with excellent or outstanding examples of practice
5	Very good	Outcome was achieved in full with major strengths in some areas
4	Good	Outcome was achieved with minor shortfalls. There are important strengths with some areas for improvement
3	Adequate	Outcome was just achieved. Strengths just outweigh weaknesses and there are some significant shortfalls
2	Weak	Outcome was not achieved at all and there are important weaknesses in practice in some areas
1	Unsatisfactory	Outcome was not achieved at all and there are major weaknesses and concerns requiring immediate action

The Social Work Inspection Agency (SWIA) currently uses six categories to describe performance. These categories, with some modifications, were adopted for the integrated inspection process, to rate each of the outcome indicators. (Extended definitions of criteria were also available to services being inspected and inspectors.) Each category is judged in response to the question, “To what extent was the outcome achieved?” (Table 3).

A summary of these ratings for each of the three partnerships in Ayrshire is given in Table 4. Full details of each of the 11 outcome indicators, including sub-indicators for each, are given in the full joint inspection report.

Table 4

Overall ratings for service evaluation using quality outcome indicators

Quality outcome indicator	North Ayrshire partnership	East Ayrshire partnership	South Ayrshire partnership
1. Enabling and sustaining independence	Good	Very good	Excellent
2. Promoting inclusion	Good	Good	Very good
3. Meeting Healthcare Needs	Good*	Good*	Good*
4. Safety and protection	Good	Good	Good
5. Record keeping and communication	Good	Good	Good
6. Meeting staff needs	Good	Very good	Good
7. Developing partnership working	Good	Good	Good
8. Leadership and direction	Adequate	Good	Good
9. Financial resource and information management	Good	Good	Good
10. Lifelong learning	Very good	Good	Good
11. Capacity for improvement	Good	Very good	Very good

‘Meeting Healthcare Needs’ was evaluated across the whole of Ayrshire, as NHS Ayrshire and Arran provide healthcare services across all three local authority areas.

Quality outcome indicator evaluation

In the main inspection report [1], 35 separate recommendations for improvement were made, and 19 good practice examples were highlighted. These ranged across the 11 quality outcome indicators, for all three partnership areas.

Objectives of integrated inspection

The main objectives of integrated inspection had been agreed in the planning phase, which lasted six months. It was from these general objectives that a set of suitable outcome indicators was developed, to give agreed measures of quality. The original objectives are given in Table 5 [4], with some preliminary comments on the success of each. This inspection is being formally evaluated by independent consultants, and findings from this will be made available by SWIA2.

Discussion

The main challenge of developing and using a model for multi-agency, integrated inspection was to make that model and the related inspection activities complex enough to meet the need, but simple enough that the process and the outcomes could clearly be understood by everyone involved. Overall, this first integrated inspection was a qualified success, both in terms of the model used and the process. The inspection was carefully planned and pre-inspection and inspection activities sought to fully involve all major stakeholders. The majority of the main objectives of the integrated inspection were achieved (Table 5). Outcome indicators used for the inspection were robust and focussed on actual outcomes for service users. The outcomes identified by service users and carers were included in the model of inspection, in line with national policy on service user involvement at all levels [11], and staff in services were clear about the link between outcome indicators used and the work they were doing.

The data collected were a combination of case record audits, review of policy and procedure documents and interview responses from services users, proxy respondents and carers about the quality of services.

The inspection was only a qualified success however, as it left a number of questions unanswered. The most important of these were about difficulties accessing health and education records, the timetable for future integrated inspections, and leadership of the process. These questions will be discussed in turn.

Inspectors were not able to scrutinise service users' health records, because of existing Scottish legislation regarding confidentiality, and the lack of a local protocol. It was hoped that a local protocol might be agreed between the health services and inspectors before the inspection, but this did not happen. Instead a self-audit system by the health authorities was used. Staff from the NHS Ayrshire and Arran inspected the records then reported the results to the inspectors. This was a ‘second best’ to independent inspection of the health records. Similarly the number of education records accessed was very small and not statistically significant for the population. This was because only a select few inspectors on the integrated team (inspectors from Her Majesty's Inspector of Education HMIe) had the required authority to view these records. Clearly, if inspections in the future are to be truly ‘integrated’ and look at proportional numbers of social work, health and education records then all members of the integrated team should have equal jurisdiction to inspect the data available, i.e. all records.

In total, the time from planning the first integrated inspection to final publication of the first report was 16 months (December 2005–April 2007). Whilst some of the time taken for initial planning and development of an inspection model would not be necessary in subsequent integrated inspections, there are some practical concerns about how long it would take to inspect all of Scotland's 15 National Health Service areas, with their local authority partners. No schedule of inspections has yet been set, and even on the most optimistic timescale it would take between 3–5 years to inspect all areas. This means that some health authorities providing services for people with learning disabilities will not have been inspected for at least 7 years, which is unacceptable. The process of multi-agency, integrated inspection will need to be ‘streamlined’ if it is to be effective nationally. The subsequent inspections will need to be as thorough as this first pilot inspection, but with shorter lead in times and a final report within a few weeks, rather than many months.

One of the inspection agencies (SWIA) took a lead role in the first integrated inspection. This management of the process was essential for effective communications and co-ordination during the planning of the inspection, the inspection activities and production of the report. As the lead agency, SWIA committed by far the most material resources and personnel in preparing and carrying out the inspection. Because the precedent has been set, it seems likely that the same inspection organisation will act as the lead agency for future inspections. For this arrangement to succeed for future inspections, it will be necessary for all the other inspection agencies involved to commit to the process, by pledging proportional amounts of time and resources, perhaps in the form of a formal memorandum of agreement.

There remains some anxiety in individual agencies about being absorbed into some future ‘Joint Inspectorate’ body and it will be necessary to clarify the continuing role of each of the individual organisations in the context of integrated inspection.

Different professional and voluntary bodies have different cultures, and partnership working is thus a challenge. Whether any one organization is best placed to act as an ‘honest broker’ bridging such cultural differences remains to be seen.

Following the publication of the integrated inspection report the next steps will be to ask each partnership to prepare an action plan, setting out clearly how the partnerships will implement the recommendations of this report. When the integrated inspection process is well established it is envisaged that the role of inspection will be principally one of verification, rather than inspection, of the self-evaluation information and the methods being used by services. Although the first integrated inspection has given some hope that this may be possible in the future, establishing a robust and repeatable process for inspecting services within a reasonable timeframe remains the first priority. A second priority is to monitor the effects of this first inspection; inspecting services should improve them, and the impact of the inspection on the quality of services needs to be evaluated. It is important to see the information collected as baseline data, which can be compared with data collected in the future, either by the service itself, or in a follow-up inspection.

Finally, an unresolved question from this first integrated inspection is whether the inspection process is driven by compliance or by commitment of both the inspectors and those being inspected [4]. Are services complying with imposed outcome indicators, or are they committed to the process of inspection, and are the independent social work, health and education inspectorates complying with government policy, or have they fully committed to joint inspection, and accepted the need to commit whatever is necessary in time and resources to make the process work?

An independent evaluation of this first joint inspection process has been commissioned by the Scottish Executive. This report will be made available in July 2007 to a Joint Strategic Group, who will then decide how and when the joint inspection process will be repeated. These decisions will be a good test of whether the joint inspection process is one of “compliance or commitment” (2006)—compliance with policy, or real commitment for all of the existing inspection agencies.