Quality of regional individual needs assessment agencies regulating access to long term-care services: a client perspective.

PURPOSE: To determine if and how the outcome quality from a client perspective is related to process characteristics and structure of Regional Individual Needs Assessment Agencies (RIOs) regulating access to long-term care services in The Netherlands. THEORY: Because of decentralised responsibilities, ultimo 1999 85 RIOs were set up. RIOs differ in their structural and process characteristics. This could lead to differences in client quality. Insight into factors relating to client quality (e.g. client satisfaction) can improve the needs assessment process.
METHODS: Eighteen RIOs participated in this study. These RIOs each selected 120 clients, filled in forms about their needs assessment procedures and sent them a questionnaire assessing judgements, experiences and satisfaction with the RIO.
RESULTS: We received 1916 RIO-forms and 1062 client questionnaires. Eighty-two percent of the clients were satisfied with the RIO, the percentages not satisfied clients varied from 10 to 29% among items and working procedures. Satisfaction is mostly related to what is actually done for the client. Information aspects and providing choices are important determinants of client quality with the RIO.
CONCLUSION: In improving quality seen from a client perspective, one should focus on what is actually done for the client, rather than looking at the RIOs structure.

Introduction

Like in other countries [1-3], in the Netherlands health care reforms are taking place to integrate care. Following the discussion of Kodner and Kay Kyriacou [2] integrated care can be defined as:

a discrete set of techniques and organisational models designed to create connectivity, alignment and collaboration within and between the cure and care sectors at the funding, administrative and/or provider level.

The goal of this integration is to enhance quality of care, quality of life and consumer satisfaction, and to make the system more efficient for clients with complex problems cutting across multiple sectors and providers [2]. In providing tailor made care, needs assessment is an essential part of the process. In their report on “Needs Assessment and tailor made care” the National Council for Public Health (NRV) in 1994 defined needs assessment as [4]:

the result of a formalised process of determining in an objective way the need for care and thereafter, in that respect, determining reasonable care in terms of type, and magnitude.

Needs assessment should provide the necessary information to make sound decisions on possible resources and care services in order to effectuate the claim for care. The term “objective” indicates that needs assessment can be verified. The definition also indicates that applicants and their need for care are the central focus. The assessment should not be determined by scarcity of supply up front. The term ‘care’ should be seen in a broad perspective and not be restricted to one type of service because it concerns the total need of the client. Not only factual care but also health care related services such as assistive devices, home adaptations, meals on wheels or other assistance services are meant. The result of the assessment is presented in a formalised advice, which may be judged in order to indicate whether or not the assessment justifies the claim for care. Thereafter care will be allocated, and the assessment should be made in terms of actual care supply.

The process of needs assessment is complicated due to the varied target group. In health care we are dealing with groups of people who have life long disabilities and an increasing number of people who experience disabilities in later life. These groups differ in their personal capabilities and demands and the disablement process they experience [5]. This means that the individual needs for care will also vary. This may be influenced by many factors like the amount of informal care available, environmental factors like housing or tasks and roles people have in life [5]. The needs assessor should assess how this imbalance can be restored. He has to weigh a person's capabilities and the tasks he or she wants to perform in terms of ‘strength to bear or handle the care giving situation’ and the burden.

Since 1966 long term care in the Netherlands is financed by a social insurance with the name Exceptional Medical Expenses Act (AWBZ). Private, non-profit organisations provide this care. In previous years many of them existed locally. Nowadays, after many mergers, regional monopolists exist mostly per chain of type of care: for instance only residential home care or only home health care. Each of these providers had their own entry system with nurses or social workers visiting new clients at home and matching the client's needs to the available supply. Since the 1990s a remarkable organisational modernisation of the AWBZ is taking place with which the Dutch government tries to achieve tailor made and integrated care. The government stimulates independent (of financer, client and care supplier) objective (by means of uniform, transparent and reproducible needs assessment procedures) and integral (looking at different need areas and solutions) service delivery [6,7]. This also means that supply should be finely tuned to the available societal needs instead of the other way around.

Part of this modernisation is the creation of a one-entry system for the AWBZ clients of home health care, residential homes and nursing homes. In these so-called Regional Individual needs assessment Agencies (RIOs), their needs are assessed independent of available supply. The RIO may, therefore, be seen as the start of the chain “needs assessment-allocation-service provision” (see Box 1). The municipality is responsible for the functioning of the RIO but is not the actual provider. The RIO sends the formalised advice to the Regional care insurance that distributes the budget for long-term care services. They check whether the advice is congruent and legitimate and decide on the allocation of care. This separation of needs assessment and care provision should also provide more insight into the existing needs of the population and the amount of care that should be available to fulfil these needs [6].

The service delivery process of long-term care provision can be divided in different phases

The pre-phase consists of selection, entry (first contact) and application. During the selection phase the decision is made to apply for a service, this may be performed by the client or his environment or the client is referred by a professional (e.g. by a hospital, home-care institution, old peoples home/nursing home, general practitioner, municipality). The entry and application procedure may be with the RIO or in case of a mandatory construction, with a home-care organisation. The first contact may be done by telephone or physically by going to a central or peripheral office (e.g. municipality office).

In the main-phase the actual need assessment takes place. Depending on the demand and situation of the client more or less expertise is necessary to assess the clients need and to formulate the advice. We distinguish the following three working procedures for needs assessment:

A short procedure: needs assessment takes place by telephone or on paper.

A standard procedure: needs assessment takes place during a face to face contact (during a home visit). No specialists are consulted.

An extensive procedure: as the standard procedure, but discussion in a multy disciplinary team takes place and specialist may be consulted.

Procedures could also be divided on the basis of the type of application: first application (no information available yet), application for reassessment (previous formalised advice is outdated) or a renewed assessment (clients situation changed and other care is necessary). Another way of handling applications is according to urgency, sometimes a situation is that urgent that care is provided before the formalised advice is given. In the post-phase the allocation of care takes place. In this phase the Regional care insurance agencies, care providers and clients deal with each other. Although the post-phase is not their responsibility (because of independence), RIOs may influence this phase in the following ways:

In the formalised advice they may present a range for the amount of care that should be given. This way the care provider can vary the amount of care in time without the necessity for a reassessment or renewed assessment procedure.

They may anticipate on the post-phase by formulating temporary alternative care options in case the entitled care is not available yet (e.g. home-care in case of a waiting list for a home for the elderly).

They may present an ultimate date at which care should be provided, although this has no legal consequences yet.

They may present care providers only with the formalised advice or also give them the clients dossier, which means that the provider does not need to perform an assessment as well to deliver adequate care.

They may have structural discussion with care providers about groups of clients, types of formalised advices. The client judges this post-phase by the RIOs product: the content of the formalised advice. Also important in this phase is the access to deliver a complaint or objection against the decision made.

By creating the one entry-system it is now possible to decide upon different types of care at the same time. In case a client applies for residential care, it is now easier to allocate the same quality of care at the person's home. For example by indicating specialised home care and meals on wheels. The one entry system will be expended with all types of AWBZ services: also care for the physically handicapped, mentally retarded and psychiatric patients will be included [8]. Currently, also the provision of other types of services like those for transportation, housing and wheelchairs (provided through the Service for the Disabled Act (Wvg)) is in the process of being put under the responsibility of the RIOs. This way a first step is made in the process of integrating access to services that are financed differently.

In 1995 the possibility of providing a personal budget (PGB) for home health care was introduced. Instead of receiving care-in-kind, clients with a PGB are able to organise and finance their own care within the boundaries given. Also, for the other sectors personal budgets were introduced and presently harmonisation of the different budgets takes place in order to enhance the integration of different care sectors.

Because the installation of RIOs was a political decision, reinforced by pressure from client and patient organisations, the formation of RIOs evolved with a lot of tension between the different stakeholders involved (municipalities, Regional care insurance agencies, care providers, referrers and patient organisations) [9]. RIOs differ in structural and process characteristics due to historic local differences in how the provision of care was organised and the decentralised responsibility to municipalities in how to set up the RIOs. RIOs for example may differ in the type of administration (non-profit or municipal organisation), the service package (only needs assessment and providing information or also performing a legitimacy check and monitor waiting lists), performing needs assessment for other types of services (e.g. Wvg services or not) and if they gave a mandate to home care organisations to perform minor requests themselves or not. Some of these differences are presented in Table 1.

Table 1

Structural characteristics of regional individual needs assessment agencies

	All RIOsa (n=73)
Administrative organisation
Type of administration (%)
non-profit organisation	71
municipal organisation	29
RIO has a client or user platform
yes	8
no	87
in development	4
The RIO board is the formal employer of the personnel
yes	56
no	33
partly	11
The RIO has employees detached by other organisations
yes, by home-care organisations	20
yes, by hospitals	12
yes, municipal health offices	22
yes, by municipalities	23
no	45
Working area and tasks
Service package (%)
basicb	36
extensivec	63
Performing assessments for applications concerning the Service for the Disabled Act(%)d
yes	45
no	55
Working procedures
The RIO has protocols concerning the working procedures
yes	62
no	1
in development	29
other	8
Place were applications enter the RIO (%)
central office within the RIO	83
peripheral office of the RIO	18
patient transfer office within hospital	48
offices of other organisations	29
Mandate of small home-care requestse(%)
yes	30
no	70
Size of service region (number of inhabitants * 1000) (mean, standard deviation)	193.7 (119.4)
Number of municipalities the RIO works for (mean, standard deviation)	6.3 (4.4)

aThe figures are derived from a survey among the managing directors of 73 of the 85 RIOs existing ultimo 1999, providing facts and figures on the development of their RIO [9]; b needs assessment + information & advice; c basic + other types of services like performing a legitimacy check; d transportation facilities, wheelchairs and small home adoptions; e unknown for 1 RIO.

The role, performance and effects of the RIOs is much debated [7,10,11]. It is thought that the RIOs led to more bureaucracy and inequalities in care provision. In 1999 a first start was made in conducting a national evaluation to assess the performance and development of the RIOs [12]. The results presented in this paper are derived from a part of this national evaluation, focusing on the quality of RIOs seen from a client perspective. Client quality is here defined as how clients judge the service [13]. The degree of client satisfaction is an outcome from which inferences about the quality of the process and structure can be made [14,15].

The purpose of this study is to determine if and how the outcome quality from a client perspective of RIOs is related to RIOs structural and processes characteristics. Insight in factors relating to client quality (e.g. client satisfaction) can improve the needs assessment process.

Methods

Structure, process and outcome

In order to evaluate the quality of performance of the RIO, three kinds of information are available: the structure, process and outcome. Structure is defined as “physical and organisational properties and settings in which care is provided”, process is “what is actually done for the client”, and outcome is “what is accomplished for the client”. The quality of the structure will influence the quality of the process and the outcome [14,15].

To give an answer to the research question, information on the structural characteristics of RIOs was derived from a part of the national study performed among managers of the RIOs. They provided “facts and figures” on the development of their RIO [9]. The way information on the process and outcome was gathered is presented below.

Study design

Selection of the RIOs

From the 73 RIOs participating in the first part of the national evaluation [9] 32 RIOs were selected for participation in this study. To reduce the influence of particular problems very small and very large RIOs may have, a selection was made on the basis of the number of inhabitants in the service region (RIOs who serve 120,000 to 350,000 inhabitants). Another selection was made on the basis of the number of municipalities the RIO works with (minimal 3 and maximal 11 municipalities). In this way the positive influence on efficacy and uniformity when only dealing with one or two municipalities is reduced. The influences of specific problems RIOs with very large numbers of inhabitants may encounter were excluded as well. The last selection was made on the basis of months of existence (start before 1-1-1999) of the RIO. In this way the influence of the quality of needs assessment procedures due to starting problems of younger RIOs was excluded. This resulted in 32 of the 73 RIOs meeting the criteria. A random sample of 20 RIOs from these 32 was approached for participation. For each RIO not able to participate a new one was drawn from the 12 remaining.

Selection of clients

In order to select clients, a selection was made using the needs assessment procedures as described in Box 1: the short, standard and extensive procedure. The three different procedures could lead to different experiences in satisfaction. Therefore, per RIO for each procedure, starting from a fixed point in time, the first 40 clients who where sent a ‘formalised advise’ were included in the study (120 clients per RIO in total). The total study period was 5 months, starting March 2001. Applications for reassessment and urgent cases were excluded.

Instruments

Organisation and structure of the RIOs

From the national study on facts and figures of the RIOs the following data were used for further analysis: service package, performing assessment for applications concerning the Service for the Disabled Act, mandate for small home-care requests, central entry point for application. Other data, as for example information on the number of employees, type of computerisation and use of protocols were not used for further analysis because these are frequently subject to change [9].

RIO-form and client questionnaire

For each of the 120 clients included, the RIO filled in a coded form to provide more detailed information on the needs assessment procedure of the client. Items on this RIO-form were: gender, age, living arrangements, first application of the client or not, type of needs assessment procedure, days between entry and formalised advise given, days between entry and home visit, number of contact moments with the client, type of advise (positive, negative or changed), referred by a professional or not, type of care demanded.

The RIOs sent the client a questionnaire with a code corresponding to that on the RIO form. The client returned the anonymous completed questionnaires in a postal envelope to the researches. All clients received a reminder after two weeks. The questionnaire consisted of items concerning:

Background information of the client: gender, age, assistance with filling in the questionnaire, self-assessed health and the Barthel Index (BI) [16]. The BI was used to measure the degree of independence in activities of daily living (ADL). Scores: very severely disabled (0–4), severely disabled (5–9), moderately disabled (10–14), mildly disabled (15–19), independent (20).

Clients judgement on aspects of the primary process of the RIOs service delivery (see Table 4).

Table 4

Clients experiences with the RIO, differences between procedures

Aspects of process	Na	Answer (%)	Difference between proceduresb
RIO was able to address all questions clients had	954	91
Client received letter with formalised advice	981	69
Care possibilities other than applied for were discussed	964	52	**
A choice between a person-linked budget or regular care in kind was given	905	23	*
One did reckon with the amount of care the informal carer giver gives	467	40
An ultimate date at which care should be provided was stated in formalised advice	754	47	***
The range of amount of care was stated in formalised advice (e.g. 4 to 8 hours home care)	645	47	*
Temporary alternative care is mentioned in case entitled care is not available (e.g. in case of waiting lists)	534	34
Outcomes
Client understands letter with formalised advice	676	76	**
Client knows how to put forward a complaint/objection against decision	999	54
Client appreciates discussion of care possibilities other than applied for	964	58
Client appreciates given the choice between a person-linked budget or regular care in kind	904	40
Client is of opinion that one did reckon enough with the amount of care the informal care giver gives	467	40
Client agrees with the formal advice given	972	90
It is clear to the client how the RIO arrived at the formalised advice	973	78
Client appreciates it when ultimate date at which care should be provided is stated in formalised advice	754	75
Client appreciates it when range of amount of care is stated in formalised advice (e.g. 4 to 8 hours home care)	645	62
Client appreciates it when alternative care is mentioned in case entitled care is not available (e.g. in case of waiting lists)	534	71
Client states the number of persons he/she had contact with as ok.	976	87
Client knows what will happen further, now the formalised advice is given	958	56	***

a number of applicable answers; b significant differences between working procedures; short, standard and extensive; *p<0.05; **p<0.01; ***p<0.001.

Client satisfaction with the RIO service delivery process was measured with seven items used in a previously conducted study [17]. The item on satisfaction with “the advised care” was added. For each of the satisfaction items the user was asked to rate his or her satisfaction using the following scale: 5. very satisfied, 4. quite satisfied, 3. more or less satisfied, 2. not very satisfied, 1. not satisfied at all. The Cronbach's alpha coefficient of internal consistency reached 0.91 for the set of eight items. A RIO overall satisfaction score was calculated by adding the ratings of the valid responses and dividing this figure by the number of valid responses. Cases with three or more missing item responses were scored as missing for the RIO overall satisfaction score. To present the percentage of clients, who were ‘not fully satisfied’ with aspects of the service delivery process, the individual satisfaction scores were categorised into ‘not fully satisfied’ (scores 1 and 2) and ‘satisfied’ (scores 3, 4 and 5).

The client questionnaire, the RIO-form and the study procedures and logistics were tested in a pilot study with two RIOs.

Analysis

For all statistical analysis the SPSS package version 9.0 for Windows 95 (Statistical Package Social Sciences International BV, Gorinchem, the Netherlands) was used. Variables were compared according to their distribution with chi-square or ANOVA analysis where appropriate. In order to determine the relation between client overall satisfaction and possible determinants, Mann-Whitney test, Kruskal-Wallis test, or Spearman Rank correlation was used where appropriate.

Results

Response

RIOs

All 32 RIOs had to be approached in order to find 20 RIOs agreeing to participate (38% non-response). Two of them dropped out before start of the study. Reasons for not wanting to participate and for drop out were: shortness/sickness of staff, reorganisation, backlog in assessment of client applications, no time. Looking at organisational and structural differences between participating and non-participating RIOs, participating RIOs more often had a central entry point for application than those not participating (respectively 94% and 57%) (Table 2). The participating RIOs also more often had a more extensive service package (respectively 78% and 43%). These differences were not significant.

Table 2

Regional Individual needs assessment agencies (RIOs): aspects of organisational structure of the total study population, non-participating and participating RIOsa

	Total study population (n=32)	Participating RIO (n=18)	Non-participating RIO (n=14)
Type of administration (%)
non-profit organisation	69	61	79
municipal organisation	31	39	21
Service package (%)
basicb	38	22	57
extensivec	63	78	43
Performing assessments for applications concerning the Service for the Disabled Actd
yes	38	33	43
no	63	67	57
Mandate of small home-care requestse(%)
yes	26	29	21
no	74	61	69
Central office within the RIO where applications enter (%)
yes	78f	94	57
no	22	6	43

aThe figures are derived from a survey among the managing directors of 73 of the 85 RIOs existing ultimo 1999, providing facts and figures on the development of their RIO [9]; b needs assessment + information & advice; c basic + other types of services like performing a legitimacy check; d transportation facilities, wheelchairs and small home adaptations; e n=17; f there is a significant difference (p-value<0.001) between participating and non-participating RIOs.

The RIO-form and client questionnaire

The 18 RIOs returned 1916 RIO forms (88.7% of the planned 2160). We received 1062 client questionnaires of which in 42 cases we did not receive the corresponding RIO forms. This implies that at least 1958 client questionnaires were sent. Some of the RIOs were not able to send all their 120 questionnaires, this because of logistical problems or because they did not manage to include 40 clients for the extensive working procedure within the time span of the study.

A client non-response analysis was made on the basis of the information on the RIO-forms. No significant differences were found for gender, age, the working procedure, the days between entry and the formalised advice, the days between entry and the home visit being performed, and the formalised advice. Respondents were significantly more often referred by a professional than non-respondents (56% and 49% respectively). On average they also had more contact with the RIOs than non-respondents did (with 1.4 and 1.2 times respectively).

Needs assessment procedures and dif-ferences in background characteristics

In Table 3 the results of the RIO-form and client questionnaire are presented for those clients who returned the client questionnaire. As discussed before the total results of the RIO-form did not significantly differ from those clients who did not return the questionnaire. When looking at the results of the RIO-form they showed that almost three-quarter of the respondents was female. The group had a mean age of 72 years with a median of 77 years. More than half (53%) lived alone and for 45% of the clients it was their first application. Almost half (49%) of the clients was referred by a professional. On average it took 20 days (median 10) before the formalised advice was given. In case home visits are performed (during standard and extensive procedure) it took on average 21 days before the visit was made (median 10). In 92% of the cases the needs assessment resulted in a positive advice for the care demanded. Except for gender, the background variables of clients differed significantly for the three assessment procedures. Clients who went through the extensive procedure were older, more often lived alone, and more often had applied for a service before. The time between entry and home visit or formalising the advice was longer and there were relatively more negative or changed advises. Within the short procedure the assessment mostly concerned requests for home care. Within the extensive procedure, especially requests for enrolment in old people and nursing homes were treated.

Table 3

Differences in client characteristics between working procedures

	Short procedure n=293	Standard procedure n=435	Extensive procedure n=257	Total n=985a
Figures based on RIO-form
Female (%)	75	76	73	75
Mean Age (SD)	67 (17)	70 (17)	80 (10)	72 (16)	***
P50 (P25–P75)	73 (56–79)	76 (64.8–82)	82 (77–86)	77 (67–83)
Living alone (%)	51	45	68	53	***
Housing (%)
living independently	95	92	79	89	***
living dependently	5	8	21	11
First application (%)	48	53	30	45	***
Enrolled/referred by (%)
personally/family/friends	41	61	48	51	***
professional	59	39	53	49
Days between entry and formalised advice
mean (SD)	9.8 (18.2)	21.2 (29.6)	30.7 (27.6)	20.3 (27.3)	***
P50 (P25–P75)	3 (0–11)	10 (5–24.5)	22 (11–43)	10 (3–25.5)
Days between entry and home visit
mean (SD)	n.a.	19.8 (28.6)	26.0 (34.9)	22.0 (31.2)
P50 (P25–P75)		9.0 (4.0–22.0)	15.5 (7.0–35.3)	11.0 (5.0–28.0)	*
Days between home visit and formalised advice
mean (SD)	n.a	2.0 (5.5)	6.4 (9.7)	3.6 (7.6)	***
P50 (P25–P75)		0 (0–1)	5 (0–8)	0 (0–5)
Number of times contact with client: mean (SD)	1.3 (1.6)	1.4 (1.1)	1.4 (0.8)	1.4 (1.2)
Advice on assessment (%)
Positive	97	92	88	92	**
negative or changed	4	8	12	8
Demand (%)
home care	88	62	11	57	***
partly institutional care	1	5	17	7	***
enrolment residential home	2	19	51	22	***
enrolment nursing home	4	5	28	11	***
welfare services		0	0	0
transportation services (SDA)	3	3	0	2	*
wheelchairs (SDA)	0	1		1
home adaptations (SDA)	1	5	2	3	**
Figures based on client questionnaire
Received help with filling the questionnaire % Number of persons client had	48	61	86	64	***
contact with
none	12	5	6	7	**
one or two	65	67	65	66
three and more	23	29	29	27
Self reported health (%)
excellent-good	23	23	15	21
fair	30	29	30	29
moderate-poor	46	49	56	50
Barthel Index, mean (SD)b	17 (4.1)	16 (4.5)	14 (5.4)	16 (4.8)	***
P50 (P25–P75)	18.5 (15–20)	18 (14–20)	16 (11–19)	18 (14–20)

a Total not equal to n=1062: in 77 cases the procedure is unknown; b Barthel Index range 1 very disabled-20 independent; P50 (P25–P75): median with 25th and 75th percentile; SDA=Service for the Disabled Act; *p<0.05, **p <0.01, ***p<0.001.; n.a.=not applicable; SD=standard deviation.

When looking at the results of the client questionnaire it was noticed that most clients received help with filling in the questionnaire, especially in case of the extensive procedure. The self-reported health could be called bad, 79% perceived their health as fair to poor. The Barthel Index indicated that 25% of the clients was very severely to moderately disabled. The degree of independence differed significantly between the working procedures; clients assessed within the extensive procedure were relatively more dependent.

Client experiences with the RIO: process and outcomes

Client experiences with the RIO are presented in Table 4. The vast majority (91%) of clients felt that the RIO could address all their questions concerning care or services. Almost a third of the clients stated that they did not receive a letter with the formalised advice. The percentage of clients receiving a letter and understanding it differed between procedures (80%, 80% and 60% for the short, standard and extensive procedure, respectively). In more than half of the cases other care possibilities than applied for were discussed, this percentage differs significantly between the procedures (44%, 56% and 55%, respectively). For all procedures more than half of the clients would appreciate it when other care possibilities were discussed. Nearly a quarter of the clients was offered the choice between a person-linked budget and regular care in kind (19%, 27% and 20%, respectively) but about 40% of all clients would appreciate having the choice. In the short procedure more often an ultimate date at which care should be provided was stated in the formalised advice than for the standard and extensive procedure (62%, 39% and 46%, respectively). Three quarters of the clients though would appreciate this being mentioned.

Almost half (46%) of the clients did not now how to put forward a complaint or objection against the decision made. Of the clients having inmates giving informal care (45%), 75% thought that the amount of informal care given was taken into account sufficiently. Although 90% agreed with the outcome of the assessment, for a fifth of all clients it was not clear how the RIO arrived at the advice. In more than half (56%) of the cases clients did not know what would happen once the advice was given. For the short, standard and extensive procedure these figures were respectively 34%, 47% and 52%.

Client satisfaction with aspects of the needs assessment process

In general overall satisfaction with the RIO was high and did not significantly differ between the working procedures (Table 5). When looking at item level, in most cases the percentage of clients being not fully satisfied was the highest for the short procedure. Only when satisfaction about the advised care was concerned more clients in the standard procedure were not fully satisfied. About a quarter (26%) of the clients was not fully satisfied with the quality of information they received from the RIO. Almost a quarter (23%) was not fully satisfied about the access and time period between entry and receiving the advice. In general clients were most satisfied about the service (86%). When asking clients to judge the satisfaction with all aspects in one item, then a fifth of the clients that went through the standard procedure was not fully satisfied.

Table 5

Client satisfaction with aspects of the RIOs needs assessment processa

	Short procedure	Standard procedure	Extensive procedure	Total
Percentage ‘not fully satisfied’ clients
1. quality of information	29	25	25	26
2. access	29	22	23	24
3. co-operation	23	22	19	22
4. competence	20	14	14	16
5. service	18	14	10	14
6. own opinion	18	19	13	17
7. time period	24	22	23	23
8. advised care	17	23	20	21
RIO-satisfaction scoreb	4.1 (0.8)	4.1 (0.8)	4.1 (0.7)	4.1 (0.7)
P50 (P25–P75)c	4.1 (3.6–4.8)	4.3 (3.8–4.8)	4.3(3.8–4.8)	4.3 (3.8–4.8)
Percentage not fully satisfied clients when judging all aspects together (1 item)d	16	20	17	18

a Of the 1062 clients 1002 answered 1 or more items concerning this table; b For 945 clients a RIO satisfaction score could be calculated. Range 1 not satisfied at all – 5 very satisfied; c P50 (P25–P75): median with 25th and 75th percentile; d taking all aspects into account, how satisfied are you about the assessment of your application?

Aspects related to overall satisfaction with the RIO

A RIO overall satisfaction score was calculated for 945 (89%) of the 1062 respondents. The non-parametric Kruskal-Wallis test showed that there is a significant difference in overall satisfaction between the 18 RIOs (alpha=0.034). The scores range from 3.97 (SD 0.69) to 4.42 with a mean overall score of 4.11 (SD 0.74). When removing the scores of two outlying RIOs (scores 4.42 and 4.40, respectively) no differences in satisfaction exist between the 16 remaining RIOs. The two outlying RIOs did not obviously differ from the other RIOs when looking at structural characteristics as presented in Table 1.

In order to see which items are related to client satisfaction with the RIO, all the items in Table 2, 3 (except demand) and 4 were related to the client's overall satisfaction scores. Because of the number of tests performed a p-level of 0.01 was considered as significant. The analysis first was performed with all the RIOs and then without the two outlying RIOs, both methods led to the same results, presented in Table 6. As shown in Table 6 most relating items concerned the outcome and process. Because the structure of RIOs concerns another level of information gathering, in Table 7 satisfaction scores per type of RIO are presented. For none of the structural characteristics an association with the overall satisfaction score of the client was found.

Table 6

Aspects associated with RIO overall satisfaction

Background variables
First application client	x
Client received help with filling the questionnaire	x
Day's between entry and advice	x
Day's between entry and home visit	x
Client is a woman	x
Client lives independent	x
Client lives alone	x
Age	x
Self reported health	x
Aspects of the process
RIO was able to address all questions clients had	+++
Temporary alternative care is mentioned in case entitled care is not available (e.g. in case of waiting lists)	+++
The range of amount of care was stated in formalised advice (e.g. 4 to 8 hours home care)	+++
Care possibilities other than applied for were discussed	++
The number of persons the client had contact with	++
An ultimate date at which care should be provided was stated in formalised advice	x
A choice between a person-linked budget or regular care in kind was given	x
Outcomes
Client is of opinion that one did reckon enough with the amount of care the informal care giver gives	+++
It is clear to the client how the RIO arrived at the formalised advice	+++
Client agrees with the formal advice given	+++
Client states that number of persons they had contact with is ok	+++
Client knows what will happen further, now the formalised advice is given	+++
Client knows how to put forward a complaint/objection against decision	+++
Client appreciates it when a temporary alternative is mentioned in case entitled care is not available	+++
Client understands letter with formalised advice	+++
Client appreciates it when ultimate date at which care should be provided is stated in formalised advice	++
Client appreciates the choice between a person-linked budget or regular care in kind	++
Client received a positive advice	x
Client appreciates the discussion of care possibilities other than applied for	x
Client is of opinion that one did reckon enough with the amount of care the informal care giver gives	x
Client appreciates it when range of amount of care is stated in formalised advice	x

x tested but not significantly related to overall satisfaction score; ++ significant at 0.01 level; +++ significant at 0.001 level; Mann-Whitney test or Spearman Rank correlation where appropriate.

Table 7

RIO structurea and satisfaction scores

	Clients n(%)	RIO overall satisfaction scoreb mean (SD)	Not fully satisfied clients 1 itemc (%)
Type of administration
non-profit organisation	590 (38)	4.1 (.7)	18
municipal organisation	355 (62)	4.1 (.7)	19
Service Package
basicd	224 (26)	4.1 (.7)	19
extensivee	721 (74)	4.1 (.8)	18
Performing assessments for the Service for the Disabled Actf
yes	321 (44)	4.1 (.7)	17
no	624 (56)	4.1 (.8)	19
Mandate of small home care requestsg
yes	36 (27)	4.1 (.8)	20
no	653 (73)	4.1 (.7)	18
Central office within the RIO where applications enter
yes	915 (97)	4.1 (.7)	18
no	30 (3)	4.0 (.8)	24

a structure aspects from the study of Schrijvers et al. [9]; b range 1 not satisfied at all – 5 very satisfied; c taking all aspects into account, how satisfied are you about the assessment of your application?; d needs assessment + information & advice; e basic + other types of services like performing a legitimacy check; f transportation facilities, wheelchairs and small home adaptations; g unknown for 1 RIO; SD=standard deviation.

Discussion

Results

The purpose of this study was to determine if and how the outcome quality from a client perspective of Regional Individual Needs Assessment Agencies (RIOs) is related to the processes and structure of these RIOs. Although most clients agreed with the given advice (90%) and 92% received the type of care they asked, and satisfaction in general was high (82%), when looking at the needs assessment process the percentages not fully satisfied clients varied from 10% to 29% among items and working procedures.

When transparency of the needs assessment procedure is concerned the quality of information is important. More than a quarter was not fully satisfied with the quality of information they received. A high percentage of clients would appreciate it if the formalised advice contains an ultimate date for realisation of care, a range for the amount of care to be realised, and an indication of temporary alternative care in case the advised care cannot be delivered. More than half of the clients did not know what would happen after the advice was given or how to put forward a complaint. These items were significantly related to satisfaction and are, therefore, important to pay attention to.

Where efficiency is concerned improvements could be made as well. Almost a quarter of the clients was not fully satisfied with the time period between entry and the formalised advice. The time period was especially long for the standard and extensive procedure. In these cases also a home visit was performed, the organisation of which seemed to consume most of the time. After the visit, the formalised advice was given within a couple of days. Also the number of persons clients had contact with is related to the level of satisfaction. Almost a quarter was not fully satisfied about the competence of the professional involved in the process.

The fact that 92% of the clients received a positive advice, meaning that they received the type of care they applied for, raises the discussion if efficiency improvements could be made. Why maintaining an expensive needs assessment system when most requests are honoured? Of course the amount of care has to be decided upon, but is it really necessary to assess all types of requests the way it is organised now? More attention should be paid to standardisation of certain types of requests. This would also have an effect on the speed of the needs assessment process.

When looking at integration of care, in most cases (91%) clients stated that the RIO could address all their care questions. It is, therefore, remarkable that almost half of the clients stated that other options for care or services than applied for were not discussed and that more than half of all clients would have appreciated to have this opportunity. The range of services RIOs offered had no influence on client satisfaction, but from a political point of view expanding the tasks of RIOs with for example needs assessment for services of the Service for the Disabled Act is seen as an improvement of professionallity and integration of care [18]. Like is stated by Øvretveit [13], the way in which clients judge or perceive a health service is related to what they think the service will or should provide. A client is dissatisfied when their experience of the service is less than their expectations or assumptions. So when clients know what to expect from the RIO (what is the type and range of service delivery) it is likely that the organisation and structure does not influence the satisfaction level, as we found in this study. When expectations are low (for example because of lack of information) then there is less reason for dissatisfaction. Most clients do not have personal experience with other RIOs to compare with. This could be an explanation for the finding that the working procedure does not influence the level of satisfaction. It is the details and personal experiences with the RIO, which compile the level of satisfaction.

Limitations of the study

We realise that the study has some restrictions. In selecting the 32 RIOs the figures are not representing the general Dutch situation but those of the modal RIO. Because a selection was made on the basis of the population density and the number of municipalities RIOs work with, no inferences could be made about the effect of these factors on satisfaction.

Looking at RIOs reasons for non-participation, it is possible that participating RIOs were better organised. They more often had a central entry point within the RIO, which would make the inclusion of clients easier and RIOs more likely to participate. They also more often had an extensive service package, which could be an indication of being better organised/more professional. These factors may bias the satisfaction levels, implying that the percentage of clients not fully satisfied is an underestimation.

Implications

Legally the post-phase (Box 1) is not the concern of the RIO, but it seems necessary to provide clients with more information explaining what will happen after they received the formalised advice. Here we also have to note that satisfaction with the needs assessment process and the RIO does not guarantee satisfaction with the total chain of service delivery. Waiting lists and actual care provision are the concern of the Regional care insurance agencies but the quality of this post-phase will have an impact on the client's overall satisfaction with service delivery and the care provided. The main goal of clients is not needs assessment but the actual delivery of services fitting their needs. If the last stagnates because of waiting lists and organisational gaps, the care indications formulated in the advice are without value. In this respect, assessing the quality of the post-phase in terms of client satisfaction and effectiveness of the services provided seems important.

Looking at the results we may conclude that the goals of independent tailoring, objectivity and working from an integrated approach seem to be partly reached. Partly, because of three reasons. Firstly, formally the RIO is responsible for the needs assessment of Wvg services, but this is not yet completely effectuated. Secondly, the fact that 92% of the clients received a positive advice may be because their demand is supply-oriented and/or because RIOs do not follow an integrated approach. Thirdly, the finding that according to 48% of the clients no other care possibilities than asked for were discussed, indicates that integration of care is not fully reached yet. Although the RIO is responsible for the needs assessment of home care, residential and nursing home care, the responsibility for other care sectors like mental health care and that for the physically disabled still has to be established. This could lead to further integration of the different sectors.

Although the introduction of client linked personal budgets is a hot political topic in the Netherlands, this paper showed that the majority of clients do not prefer them. This could be because of lack of information about this new option. Another reason could be that clients are too disabled or too ill to shop with their personal budget around different providers, asking for offers and playing the role of employer. This result means that the provision of care-in-kind will remain important and together with the personal budgets should be presented as two equal alternatives.

In terms of client satisfaction the quality of the RIO is perceived as high but improvements can be made. The results indicate that it is more important to focus on the needs assessment process itself (what is actually done for the client), rather than on structural aspects like organisational and management structures of RIOs. A complete analysis of the relation between client satisfaction and the organisational RIO characteristics though could not be established with the available data. Providing information and giving clients a choice are important issues to focus on in improving quality from a client perspective.