Do integrated mental healthcare organisations facilitate process quality in the treatment of people with schizophrenia and related psychoses?

OBJECTIVE: The objective of this study is to investigate the influence of mergers of ambulatory and mental healthcare organisations on the process quality of care for persons suffering from schizophrenia or related psychoses. THEORY: On the basis of the theory of Donabedian we assume the relationships between three types of quality in healthcare: structure quality, process quality and outcome quality. This study focuses on the influences of structure quality, i.e. years since merger and catchment area size upon process quality.
METHODS: Criteria according to Tugwell for evaluating healthcare were used to describe the process quality of schizophrenia care, resulting in a process quality questionnaire with 6 subscales and 21 items. Leading psychiatrists of 31 Dutch mental healthcare organisations, covering 89% of the country, answered the questionnaire. Both programmes and documents from the responding institutions and schizophrenia projects were analysed. Correlations of two determinants, age of the merged organisation and catchment area size, were made with total scale scores and the sub scores of the questionnaire.
RESULTS: The response rate was 97% (31/32). Twenty-two organisations (71%) had a score of more than 50% on the used scale, 8 (29%) scored less. Two evidence-based interventions were implemented in more than 50% of the organisations, three in less than 50%. A low degree of implementation occurs in establishing care for people with schizophrenia from ethnic minorities, standardising diagnostic procedures and continuity of care. No significant relationship between the age of the merged organisation ('age') and the total process quality of schizophrenia care was found, however, the relationships between age and the subscales availability of interventions and integrated treatment were significant. No association was found between the size of the MHO's catchment area and any of the used subscales.
CONCLUSIONS: The age of integration of residential and ambulatory mental health institutions correlates significantly with two subscales of process quality of schizophrenia care, i.e. availability of interventions and treatment. Catchment area size is not significantly associated with process quality or any of the subscales. Despite the mentioned positive effects, the overall picture of schizophrenia care is not very positive. Additional forces other than merely integration of ambulatory and residential services are needed for the further implementation of evidence-based interventions, diagnostic standards and continuity of care. The development of a national 'schizophrenia standard' (like in other countries) in relation with implementation plans and strategies to evaluate care on a regional level is recommended as well as further research on patient outcomes in relation to mergers of mental healthcare organisations.

Introduction

World-wide, schizophrenia has a stable incidence of 0.16–0.42 per 1000 and a prevalence of 0.5–1.6% [1]. On the basis of these figures, about 100,000 people suffering from schizophrenia would be indicated in the Netherlands (population: 16 million). A recent Dutch epidemiological study pointed out that between 70,000 people between the ages of 18 and 65 years suffer from the disease [2]. About half of them are in the mental health care system [2]. In a quarter of the cases total recovery is expected [1]. In approximately three-quarters of the cases it has been estimated that the disease leads to a chronic condition [2]. About 17,500 patients are receiving long-term intramural care [3]. In 2000, estimates of the yearly costs directly for schizophrenia care in the Netherlands were over half a billion Euro on a total budget of 2.5 billion Euro for Mental Health; more than 75% of this amount is spent on residential facilities [2,4]. Schizophrenia therefore, is not only a disabling disease, it is also the most expensive psychiatric disease to treat in the Netherlands [2]. On a national level, the costs are comparable with those spent on diabetes or heart failure [2,4].

During the past decades, the evidence for a number of interventions in the care for people with schizophrenia has grown. Guidelines became available for the pharmacological treatment of schizophrenia in both acute and chronic courses [5,6]. The importance of giving information to patients and their family members was acknowledged and implemented by professional associations [7]. Recognising early signs of psychosis as well as an active approach towards psychotic people who are avoiding mental healthcare is important because early treatment is associated with a better outcome [5]. Scientific evidence became available for assertive community treatment [8], case management [9] (at the time of conducting our study the Cochrane study about case management by Marshall and co-workers [10] was not yet published), cognitive behavioural therapy [11,12], and rehabilitation [13]. A recent study showed that psychiatrists agree on essential components of schizophrenia care such as outreach care in the community, rapid response, and fail-safe follow-up systems [14]. In contrast with the above, research shows that routine mental health programmes do not provide evidence-based practices to clients [15]. In addition, psycho-education and cognitive therapy improved compliance in patients who did not follow their medication advice [16].

In the last decade of the 20th century, the landscape of Dutch mental healthcare changed rapidly, as was also the case in many European and North-American countries [17-21]. A large amount of innovative initiatives were taken, consisting of projects, networks, and new facilities, all strongly related to mergers of ambulatory and residential mental healthcare [17]. These mergers resulted in new Mental Healthcare Organisations (MHOs) within the same catchment area as well as MHOs in combined catchment areas. In the former case, we see a merger between an ambulatory MHO and a residential MHO; in the latter we see a merger of one or more ambulatory MHOs with more than one residential MHO [17]. Care providers and the government aimed at integration of mental health services as well as de-hospitalisation and community-based care for psychiatric patients [22]. In most of the mental healthcare regions alternatives for admission to mental healthcare institutions were developed such as home care, mobile teams, day-treatment, day activity centres, and sheltered living [17]. Bringing together the knowledge and skills of healthcare professionals from both sides in these programmes and sometimes in newlybuilt facilities resulted in a collaboration surrounding age-categories [17]. A recent development is the (re) design of care processes for specific groups of patients, i.e. schizophrenia patients [23] on a regional level by merged MHOs.

In this study, we focus on the mentioned changes in the organisation of mental healthcare and the possible influences on the quality of the schizophrenia care process. We limited our study to schizophrenia care for patients in an ambulatory and short-stay residential treatment setting (max. 6 months) and excluded long-term schizophrenia care. The latter field, including rehabilitation strategies such as supported employment and different forms of sheltered living requires further study in a different arena with different actors, such as organisations for sheltered living.

Theory

Donabedian postulates the relationships between three types of quality in healthcare: structure, process, and outcome [24]. Structure quality is defined as the conditions that underlie the care process with such aspects as professionalism, safety, accessibility, and integration. Process quality is defined as the way in which care is provided on the aspects of attitude, method, continuity, and accountability. Outcome quality is defined as the result in measurable terms covering changes in somatic, mental, and social health. The structure quality largely determines the process quality while the process quality largely determines the outcome quality.

It is from the above theory that our research objective stems: to explain the influence of mergers of mental healthcare organisations on process quality of care for persons suffering from schizophrenia or related psychoses.

As stated above, mergers result in integrated MHOs within the same catchment area as well as integrated MHOs in larger catchment areas. Therefore our research questions are:

Does the age of mergers of ambulatory and residential mental healthcare organisations influence the process quality of care for persons with schizophrenia and related psychoses?

Does the size of the catchment area of mental health organisations influence the process quality of care for persons with schizophrenia and related psychoses?

We defined ‘Age’ as the number of years since the merger whereas ‘Size’ was defined as the population size of the catchment area. In the Dutch health system this parameter is the most important variable for budget, beds, and the number of employees.

As described above, mergers occur in those organisations where there is an increase in collaboration. Our assumption is that following mergers, newly merged organisations need time to establish (for instance) uniform attitudes, methods, continuity, and accountability. We expect this collaboration to have an increasing effect on process quality in time, based on an earlier study [17]. Our second assumption is that a larger catchment area leads to an improvement in process quality; a larger organisation, which is a direct effect of the larger catchment area, is expected to provide more professionalism in establishing uniform attitudes, methods, continuity, and accountability in order to increase process quality [17].

The Tugwell criteria for evaluation (research in healthcare) were used in order to construct a questionnaire [25]. The six Tugwell criteria are: availability of interventions, reach of the care, quality of diagnosis, allocation of care, quality of treatment, and compliance. The same model has already been used in other studies [26,27].

Methods

We conducted a survey based on a specially developed questionnaire, which was completed by the researchers during a telephone interview. In order to construct a questionnaire that would be valid in a national context, open interviews were held with four directors of mental health organisations. Both the differences in size and the merging-status of these pilot-cases are comparable with the differences in cases in the main-study. The pilot-cases were not included in the study. In order to measure process quality, the questionnaire consisted of 6 subscales of questions in accordance with the six Tugwell items. The questionnaire subscales were validated. Table 1 shows the subscales (I to VI) and specified items of this questionnaire as well as the Crohnbach Alpha of each subscale and the scale in total.

Table 1

Subscales and indicators for process quality of care for persons with schizophrenia and related psychosis following the classification of Tugwell (1984), response options, Crohnbach α's, item means and item variances

Subscale	Indicator	Names subscales and	Response	Crohnbach	Mean	Variance
		subscale items	options	α
I		Availability of
		interventions
	1	Cognitive Behavioural Therapy	0–1
	2	Pharmacotherapy guidelines	0–1
	3	Assertive Community	0–1
		Treatment
	4	Case management	0–1
	5	Family psycho-education	0–1
		Subscale	0–5	0.6319	0.6387	0.0538
II		Reach of care
	6	Addicted persons	0–1
	7	Ethnic minorities	0–1
	8	Homeless persons	0–1
		Subscale	0–3	0.7303	0.3978	0.0545
III		Diagnosis
	9	Uniform procedure in MHO	0–1
	10	Uniform procedure in the	0–1
		region
		Subscale	0–2	0.9080	0.2581	0.0000
IV		Allocation of care
	11	Joint care allocation	0–1
	12	Rapid response to outpatient	0–1
		referral
		Subscale	0–2	0.8646	0.8387	0.0000
V		Treatment
	13	Transmural Treatment plan	0–1
	14	Transmural casemanager	0–1
	15	Transmural medical record	0–1
		Subscale	0–3	0.7373	0.5699	0.0857
VI		Compliance
	16	Medication	0–1
	17	Outreach if not on	0–1
		appointment
	18	Continuity youth–adult	0–1
	19	Continuity adult–elderly	0–1
	20	Continuity when admitted	0–1
	21	Continuity when obligatory	0–1
		discharge
		Subscale	0–6	0.8417	0.6452	0.0291
I–VI		Total scale	0–21	0.7657	0.5791	0.0562

The first subscale included the availability of interventions as follows: cognitive therapy, family psycho-education, assertive community treatment, neuroleptic treatment, and case management. Rehabilitation was not included (as explained in the introduction). Within the subscales, 2–6 items were included based on the problems that had been reported by schizophrenia experts [4]. In subscale 2 we asked for collaboration with institutes for drug-abuse, ethnic minorities, and the homeless in order to diagnose and treat people with schizophrenia and related psychosis. In subscale 3 we catalogued the use of uniform diagnostic procedures within the MHO and within the region. In sub-scale 4 we catalogued whether or not institutions in the region collaborated on the allocation of care and whether schizophrenic patients could get rapid access to mental healthcare. In subscale 5 we asked for standardised ‘transmural’ (i.e. a continuum of ambulatory and residential) treatment-modalities such as the availability of a treatment plan. In subscale 6 we catalogued the compliance, especially as a trait of mental healthcare in solving problems in the continuity of care, such as barriers between age-categories and between ambulant and residential care. In total, 21 items were included in the questionnaire.

The outcome measures are the sub scores as well as sum score of the six different sub scores of the completed questionnaires. The questionnaires were taken from directors of mental healthcare institutions and psychiatrists responsible for programmes for patients with schizophrenia. This was done in 32 of all 36 healthcare regions in the Netherlands. These 36 healthcare regions cover the whole country. The four pilot-cases were not included in the study.

The questionnaire was filled in by one of the three researchers (DR, AS and VB), all experts in the field, during the telephone interviews. If questions or answers were not clear, more questions were asked and related information was gathered. The respondent authorised the completed questionnaires. In one case no authorisation was obtained within the available time so the data were not used. Answers were put into a statistical database. The 32 MHOs were asked to forward documents about their programmes, yearly reports and production reports and other relevant documents. Yearly reports of the MHOs were studied in the library of ‘GGZ Nederland’ (The Dutch Association of Mental Healthcare), the branch-organisation of mental health institutions.

Answers were compared with the documentation obtained from the MHOs. A positive answer was scored when evidence was found in the documentation of the MHO for that answer. If not, the answer was scored as negative. This was the case in eight answers out of 31×21 answers. Sum scores per MHO were determined per category and in total. The appendix shows the 21 items and the 6 subscales. These six categories provide us with a differentiated picture of the aspects of process quality as defined by Tugwell. The sum score of the 21 items gives an indication of process quality in total.

The determinants we defined were: 1. ‘Age’: number of years since merger (not merged=–1; merger in the year of study=0) 2. ‘Size’: the number of inhabitants of the catchment area as stated in the yearly reports.

Results

Initially, all 36 MHOs, covering all regions of the country, responded. The four pilot-cases were not included in the results making the number of cases that were included 32 (89%). A questionnaire was completed in 32 out of 32 cases and authorisation was obtained in 31 cases (97%). In 94% the respondent was a psychiatrist, in 75% (24 of the 32) with a function in operational patient-care, and in 25% (8 of the 32) a member of the board of directors. The average experience with schizophrenia was 18.3 years ranging from 5–26 years.

In Table 2 the characteristics of the respondents and their MHOs are shown. On 01-01-2000, 26 of the 31 MHOs were merged. Between 0 and 7 years had passed since merger with an average of 3 years.

Table 2

characteristics of respondents and their MHOs

	No.	%
Respondent is psychiatrist	30	94
Respondent has executive role	24	75
Respondent is board member	8	25
Years of experience	5–26 (mean: 18.3)	–
Merged	26	84
Not merged	5	16
Years since merger	0–7 (mean: 3)	–
Size catchment area (×1000)	200–1000 (mean: 457)	–
Small catchment area (<400,000)	15	48
Large catchment area (>400,000)	16	52

Table 3 shows the number of MHOs (out of 31) that score positive (one point) on the different items. ACT and case management were available for schizophrenic patients in more than half of the MHOs, but cognitive behavioural therapy and psycho-education in less than half. Collaboration with other organisations for reaching people with a combined diagnosis of psychosis and addiction was better established than for people from ethnic minorities with psychosis. Diagnostic procedures were standardised in less than a quarter of the MHOs. Waiting time for a first visit to the outpatient department was not longer than four weeks in a single case. Continuity of care was a problem, especially for young people with psychosis who pass the 18-year border and for patients who need admission to a clinical treatment setting.

Table 3

number of MHOs (n=31) with indicators for process quality of care for persons with schizophrenia and related psychosis according to the classification of Tugwell (1984)

Subscale	Indicator	Item	No of 31	%
			institutions
			positive
I		Availability of interventions
	1	Cognitive Behavioural Therapy	10	32
	2	Pharmacotherapy guidelines	20	65
	3	Assertive Community Treatment	23	74
	4	Case management	25	80
	5	Family psycho-education	11	35
II		Reach of care
	6	Addicted persons	17	55
	7	Ethnic minorities	4	13
	8	Homeless persons	16	52
III		Diagnosis
	9	Uniform procedure in MHO	8	26
	10	Uniform procedure in the region	7	23
IV		Allocation of care
	11	Joint care allocation	27	87
	12	Rapid response to outpatient referral	27	87
V		Treatment
	13	Transmural treatment plan	28	90
	14	Transmural case manager	15	48
	15	Transmural medical record	10	32
VI		Compliance
	16	Medication	26	84
	17	Outreach if not on appointment	27	87
	18	Continuity youth–adult	15	48
	19	Continuity adult–elderly	19	61
	20	Continuity when admitted	14	45
	21	Continuity when obligatory discharge	16	52

The maximum score of the MHOs on the 21-item list was 19, the minimum score 5. Eight MHOs scored between 5 and 9 points, 14 MHOs between 10 and 14, 9 MHOs between 15 and 19 points. The median score was 12 (SD 3.4). Seventeen (55%) MHOs had a score equal to or higher than 12.

Figure 1 shows a scatter graphic with all cases on a diagram of years since merger (X-axis) and the total score of process quality (Y-axis). Non-merged organisations are valued with –1 years since merger. Four MHOs with the same coordinates as others are not separately represented. A Pearson correlation was computed which demonstrated that there was no significant correlation between ‘years since merger’ and ‘total score of process quality’ (R=0.383, p=0.068 two-tailed). The percentage of linear explained variance was computed which demonstrated that 14.7% of the total variance was explained by the linear relation between ‘years since merger’ and ‘total score of process quality’. Significantly correlating with ‘years since merger’ are the sub scores on ‘availability of interventions’ (r=0.391, p=0.029, two-tailed) and ‘treatment’ (r=0.380, p=0.035, two-tailed) (see Appendix).

Figure 1

Association of years since merger and sum score of process quality.

Catchment area size, computed in the same linear way, had no significant correlation with process quality or any of the sub scores (see Appendix). We also computed a t-test, comparing the group of institutions with a ‘larger’ (L) catchment area of 400,000 (n=16) with those ‘smaller’ (S) than 400,000 inhabitants (n=15). With this test we also found no significant difference in process quality sum score between those two groups (p=0.98, two-tailed).

Finally, we conducted a contrast-group-analysis with the four highest scoring (HS; scores 18 or 19) MHOs and the four lowest scoring (LS; scores 5–8) MHOs. The median age since merger in the HS group is 3 years (SD=0.5), in the LS group 1 year (SD=2.3).

The median size of the catchment area in the HS group was 450,000 (SD=310,913) and 380,000 (SD=124,766) in the LS group.

Discussion

We found no significant relationship between the age of the merged organisation (‘age’) and the total process quality of schizophrenia care as measured with the constructed rating scale. However, two subscales, ‘availability of interventions’ and ‘treatment’ gave significant association with ‘age’. Perhaps the average number of years since the mergers, three, is too short to show the positive relationship that we might expect. It is well known that in the first two or three years since a merger, organisations spend their time on rearranging teams and procedures. A study like this should therefore be repeated after six or seven years.

No association was found between the size of the MHO's catchment area and any of the used quality items for schizophrenia patients. Larger areas do not seem to be at an advantage over small areas in the aspects of schizophrenia care that were studied. An explanation for this finding might be that the development of establishing schizophrenia care programmes is still in an early phase, so existing programmes profit in this phase more from available pioneers than from large flows of patients.

Our more detailed findings, a low degree of implementation of evidence-based interventions, a limited collaboration on a regional level, the poorly standardised diagnostic procedures, and the problems with continuity of care match with other research findings [15] as well as with criticism given by schizophrenia experts, patients and their families [3]; in this critical report we make a number of recommendations that we can underline after this study. Our findings match with recent research on the development of care programmes in general in the Netherlands: the authors of this piece of research state that the developed programmes lack an evidence-base and that they are not client-oriented [23]. They plead for a national facilitating ‘program for the development of programs’, with which there is already some experience on a small scale. Our findings are also in accordance with reports that professionals don't always follow evidence-based guidelines [15,28]. In the documents we studied, for example, we found no specific implementation plans (‘what must be established at what time by whom?’) but merely descriptions or the designs of the desired care. Therefore, a sound action plan is needed [29]. Experts recommend the use of toolkits to promote the consistent delivery of such practices; these toolkits include written material, web-based resources, training experiences and consultation opportunities [28].

Since research on the effect of mergers in mental health is not available, we have to compare our findings with research done in general hospitals. Our finding that mergers may have a positive effect on care processes is coherent with the finding that (general) hospitals profit from increased service capability after merging [30]. Posnett [31] mentions different pressures to increase the concentration of hospital services, assuming reduction of costs and increasing average volumes of activity by clinicians but despite the lacking evidence of improved patient outcomes. We recommend a further study to support the assumption that mergers, i.e. integration and concentration of mental healthcare institutions lead to improved patient outcomes and cost reduction.

Our results are not fully in accordance with another finding in general hospitals that mergers result in slowing rates of pre-existing trends [32]. We find most of the evidence for the described positive effect comes from institutions that are still in their first two or three years after merger. Therefore, we cannot exclude the possibility of an ‘injection’ effect of merger on the described aspects of process quality, which of course could be an effect of initial enthusiasm. Since there is such a small number of mergers with a duration longer than 5 years in the Netherlands, this effect cannot be judged thoroughly at the moment. Therefore, the long-term effect of mergers on quality of care needs serious attention in the future.

The present study has five limitations that we would like to mention explicitly. Firstly, we omitted all of the long-stay care. This means that a generalisation of the results is limited. The theory we used creates a second limitation: process quality is assumed to have an effect on outcome quality. However, in practice, improvement of process quality is not a sufficient condition or guarantee for a satisfying outcome quality. A third possible limitation is the subjectivity in the answers of the respondents, even though they are both experienced and critical. This is a general limitation of questionnaire-based research in which wishful thinking may occur. We tried to alleviate this bias by using experienced interviewers and by data triangulation of questionnaires and documents. The interviewed informants were not aware of the study hypotheses but they were aware of some scepticism of the interviewers about mergers in mental health-care; this as a result of a former article [17] they published, which was known to these informants. The fourth limitation exists because of the fact that all the information we gathered through questionnaires, interviews and documents came from the side of the care provider and not from the patients or their families. Despite this subjectivity we found very few ‘social likely’ answers. Further research should incorporate different opinions.

The choice and the values of the different items in our rating scale form the fifth limitation. The choice of the questionnaire scales, following Tugwell and the different items in the rating scale are consistent with the Scottish Clinical Standard for Schizophrenia [33], which was not yet available at the onset of this study. The item ‘case management’ would have been left out if the knowledge of the Cochrane study on this subject was available at the beginning of the study. We deliberately addressed no ‘weight’ to the different items and subscales, which would have brought unavoidable subjectivity in the results. On the other hand, this choice brings with it a high degree of arbitrary weight pertaining to the items or subscales.

Of course other variables than those considered in this study might be associated with the extent of integration such as effects on cost reduction, human resources and training programmes.

We recommend the development of schizophrenia standards on a national level, in which evidence-based interventions are included as well as guidelines for the reach of the care, access, speed, assessment, continuity of care and compliance. Following this, further steps can be taken such as: regional implementation plans based on that standard, the use of already developed tool-kits and research aimed at the effects of the implementation. In measuring effects of implementation, we suggest that besides data of outcome quality like quality of life there should be a focus on the opinions of patients and their families about the provided care.

Table A1

	Years since merger (age)		Size catchment area (size)
Subscale	Pearson corr.	Sig (2-tailed)	Pearson corr.	Sig (2-tailed)
I	0.391*	0.029*	0.112	0.549
II	0.270	0.142	0.292	0.111
III	0.048	0.797	0.008	0.966
IV	0.081	0.666	-0.052	0.782
V	0.380*	0.035*	0.103	0.583
VI	0.059	0.751	0.108	0.564
Total scale	0.331	0.068	0.167	0.368

Significant at the 0.05 level.