Community-care unit model of residential mental health rehabilitation services in Queensland, Australia: predicting outcomes of consumers 1-year post discharge.

AIMS: Community care units (CCUs) are a model of residential psychiatric rehabilitation aiming to improve the independence and community functioning of people with severe and persistent mental illness. This study examined factors predicting improvement in outcomes among CCU consumers.
METHODS: Hierarchical regression using data from a retrospective cohort (N = 501) of all consumers admitted to five CCUs in Queensland, Australia between 2005 and 2014. The primary outcome was changed in mental health and social functioning (Health of the Nation Outcome Scale). Secondary outcomes were disability (Life Skills Profile-16), service use, accommodation instability, and involuntary treatment. Potential predictors covered service, consumer, and treatment characteristics. Group-level and individualised change were assessed between the year pre-admission and post-discharge. Where relevant and available, the reliable and clinically significant (RCS) change was assessed by comparison with a normative sample.
RESULTS: Group-level analyses showed statistically significant improvements in mental health and social functioning, and reductions in psychiatry-related bed-days, emergency department (ED) presentations and involuntary treatment. There were no significant changes in disability or accommodation instability. A total of 54.7% of consumers demonstrated reliable improvement in mental health and social functioning, and 43.0% showed RCS improvement. The majority (60.6%) showed a reliable improvement in psychiatry-related bed-use; a minority demonstrated reliable improvement in ED presentations (12.5%). Significant predictors of improvement included variables related to the CCU care (e.g. episode duration), consumer characteristics (e.g. primary diagnosis) and treatment variables (e.g. psychiatry-related bed-days pre-admission). Higher baseline impairment in mental health and social functioning (β = 1.12) and longer episodes of CCU care (β = 1.03) increased the likelihood of RCS improvement in mental health and social functioning.
CONCLUSIONS: CCU care was followed by reliable improvements in relevant outcomes for many consumers. Consumers with poorer mental health and social functioning, and a longer episode of CCU care were more likely to make RCS improvements in mental health and social functioning.

Introduction

The community care unit (CCU) model of psychiatric rehabilitation was established in Australia based on the successes in transitioning former long-stay psychiatric inpatients back into the community demonstrated in the United Kingdom (Trauer et al., 2001). Contemporary CCUs provide time-limited, clinically-oriented residential psychiatric support focused on improving multiple aspects of personal functioning – primarily living skills development and community integration – in the context of overall mental health (Parker et al., 2019). These clustered, independent living units provide 24-h rehabilitation support to people whose functioning is affected by serious mental illness (Meehan et al., 2017). Most CCU consumers will have experienced recurrent and lengthy hospitalisations (Meehan et al., 2017). The costs associated with the provision of intensive rehabilitation care in a residential setting support the need for accountability regarding service effectiveness (Iyer et al., 2005; Murugesan et al., 2007).

There is limited evidence available considering the outcomes of community-based residential rehabilitation (Parker et al., 2019). Psychiatric rehabilitation aims to improve psychosocial functioning and facilitate personal recovery (Farkas and Anthony, 2010); hence, mental health symptoms and social outcomes may reflect appropriate outcome foci (Tulloch et al., 2008). However, societal outcomes such as hospital use (Bunyan et al., 2016) and accommodation stability (Killaspy and Zis, 2013) are slower to change and better reflect the broader costs associated with schizophrenia (Yoon et al., 2013).

Evaluations of residential psychiatric rehabilitation services have considered a range of outcomes but tend to individually focus on only one or two relevant domains (online Supplementary Table 1). These studies have found improvements in mental health and social functioning (Gonda et al., 2012; Maxwell et al., 2018), but have not addressed measures of independence which have been found to improve following engagement with other rehabilitation services (Killaspy and Zis, 2013; Yoon et al., 2013). Furthermore, some studies have relied on single time-point outcome measures such as time to re-hospitalisation or inpatient days post-discharge (Grinshpoon et al., 2007).

There has been limited research examining predictors of rehabilitation outcomes (Vita and Barlati, 2019). A range of predictors have been identified (see Table 1), but studies generally fail to consider a broad range of relevant predictors; notably cognitive functioning, a strong predictor of positive outcomes in vocational rehabilitation (Evans et al., 2004) and of functional outcomes generally for people diagnosed with schizophrenia (Brune et al., 2011). Moreover, data is often collected from multiple service locations without examining the effect of site and other geographic factors. It is plausible that outcomes may be impacted by site-specific practices or the availability of other services in the region.

Table 1.

Summary of studies evaluating the outcomes of psychiatric rehabilitation in people with schizophrenia

Study	Sample	Context and country	Design	Outcome measures	Independent predictor variables	Main findings
Chatterjee et al. (2009)	236 patients (141 males)Age: 20–45 years (74.2%)Primary diagnoses: Schizophrenia (55.5%), bipolar (27.7%) and other psychoses (16.8%)	Community-based rehabilitation in rural India; minimum enrolment in the program was 12 months with a median period of 46 months	Longitudinal with assessments at admission and discharge over a 3-year follow-up period	Disability assessed with Indian Disability Evaluation Assessment Scale (IDEAS)	Marital status, primary diagnosis, duration of illness, medication adherence, household assets, family support, self-help group membership, program drop-out	Improvement was marked (⩾40% change from baseline) in 50%, moderate (20–40% change from baseline) in 40%, and minimal (<20% change from baseline) in 10% of participantsPositive predictors of reduced levels of disability were lower baseline disability, family engagement with the program, medication adherence and engagement with a self-help group; negative predictors were lack of formal education, diagnosis of schizophrenia and dropping out of the program
De Girolamo et al. (2014)	403 patients (2/3 male)Mean age: 48 ± 10, range 19–64 yearsPrimary diagnoses: Schizophrenia-spectrum (67.5%) and personality disorders (17.9%)	23 medium-long-term residential facilities in Italy; mean length of stay: 4.2 years ±5.5 (median = 2.2)	Longitudinal with 1-year follow-up	Likelihood of home discharge	Socio-demographic (e.g. primary diagnosis, illness duration, age), psychosocial variables (e.g. social support, inactivity)	Positive predictors of home discharge were shorter illness duration, available social support in the last year, and a diagnosis of unipolar depression
Gonda et al. (2012)	337 patients (170 male)Mean age: 33.6 ± 9.8, range 18–61 yearsPrimary diagnoses: Schizophrenia (68%) and schizoaffective (20.8%) disorders	2 inpatient psychosocial rehabilitation units in NSW, Australia; mean length of stay: 111 ± 73, range 6–602 days	Longitudinal with assessments at admission, discharge and 3-month follow-up	RCS improvement on BPRS-E, HoNOS and K10 (RCS improved/not improved)	Age, gender, length of stay, primary diagnosis and co-morbid diagnosis	Between 32 and 49% of the patients made an improvement; between 20 and 32% made RCS improvement across the three outcome measuresPositive predictors of RCS improvement on psychiatric symptomatology (BPRS-E) were schizoaffective disorder [exp(β) = 3.52, p < 0.05] and co-morbid alcohol abuse disorder [exp(β) = 2.29, p = 0.053]
Grinshpoon et al. (2007)	4160 patients (2413 male) discharged from their first-in-life psychiatric hospitalisationPrimary diagnoses: F20–F29 or F30–F39 (100%)	Two cohorts of patients (discharged 1990–1991 and 2000–2001), to assess the effects of the Rehabilitation of the Mentally Disabled Act (RMDA) passed in 2000, Israel	Longitudinal with 3-year follow-up	Time to hospital re-admission	Age, gender	Re-admission for all patients was lower among females [exp(β) = 0.13, p < 0.001] and negatively associated with age [exp(β) for 20–44 y old = 0.13, p = 0.001 and for 45–64 y old = 0.414, p < 0.001]For the 2000–2001 cohort, long hospitalisation (more than 6 months) was associated with reduced likelihood of re-admission during follow-up [exp(β) = 0.26, p = 0.056]
Killaspy and Zis (2013)	141 patients (84 male)Mean age: 44 ± 13 yearsPrimary diagnoses: Schizophrenia or schizoaffective disorder (93%)	Mental health residential rehabilitation service in London, UK, incl. 2 shorter-term inpatient units (n = 47), 3 community units (n = 44), 4 community-supported accommodation facilities (n = 50)	Retrospective 5-year survey-based study	Change in independence: positive outcome – achieving and sustaining community placement for inpatients and progressing or sustaining a less supported community placement for community patients (progressed/remained stable/relapsed)	Age, history of physical abuse, medication non-adherence, challenging behaviours, social function communication, involuntary treatment	Positive outcome was achieved by 50 (40%) of the patients; 13 (10%) moved to independent accommodation and successfully sustained their tenancy; 33 (27%) remained in a placement with a similar level of support; 41 (38%) moved to more supported placement and/or had a psychiatric admissionIncreased age was associated with a reduced likelihood (OR = 0.93, p = 0.003) and medication non-adherence with increased odds of a negative outcome (OR = 33.57, p < 0.001)
Lim et al. (2017)	246 patients (161 male)Mean age: 37.9 ± 9.4 yearsPrimary diagnosis: Schizophrenia (100%)	6 community-based psychiatric rehabilitation programs in Los Angeles, USA	Longitudinal with assessments at admission, and at 6 and 12 months after admission	Recovery based on 4 criteria (be in symptomatic remission, demonstrate adequate work and social functioning, and no psychiatric hospitalisation)	Demographic (gender, education),clinical (e.g. symptomatology, length of illness, medication use) andpsychosocial characteristics (e.g. intrinsic motivation, social support)	Recovery was recorded in 19.8 and 7.5% of patients at 6- and 12-month follow-up, respectively (n = 146)Higher levels of intrinsic motivation [exp(β) = 1.68], positive family relationships [exp(β) = 1.32], role functioning [exp(β) = 1.34] and social functioning [exp(β) = 1.40] at admission predicted recovery at 6-month follow-up (all p < 0.05)
Maxwell et al. (2018)	Clinical group: 210 patients (144 male)Comparison group (mental health sample functioning independently in the community): 114 adults (57 male)Primary diagnosis: Schizophrenia (100%)	1 inpatient mental health rehabilitation unit in NSW, Australia	Longitudinal with assessments at admission, discharge and at least 1 year post-discharge	RCS (based on cut-off 3) on HoNOS; LSP-16; and K10	Age, gender, marital status, type of usual accommodation, country of birth, secondary diagnosis, length of stay, HoNOS total and subscale scores and LSP total and subscale scores at admission	Positive predictors of RCS improvement on HoNOS total scale were HoNOS Behaviour [exp(β) = 14.57 p < 0.01] and Impairment subscales scores [exp(β) = 18.87, p < 0.05] at admissionPositive predictors of RCS improvement on LSP total scale were LSP Socialisation [exp(β) = 10.23, p < 0.05] and Withdrawal subscales scores [exp(β) = 10.23, p < 0.05] at admission
Yoon et al. (2013)	9208 adultsMean age: 41.1 ± 15.6 yearsDiagnoses: Schizophrenia (63%), bipolar (48%), other mental illness (52%), substance abuse (54%)	Intensive case-management community-treatment program, California, USA; mean tenure: 10.8 ± 8.2 months	Longitudinal followed up to 4 years	Residential transition to different types of living arrangements	Length and continuity of program participation, age, gender, diagnosis, education, race	Positive predictors of independent living arrangement were uninterrupted program participation, having a diagnosis of bipolar disorder (relative to schizophrenia), and any other diagnosis, such as depression or personality or anxiety disorder (relative to schizophrenia or bipolar disorder)

BPRS-E, Brief Psychiatric Rating Scale-Expanded version; HoNOS, Health of the Nation Outcome Scales; K10, Kessler 10; LSP, Life Skills Profile-16; NSW, New South Wales; OR, adjusted odds ratio; RCS, reliable and clinically significant; SLOF, specific levels of functioning; UK, United Kingdom; USA, United States of America.

The importance of assessing the reliable and clinically significant (RCS) change rather than statistical significance in outcome evaluation has been emphasised (Maxwell et al., 2018). Instead of assessing change at the group level (e.g. considering average scores), this approach considers the magnitude of changes for individual participants (e.g. their difference between pre-admission and post-discharge scores) and allows for comparison with normative data. Such assessments may provide a more appropriate measure of rehabilitation outcomes (Trauer, 2010). Several studies have reported RCS improvement following psychiatric rehabilitation (Barbato et al., 2007; Murugesan et al., 2007; Chatterjee et al., 2009; Gonda et al., 2012; Killaspy and Zis, 2013; Maxwell et al., 2018). However, the availability of appropriate functional population data remains a challenge. For example, Murugesan et al. (2007) reported RCS psychosocial functioning improvement following in-patient psychiatric rehabilitation in Australia using an Italian community sample for the functional population data.

The present study measured the extent of improvements in outcomes among consumers accessing CCUs in Queensland between 2005 and 2014, and explored predictors of these outcomes. The approach taken expands on previous studies by including a broad range of relevant outcomes and potential predictors. Based on previous research (e.g. Gonda et al., 2012; Killaspy and Zis, 2013; Maxwell et al., 2018), we hypothesised that CCU care would be followed by RCS improvement in mental health and social functioning between the year pre-admission and post-discharge. Improvements were also expected in service use, accommodation instability, disability and reduced involuntary treatment. Additionally, it was expected that outcomes would be predicted by a range of service-, consumer- and treatment-level factors.

Method

Design

The cohort was derived retrospectively from linked administrative data for consumers' first admission to any of the five CCU sites in Queensland, Australia, between 2005 and 2014. Consumers were excluded if they did not meet the discharge criteria: a period of separation >28 days by 31 December 2014. Pre-admission measures covered the 365 days pre-admission; post-discharge measures covered the 365 days post-discharge. The Metro South Addiction and Mental Health Services Human Research Ethics Committee [HREC/15/QPAH/392] provided ethical clearance.

Measures

Dependent variables

The primary outcome was a change in mental health and social functioning, measured by the Health of the Nation Outcome Scale (HoNOS; Wing et al., 1998). The HoNOS is a 12-item clinician-rated questionnaire routinely used in outcome evaluation in Australia and internationally (Burgess et al., 2015) with demonstrated validity, reliability (Burgess et al., 2015; Burgess et al., 2017) and sensitivity to change (Slade et al., 1999). Secondary outcomes were changes in: psychiatric service use (number of combined acute and non-acute, psychiatry-related bed-days, emergency department (ED) presentations); accommodation instability (number of changes in primary residence); involuntary treatment and disability (assessed using the Life Skills Profile (LSP-16) (Trauer et al., 1995)). The use of the HoNOS and LSP-16 in Australia is supported by an established national assessment protocol and training processes to facilitate inter-rater reliability (Burgess et al., 2015). These measures are administered by clinicians at admission, 3-monthly review and discharge, with raters likely to differ across collection instances.

For HoNOS and LSP-16 the highest total scores (i.e. the poorest functioning/disability) recorded in the pre-admission and post-discharge period were used. This approach aimed to capture a picture of consumers' functioning in the community, rather than measurements at admission and discharge that may be impacted by the anticipated support at admission and relative stability at discharge. The change in psychiatric service use and accommodation instability was calculated by comparing the frequency between the 365 days pre-admission and 365 days post-discharge. The change in involuntary treatment status was determined by comparing status at admission and discharge from a CCU. In Queensland, mental health legislation permits involuntary treatment in both community and inpatient settings for consumers with a mental disorder requiring immediate treatment, with an associated imminent risk to self or others.

Independent variables

Based on the previous literature and the limitation of the data-set 18 potential predictors were organised across five levels (see Table 2): Level 1 – time (reflective of changes in service models over time); Level 2 – service-model (site/episode of care characteristics); Level 3 – consumer characteristics (more stable consumer features); Level 4 – consumer impairment (arising from Level 3) and Level 5 – treatment (to address impairments).

Table 2.

A summary of statistically significant predictors (p < 0.05) across outcome variables based on the logistic regression analyses

Level		Predictor variables	Outcome variablesa
			HoNOS total score	Hospital bed use	ED presentations	Treatment status
1	Time	Year of admission
2	Service-model	CCU site			0.29
		Locationb
		Length of stay	1.03	1.02		1.05
3	Consumer characteristics	Age		1.02		0.93
		Sex
		Primary diagnosis (reference: F20–F29)		0.50
		Personality disorder as a secondary diagnosis
		Aggressive behaviour (HoNOS item 1)				2.82
		Substance use problems (HoNOS item 3)			2.31
		Physical impairment (HoNOS item 5)
4	Consumer impairment	Mental health and social functioning (HoNOS total score)	1.12		1.06
		Disability (LSP-16 total score)	0.94
		Cognitive functioning (HoNOS item 4)
5	Treatment	Total bed-based service use	0.99	1.01		0.99
		ED presentations
		POS with family present
		Treatment status at entry

CCU, community-care unit; HoNOS, Health of the Nation Outcome Scales; LSP-16, Life Skills Profile-16; ED, Emergency Department; POS, Provisions of Service.

This table summarises the results of several logistic regression analyses that were conducted to facilitate consideration of the role of potential predictors across a range of outcome variables of relevance to rehabilitation care. The table presents standardised regression coefficients for statistically significant predictors of each outcome. See Table 5 for additional information.

To compare CCUs across different locations, the postcode of each CCU location was classified into one of the ten decile rankings within Queensland, based on the Socio-Economic Indexes for Areas (SEIFA) 2016 (ABS, 2018). The locations of CCU sites corresponded to three separate rankings; one CCU was located within the 2nd ranking, two CCUs within the 8th and two CCUs within the 10th ranking.

Table 5.

Binary logistic regression predicting RCS improvement on HoNOS total score (n = 179), reliable improvement on service use (n = 495) and change towards a less restrictive treatment status between the year pre-admission and the year post-discharge (n = 501)

	Variable	B	s.e. of B	Exp(β)	95% CI
HoNOS total scorea	Length of stay (days)	0.00**	0.01	1.03	1.01–1.05
	HoNOS total score pre- admission	0.11**	0.03	1.12	1.05–1.91
	LSP-16 total score pre-admission	−0.06*	0.03	0.94	0.90–0.99
	Total bed days pre-admission	−0.00*	0.00	0.99	0.99–1.00
	Constant	−1.79	0.66	0.17
Hospital use (total bed days)b	Length of stay (days)	0.00**	0.00	1.02	1.01–1.02
	Age (years)	0.02**	0.01	1.02	1.01–1.04
	Primary diagnosisc	−0.69*	0.27	0.50	0.30–0.85
	Total bed days pre-admission	0.01**	0.01	1.01	1.01–1.01
	Constant	−0.74	0.41	0.48
ED presentationsd	Sitee	1.24**	0.48	0.29	0.11–0.73
	Substance use pre-admissionf	0.84*	0.38	2.31	1.09–4.92
	HoNOS total score pre-admission	0.05*	0.02	1.06	1.01–1.11
	Constant	−3.03	0.52	0.05
Treatment statusg	Length of stay (days)	0.00**	0.01	1.05	1.04–1.07
	Age (years)	−0.08**	0.02	0.93	0.89–0.97
	Aggressive behaviour pre-admissionh	1.04*	0.44	2.82	1.18–6.76
	Total bed days pre-admission	−0.01*	0.00	0.99	0.99–1.00
	Constant	−0.94	0.80

HoNOS, Health of the Nation Outcome Scales; B, unstandardised regression coefficients; β, standardised regression coefficients; s.e., standard error; CI, confidence interval.

*p < 0.05, **p < 0.01.

The dependent variable is 0 = no RCS improvement and 1 =  RCS improvement. The full model was significant (λ2(4) = 20.00, p < 0.001); the model accounted for 17.7% (Nagelkerke R2) of the total variance, correctly classifying 64.8% of consumers (44.8% as making RCS improvement and 78.6% as not improving).

The dependent variable is 0 = no reliable improvement and 1 = reliable improvement. The full model was significant (λ2(4) = 20.00, p < 0.001); the model accounted for 19.6% (Nagelkerke R2) of the total variance, correctly classifying 65.8% of consumers (75.8% as making reliable improvement and 50.5% as not).

The reference category is F20.x-F29.x.

The dependent variable is 0 = no reliable improvement and 1 = reliable improvement. The full model was significant (λ2(3) = 20.48, p < 0.001), accounting for 12.1% (Nagelkerke R2) of the total variance; the model correctly classified 88.2% of consumers (2.6% as making reliable improvement and 100.0% as not).

The reference category is a CCU site with 165 consumers (32.9%).

HoNOS item 3 rating of moderate of higher pre-admission.

The dependent variable is 0 = same or more restrictive status and 1 = less restrictive status. The full model was significant (λ2(4) = 53.14, p < 0.001), accounting for 31.8% (Nagelkerke R2) of the total variance. The model correctly classified 87.9% of consumers; 97.6% in the same or more restrictive group and 19.4% in the less restrictive group.

HoNOS item 1 mild or greater pre-admission.

Statistical analysis

Spss v25.0 (IBM, 2017) was used for all analyses.

Extent of improvement in outcomes

At a group level, differences between pre-admission and post-discharge scores were assessed using the Wilcoxon Signed-rank test for continuous variables, and the χ2-square test for nominal variables. At the individual level, two indicators of change in outcomes were calculated, namely statistically reliable change and RCS change. To obtain the proportion of individuals with reliable change (i.e. improvement or deterioration) and those without reliable change (i.e. stable), the reliable change index (RCI), was calculated using the Christensen and Mendoza (1986) formula:s.e.diff is a measurement of a difference and was calculated as:where s.d.1 is the standard deviation of the total score pre-admission and α is the Cronbach's coefficient of internal reliability of the outcome measure.

To assess the RCS change, a clinically significant change was first calculated, using three cut-off methods (Jacobson and Truax, 1991):

Cut-off 1: >2 s.d.s from the dysfunctional population mean;

Cut-off 2: <2 s.d.s of the functional population mean and

Cut-off 3: Closer to the functional population than dysfunctional population mean.

Cut-off 3 was calculated using the following formula:The RCS change was assumed where the change between individual's pre-admission and post-discharge scores exceeded the RCI score and an individual's post-discharge score met any of the three cut-off criteria. These cut-offs were used to assess clinically significant change in HoNOS total scores. To calculate Cut-off 2 and 3, functional population data were derived from a study of 114 individuals residing in the community, accessing mental health services in New South Wales, Australia (Maxwell et al., 2018). Inclusion criteria for the functional population being: ⩾1 mental health-related inpatient or ED admission within the past 5 years, no psychiatric admissions within the past 6-months; low scores (⩽1) on HoNOS items 1 (overactive, aggressive, disruptive or agitated behaviour) and 6 (problems with hallucinations and delusions); primary diagnosis of a psychotic disorder and no documented admission to an inpatient rehabilitation unit.

For secondary outcome variables, Cut-off 2 and 3 could not be determined as functional population data were not readily available. Additionally, distributions of scores for secondary outcomes were too skewed, restricting the calculation of meaningful Cut-off 1 scores. Therefore, only the criterion of the statistically reliable change was considered.

Predictors of outcomes

Binomial logistic regression analyses were conducted to investigate whether the proposed independent variables predicted significant changes in outcomes. Standard diagnostics were applied to test relevant assumptions and multicollinearity (online Supplementary Table 2). Sample homogeneity was explored based on admission year (2005–2009/2010–2015), CCU site, location, length of stay and primary diagnosis (F20.x-F29.x (schizophrenia and related psychotic disorders)/other disorders). Although several significant differences emerged, these were not considered of sufficient clinical relevance to warrant conducting analyses by sub-samples. The regression analysis was applied to outcomes where statistically significant differences (p < 0.05) were present at a group level.

The primary outcome was dichotomised into ‘RCS improvement’ and ‘no RCS improvement’, using the cut-off identifying the greatest proportion of improved participants (Gonda et al., 2012). Secondary outcomes were dichotomised into ‘reliable improvement’ and ‘no reliable improvement’. A hierarchical approach was followed due to expected inter-correlation between predictors (Scialfa and Games, 1987); variables were grouped into five levels and entered sequentially in blocks from Level 1 to Level 5 (see Table 2). Predictors at each level were retained in the model if they demonstrated a statistical significance in predicting a specific outcome (p < 0.05).

Results

Participant characteristics

The sample included 501 participants (349 male) with a median CCU episode of care duration of 154 days (range 0–2225, one consumer being admitted and discharged on the same day). Demographic and diagnostic information is provided in Table 3.

Table 3.

Demographic and clinical characteristics of consumers prior to admission to a CCU (N = 501)

	N	%
Demographic
Mean age and range (years)	35.7	17–81
Gender: Male	349	69.7
Country of birth: Australia	432	86.2
Education level
Year 10 or less	201	40.1
Year 12	99	19.8
Tertiary education	40	8.0
Not stated/unknown	161	32.1
Relationship status
Never married	374	74.7
Married	47	9.4
Divorced/separated	68	13.6
Widowed	8	1.6
Not stated/unknown	4	0.8
Income source
Disability pension	239	47.7
Other governmental benefits	228	45.5
No income	18	3.6
Employment-related payment	11	2.2
Not stated/unknown	5	1.0
Diagnosis and clinical symptoms
Primary ICD10 diagnosis
F10–F19 Mental and Behavioural disorders due to psychoactive substance use	13	2.6
F20.x Schizophrenia	336	67.6
F25.x Schizoaffective Disorder	54	10.9
F30–F39 Mood (affective) disorder	26	5.2
F40–F48 Neurotic, stress-related and somatoform disorders	11	2.2
F60.x Specified personality disorder	16	3.2
Other	38	7.6
Personality Disorder as a secondary ICD10 diagnosis	69	13.8
Mild or greater severity of clinical symptoms (score ⩾2)
Overactive/aggressive behaviour (HoNOS item 1)	173	54.7
Problem drinking or drug taking (HoNOS item 3)	112	35.4
Cognitive problems (HoNOS item 4)	150	47.5
Physical impairment (HoNOS item 5)	98	31.0
Psychotic symptoms (HoNOS item 6)	222	70.3

CCU, community-care unit; HoNOS, Health of the Nation Outcome Scales.

Data were missing for Primary diagnosis (4 cases; 0.8%) and for all HoNOS items (185 cases; 36.9%). One subject was accepted for care and physically present in a CCU for part of a day but then discharged the same day.

Summary of outcomes and modelling

Differences in outcome scores between pre-admission and post-discharge at a group level are presented in Table 4 (also online Supplementary Table 3). Assessments of the individual change were conducted on sub-samples of consumers with paired outcome data (179 for HoNOS, 237 for LSP-16, 495 for service use and 462 for accommodation instability). Statistically significant predictors of outcomes identified through logistic regression are summarised in Tables 2 and 5.

Table 4.

Change in outcome variables between the 365 days pre-admission and 365 days post-discharge and classification of consumers based on criteria for reliable and RCS change (N = 501)

Outcome	Group comparison						Individual treatment effects
	Pre-admission		Post-discharge					Reliable change based on RCI				RCS improvement
	M	s.d.	M	s.d.	Wilcoxon Z	p-Value	RCI	ImprovementN (%)	StableN (%)	DeteriorationN (%)	Cut-off	N (%)
HoNOS total scorea	18.44	8.17	13.14	8.82	−6.37	<0.001	6.14	98 (54.7)	53 (29.6%)	28 (15.6)	13	77 (43.0)
LSP-16 total score	17.74	8.53	17.21	8.08	−0.23	0.819	0.14	–	–	–	–
Hospital bed usea	101.54	113.01	70.39	118.33	−6.69	<0.001	5.34	300 (60.6)	70 (14.7%)	125 (25.3)	–
ED presentationsa	1.51	2.58	1.05	3.09	−5.75	<0.001	3.25	62 (12.5)	402 (81.2%)	31 (6.3)	–
Accommodation instabilitya,b	2.27	2.36	2.00	1.60	−1.52	0.128	1.98	90 (19.5)	308 (66.7%)	64 (13.9%)	–
Involuntary treatment order (ITO)	N	%	N	%	χ2-squarec	p-value
No ITO	185	36.90	241	48.10	738.60	<0.001	–				–
ITO	250	49.90	194	38.70
Forensic order/SNFP	66	13.20	66	13.20

p, statistical significance; RCI, reliable change index; RCS, reliable and clinically significant; HoNOS, Health of the Nation Outcome Scales; LSP-16, Life Skills Profile-16; ED, Emergency Department; SNFP, Special Notification Forensic Patient.

Data were missing for HoNOS total score (185; 36.9% pre-admission and 292; 58.3% post-discharge), LSP total score (248; 49.5% pre-admission and 132; 26.3% post-discharge) and accommodation instability (6; 1.2% pre-admission and 2; 0.4% post-discharge). The number of paired observations assessed by the Wilcoxon signed-rank test and used to calculate the specific RCIs was 179 for HoNOS, 237 for LSP-16, 495 for hospital use and ED presentations and 462 for accommodation instability measures. The frequency of missing data for HoNOS and LSP-15 limited the sample sizes of complete paired data (35.73% and 47.30% of the total sample, respectively). Consumers with and without complete paired data on HoNOS total score were compared using the chi-square and Mann–Whitney U test to ascertain if data were missing at random; no significant differences (p < 0.05) were identified for length of stay at a CCU, sex, age, primary and secondary diagnoses, mild/greater severity of clinical symptoms (HoNOS items 1, 3, 4 and 5), LSP total score, hospital use, accommodation instability and involuntary treatment at admission to CCU. The groups differed (all p < 0.05) on HoNOS total score (Mpaired = 19.4, Mothers = 17.2), ED presentations (Mpaired = 2.1, Mothers = 1.3) and family involvement (Mpaired = 5.0, Mothers = 1.9).

Consumers who died within 1 year since discharge (n = 6) were excluded from the analyses that included data referring to a 365-day time period post-discharge (i.e. hospital bed use, ED presentations and accommodation instability). However, these consumers were not excluded from the analysis of HoNOS data (three had complete paired data). Re-calculation of RCI without the six consumers produced the same cut-off score (6.82 rounded to 7) as the original analysis, meaning the proportion of improved consumers remained unchanged.

To minimise the bias against consumers transitioning from long-term inpatient care to a CCU, consumers with more than 300 non-acute inpatient bed days (n = 28) were excluded from the analysis of accommodation instability.

df = 4

Mental health and social functioning

Consumers demonstrated significantly reduced HoNOS scores between the pre-admission and post-discharge periods (total and subscales). For HoNOS the RCI was 6.14 (rounded to 6); based on this, 54.7% reliably improved and 15.6% reliably deteriorated. In determining RCS change: Cut-off 1 (a score of 2) found improvement in 8.9% of consumers and deterioration in 2.2%; Cut-off 2 (a score of 13) found improvement in 43.0% of consumers and deterioration in 14.0% and Cut-off 3 (score of 9) found improvement in 32.0% of consumers and deterioration in 15.4%. For the hierarchical regression analysis, Cut-off 2 was used to dichotomise the sub-sample with paired outcome data into those with and without RCS improvement (43.0 and 57.0% respectively). RCS improvement was predicted by longer episodes of CCU care, as well as higher HoNOS total scores, lower LSP-16 total score and fewer psychiatry-related bed-days pre-admission.

Service use

Significant reductions occurred in acute and total psychiatry-related bed-days, but not sub-acute bed-days, following CCU discharge (Table 4 and online Supplementary Table 4). The RCI for total bed-days was 5.34 (rounded to 5), indicating that 60.6% of consumers reliably improved and 25.3% reliably deteriorated. For the regression analysis, the cohort was dichotomised into those making and not making a reliable improvement (60.6 and 39.4% respectively). Predictors of reliable improvement in total bed-days were longer CCU episode of care, increased age, higher total bed-days pre-admission and primary diagnoses other than schizophrenia/schizoaffective disorder.

Significant reductions in ED presentations were identified at the group level. Using the individual-level data the RCI for ED presentations was 3.25 (rounded to 3). Reliable improvement was made by 12.5% consumers, while 6.3% showed reliable deterioration. For the regression analysis, the cohort was dichotomised into those who making and not making reliable improvements (12.5 and 87.5% respectively). Three statistically significant predictors of reliable improvement emerged: the presence of substance use issues (HoNOS item 3) and higher mental health and social functioning pre-admission (total HoNOS); and study site. Admission to one of the five CCU sites reduced the likelihood of improvement.

Accommodation instability

There were no statistically significant differences in accommodation instability from pre-admission to post-discharge. Using the RCI of 1.98 (rounded to 2), 19.5% consumers reliably improved (i.e. decreased the number of changes in primary residence) and 13.9% reliably deteriorated. When the 61 consumers who had transitioned from long-term in-patient care (i.e. no accommodation instability before CCU admission) were removed the difference between the mean number of changes pre-admission (2.61, s.d. = 2.34) and post-discharge (2.09, s.d. = 1.60) was statistically significant (n = 402, Wilcoxon Z = −3.85, p < 0.001).

Disability

There were no statistically significant changes in LSP-16 total scores between pre-admission and post-discharge.

Involuntary treatment

Statistically significant reduction in involuntary treatment between admission and discharge was identified. A total of 86.0% of consumers were discharged under the same treatment status; 12.6% were discharged under a less restrictive and 1.4% under a more restrictive treatment status. Logistic regression was conducted with the following two groups: less restrictive status (12.6%) and more restrictive/same status (87.4%). Four statistically significant predictors of the reduced likelihood of involuntary treatment emerged: longer episode of CCU care, mild-or-greater behavioural disturbance (HoNOS item 1) pre-admission, being younger and fewer pre-admission psychiatry-related bed-days.

Discussion

This study is the first published quantitative evaluation of contemporary community-based clinically-operated mental rehabilitation services in Australia. CCU care was followed by RCS improvement in mental health and social functioning for 43% of consumers. Consumers with poorer mental health and social functioning, and a longer episode of care were more likely to demonstrate RCS improvement. Additionally, many consumers experienced reliable improvements in relevant secondary outcomes. At the group level, significant improvements were observed when comparing the year pre- and post-CCU in mental health and social functioning, hospital bed days and ED presentations; but not accommodation instability and disability. Also, consumers were significantly more likely to be discharged with a less restrictive treatment status. These findings are generally consistent with the expected improvements following CCU care. However, the absence of change in disability was an important negative finding given the focus of care on the enhancement of independent living skills.

The observed reduction in HoNOS scores was consistent with previous studies evaluating rehabilitation service outcomes. The mean HoNOS score of consumers in the current study decreased by 4.9 points (from 18.4). This improvement is close to that observed by Macpherson et al. (2017), exceeds that reported by Barbato et al. (2007), but was less than that observed in samples of rehabilitation patients with lower baseline severity (Gonda et al., 2012; Maxwell et al., 2018). Different follow-up periods may be a factor affecting the magnitude of improvement observed across studies (Lim et al., 2017). It may be more appropriate to evaluate psychiatric rehabilitation services over longer periods given that these services focus on the improvement of social functioning in the community, which involves complex behaviours that are slow to change (Tsoutsoulis et al., 2018).

The finding of reduced mental health service use following rehabilitation care was also replicated (Grinshpoon et al., 2007; Tsoutsoulis et al., 2018). Compared to the year pre-admission, the average number of psychiatry-related bed-days in the year post-discharge decreased by 30 days (30.7%) and the number of ED presentations by 0.50 (30.5%). While our results corroborate positive trends in re-admission rates found in previous studies, it is important to acknowledge that service use is affected by system-related factors, such as access, availability and lack of alternative services.

The assessment of the individual change in consumers of CCUs revealed improvements in accommodation instability that were not detectable at a group level. A total of 19.5% of consumers made reliable improvements in accommodation instability from pre-admission to post-discharge. The failure to identify improvements at the group level was impacted by the absence of accommodation instability at baseline for 17.0% of the consumers included in the analysis. This finding highlights the relevance of focusing on individualised change criteria rather than group-based comparisons in mental health services research.

Methodological differences may explain the failure to replicate the improvements in disability (LSP-16) found in the Maxwell et al. (2018) study of an Australian inpatient rehabilitation service. Maxwell et al. (2018) considered baseline and follow-up scores as the closest available data to admission and the first available data 12 months post-discharge. In contrast, we adopted a more conservative approach comparing the highest recorded disability in the 12 months pre- and post-CCU care. Regardless, the absence of observed improvement in disability suggests that CCU services may optimise clinical recovery more than functional recovery. The high frequency of chronic disability associated with a diagnosis of schizophrenia is well acknowledged (Galletly et al., 2016), and efforts to address this should remain a focus of future research.

Measurement of the individualised change represents strength of this study. The proportion of consumers demonstrating RCS improvement is comparable to previous rehabilitation outcome studies using HoNOS (Barbato et al., 2007; Gonda et al., 2012; Maxwell et al., 2018). The use of local functional population data in the present study enhances the validity and generalisability of the findings to the Australian context. Analysis of the RCS change also identified that a minority of consumers deteriorate despite receipt of intensive rehabilitation care. Reliable deterioration occurred for 15.6% of consumers on HoNOS, and between 6.3% (ED presentations) and 25.3% (psychiatry-related bed-use) of consumers on secondary outcome measures. These adverse outcomes may relate to the stress and increasing demands associated with discharge (Gonda et al., 2012), loss of a supportive residential environment (Chopra and Herrman, 2011) and factors impacting access to mental health services such as premature discharge due to disruptive behaviour (Stopa et al., 2019). Reasons for discharge were not available, and further research to understand the profile of consumers who deteriorate despite receipt intensive rehabilitation support is warranted.

The current study also identified predictors of improvement among CCU consumers. Regression analysis indicated that RCS HoNOS score improvement was more likely among consumers with longer CCU admission length and those with higher baseline severity scores. In contrast, higher disability and psychiatry-related bed-use pre-admission were associated with a decreased likelihood of RCS improvement. Longer CCU admissions also predicted the reduced use of psychiatry-related beds and involuntary treatment. Additionally, decreased bed-use was statistically more frequent among older consumers, those with more bed-days before admission, and consumers with a diagnosis other than schizophrenia/schizoaffective disorder. The latter finding corroborates previous studies, which found that a diagnosis of schizophrenia was associated with poor rehabilitation outcomes relative to other diagnoses (Chatterjee et al., 2009; Yoon et al., 2013).

Unlike the findings by Gonda et al. (2012) where the presence of a substance use disorder was an important predictor of symptomatic improvement, the only outcome which was predicted by this variable was a reliable improvement in the number of ED presentations. Additionally, we also found that a single site emerged as a negative predictor of reliable improvement in ED presentations. This suggests differential functioning amongst the study sites and support services in the areas where they are located. This finding highlights the importance of considering site-based variation and examining both process and outcomes in mental health services research (Parker et al., 2016).

Cognitive impairment, a strong predictor of vocational rehabilitation outcomes for people affected by SPMI (Brune et al., 2011), did not emerge as a significant predictor in the present study. Explanations for this finding include possible divergence in the factors predictive of vocational rehabilitation and more general mental health rehabilitation and the inadequacy of using a single clinician-rated question as a measure of this construct. Future rehabilitation outcome studies would benefit from using more comprehensive cognitive assessments.

Limitations

The use of retrospective administrative data limited the variables available for analysis and data completeness (Connelly et al., 2016; Stewart and Davis, 2016). Adherence with routine outcomes monitoring protocols in Australia is known to be incomplete (Burgess et al., 2012). All assessments of the individual change were calculated based on a restricted sample of those with complete paired data. This limits the extent to which these findings can be relied upon to be representative of the full cohort. Missing data also precluded the use of education history as a predictor in the regression analysis.

When drawing inferences based on the RCS change it is important to consider limitations of the outcome measures and the nature of the comparison group. For example, while HoNOS total score may not be elevated, individual subscales may still indicate a clinically significant impairment in specific domains (Parabiaghi et al., 2005). Additionally, the likelihood of the ‘regression-to-the-mean’ for consumers with more severe scores needs to be considered. Furthermore, recovery and community functioning are multidimensional constructs that are influenced by decisions at the micro (individual patient care) and macro (policy) levels (Tulloch et al., 2008). Caution is needed when interpreting secondary outcomes as these will be partly dependent on factors outside of the CCU.

The predictive models offer only a partial explanation for variability in the rates of consumer improvement. Data was not available for two known predictors from the rehabilitation literature: medication adherence and engagement with formal education. Given that formal education is compulsory for all Australian children aged 5–15 years (Kennedy et al., 2014), the relevance of this variable appears less salient than medication adherence.

Finally, while controlling for several service-related variables (e.g. admission year, CCU site and length of stay), treatment-related data were not available regarding the specific therapeutic approaches received by consumers. Psychiatric rehabilitation is characterised by a mixture of practices and interventions, so information about the types of services received is critical to understanding best practice (Farkas and Anthony, 2010). The present study provides support for positive outcomes following CCU service engagement, however, further work is required to identify active therapeutic processes that may facilitate positive outcomes.

Conclusion

Most consumers showed reliable improvement and 43.0% showed RCS improvement in mental health and social functioning following receipt of CCU care. Additionally, most consumers showed reliable improvements in psychiatric bed-use. Favourable outcomes were observed at the group level for a range of relevant secondary outcomes, with the exception of improvement in disability. The failure to observe improvements in disability suggests that these services may have a greater impact on clinical than functional recovery. Significant predictors of RCS improvement in mental health and social functioning were longer duration of stay at the CCU, lower baseline mental health and social functioning and disability, as well as lower psychiatry-related bed-use pre-admission. The current results add to the small but increasing body of literature providing support for positive outcomes following receipt of psychiatric rehabilitation.