Effectiveness of an intervention to improve day centre services for people with psychiatric disabilities.

BACKGROUND/AIM: Day centres for people with psychiatric disabilities need to be evaluated for effectiveness in order to provide the best possible support. This study aimed at investigating the effectiveness of a tailor-made intervention to improve day centre services for people with psychiatric disabilities.
METHODS: The intervention was devised to bridge identified gaps in the services and lasted for 14 months. Eight centres were allotted to the intervention (55 attendees) or comparison condition (51 attendees). Fidelity to the intervention and major events in the day centres were assessed. The outcomes were degree of meaningfulness found in the day centre occupations, satisfaction with the rehabilitation received, satisfaction with everyday occupations and quality of life.
RESULTS: The fidelity to the intervention was good, but more positive events, such as new occupational opportunities, had taken place in the comparison units. No differences were identified between the intervention and the comparison group regarding changes from baseline to the 14-month follow-up in perceived meaningfulness among day centre occupations, satisfaction with everyday occupations or quality of life.
CONCLUSIONS: The intervention seemed ineffective, but the positive events in the comparison group resembled the measures included in the tailor-made interventions. This first intervention study in the day centre context has hopefully helped to generate hypotheses and methods for future research.

Introduction

Day centres for people with psychiatric disabilities may be a meaningful alternative to paid employment (Argentzell, Hakansson & Eklund, 2012a) and research has shown that satisfactory daily occupations in a broad sense, including self-care, work/studies and leisure, are associated with good health and wellbeing (Eklund & Leufstadius, 2007; Goldberg, Britnell & Goldberg, 2002). Community-based psychiatry provides rehabilitation and support programmes in day centres, which is in line with this knowledge. The centres are partly work-oriented and partly focus on other aspects of daily occupation, such as pastimes and socialising (Tjörnstrand, Bejerholm & Eklund, 2011). Day centres – sometimes denoted day treatment centres – are found in most Western countries; in Europe (Bryant, Craik & McKay, 2005; Kilian, Lindenbach, Lobig, Uhle & Angermeyer, 2001), in Canada (Rebeiro, Day, Semeniuk, O'Brien & Wilson, 2001), in the United States (Drake et al., 1994) and in Australia (Crosse, 2003). They are mostly run by community-based psychiatric services but may also be consumer run (Rebeiro et al.). Although this is thus a common and central rehabilitation alternative for a large number of people with psychiatric disabilities worldwide, no evidence exists, however, of its effectiveness (Catty, Burns, Comas & Poole, 2007). Studies so far have not indicated that people attending day centres have a better situation than non-attendees with respect to quality of life or satisfaction with daily occupations (Argentzell, Leufstadius & Eklund, 2012b; Eklund, Hansson & Ahlqvist, 2004). This raises the question whether existing day centres may be improved by identifying gaps in the services and developing strategies for how to fill those gaps with meaningful occupations. Research has shown that finding meaningfulness is one of the core experiences day centre participation may bring to the attendees (Argentzell et al.; Leufstadius, Erlandsson, Björkman & Eklund, 2008).

Although no evaluation studies exist, a few descriptive and qualitative studies can be found in the literature on participation in day centres for people with psychiatric disabilities (Gahnström-Strandqvist, Liukko & Tham, 2003), characterisations of occupations used (Tjörnstrand et al., 2011), descriptions of the target group (Catty & Burns, 2001) and characterisations of day centres vs. day hospitals (Catty, Goddard & Burns, 2005). Tjörnstrand et al. showed that day centres contain a multitude of opportunities for grading occupations, such that they can meet the needs of the attendees and provide relevant challenges. They argued that, if used adequately, day centres may form a rehabilitation chain and promote empowerment. Studies comparing day centre attendees with people having psychiatric disabilities but not attending day centres have shown that the day centre attendees tend to have more valued occupations (Argentzell et al., 2012b), a larger social network (Argentzell, Leufstadius & Eklund, 2013), and a more pronounced apprehension of having a worker role (Argentzell & Eklund, 2013). However, no differences have been identified between attendees and non-attendees regarding self-rated health, satisfaction with daily occupations, self-esteem or the quality of the social network (Argentzell et al.). Moreover, people using day centres were less prone to seek hospital care and expressed a stronger need for support regarding everyday occupations, compared to non-attendees (Eklund & Sandlund, 2012).

Research so far thus depicts a vulnerable group with strong needs for support and relying heavily on the day centres for having their needs met. Findings also indicate that the day centre rehabilitation potentials can be used more effectively (Tjörnstrand, Bejerholm & Eklund, 2013; Tjörnstrand et al., 2011). The aim of this study was thus to evaluate the effectiveness of a day centre intervention, specifically tailored for each day centre to bridge gaps identified in their services, and in that enrich and improve the services. We also wanted to investigate the degree to which the day centres adhered to the intervention. The research questions were:

Will a tailor-made enrichment of day centre services improve the services' ability to generate meaningfulness for the attendees, as perceived by attendees and staff?

Will the enrichment improve the attendees' situation with respect to satisfaction with the day centre support received, satisfaction with everyday occupations and quality of life?

To what degree did the day centres adhere to the tailor-made intervention?

Methods

This study, performed in Sweden, was a randomised controlled trial based on cluster randomisation of included units. It was approved by the regional ethical review board (No. 274/2008) and registered at ClinicalTrials.gov (No. NCT01732575).

The project covered a period of 14 months and included a baseline measurement, a follow-up after seven months and a second follow-up after 14 months.

Ordinary day centre services

There are two main types of day centres in Sweden – work-oriented and meeting place-oriented (Tjörnstrand et al., 2011). Work-oriented day centres offer scheduled work for the participants. The focus is on producing things to sell, or providing services such as catering, car washing or cleaning. Meeting place-oriented day centres, which may operate according to schedules or be based on open access, offer opportunities to play games, to do hobbies, to eat and socialise or to just relax. Some day centres encourage contacts with the community through, for example, a café open to the public or a second-hand shop. The attendees are normally there between 4 and 20 hours per week. The professional backgrounds for the staff are mostly orderlies, occupational therapists, social workers and craftsmen. Half of the centres included in this study, both intervention and control centres, did not have an occupational therapist.

The intervention

The tailor-made enrichment of the day centre service was developed in different stages. It was guided by results indicating the day centres have unused potentials (Tjörnstrand et al., 2011), by research about what can bring meaning to everyday life for people with psychiatric disabilities (Argentzell et al., 2012a; Leufstadius et al., 2008) and by discussions with a panel of users of psychiatric services, some of whom attended day centres.

The development of the intervention occurred in three steps. First, users and staff in all units filled out a questionnaire, the Estimation of Perceived Meaningfulness in Day Centres (EPM-DC) (Nilsson, Argentzell, Sandlund, Leufstadius & Eklund, 2011), described below. By analysing the responses, the research team created a feedback chart, aimed to show the staff how users and staff looked upon the opportunities for the users to engage in meaningful occupations in the unit.

The second step was an education and workshop package for the staff, designed and implemented by the research team. The first part of that package comprised of the day centre staff taking part in a one-day training session focussed on meaningful occupations and occupation-based rehabilitation, perceived meaning in everyday occupations (Argentzell et al., 2012a; Leufstadius et al., 2008) and client-centred practice (Sumsion & Law, 2006). Two workshops followed. During the first of those, the feedback mentioned above was given to the staff. Based on that, and on the steering documents for the respective units, the staff identified gaps in their services, but also practices that worked well and should be kept. The gaps served as incitements for changing the day centre to better meet the needs of the attendees and could concern additional occupations that were potentially meaningful, greater openness for occupations in the community or any other aspect of the services. The next stages during these workshops were for the staff to set goals for how to enrich and improve the services at their day centre, to suggest strategies for how to accomplish this, to identify key persons responsible for carrying out the intervention and to set a time plan for implementing the intervention. This process led to the formulation of an intervention plan for each day centre and was in accordance with implementation strategies described by Sundell and Roselius (2008), which state that for more successful implementation, the development of goals and strategies should involve the staff.

In the third and final step, the staff presented the preliminary intervention concept to the day centre attendees and received their feedback, which could result in revisions. When the attendees had approved the intervention and agreed it would potentially enrich the day centre and make attendance more meaningful, a date for implementation was set. Supervision from the research group was provided three times during the 14-month project period.

By these procedures, each unit developed and tailored its unique intervention. Examples of goals and strategies were to provide more physically demanding occupations by starting a gardening group, increase contacts with the surrounding society by opening a small shop, increase shared decision-making by introducing a weekly meeting for users and staff and improving the feedback to the users by introducing and following up individual plans.

Study context and selection of participants

The study was performed in three regions in the south of Sweden. Criteria for day centre inclusion were admitting only people with psychiatric disabilities, having at least 20 attendees on a regular basis and not participating in any other ongoing development project. Fifteen day centres were strategically selected and invited to participate in the study. They represented all of the three selected regions, were work-oriented as well as meeting place-oriented day centres and were located in both cities and rural areas. After an information meeting to which representatives for these day centres were invited, eight centres, who fulfilled the inclusion criteria, agreed to participate. They were labelled according to size, urban/rural location and work/meeting-place orientation and then grouped to form two equivalent groupings. The groupings were then allotted to form the intervention centres or the control centres. By this procedure we arrived at two comparable groups regarding characteristics of the included day centres (size, location and orientation).

Previous research has indicated that the Satisfaction with Daily Occupations (SDO) is a sensitive measure to detect differences between subjects with varying severity of mental illness (Eklund & Gunnarsson, 2008) and between groups differing on engagement in work-related occupations (Eklund et al., 2004). We, therefore, based our power calculation on the SDO. A study found a mean difference of 0.5 points on the SDO between groups of people with mental illness who had with varying structure to their everyday life (Eklund et al.2004). On the basis of the means and standard deviations from that study, we arrived at 41 participants in each condition as the desired sample size to detect a difference on the SDO of 0.5 with 80% power at P < 0.05. The inclusion criteria for the participants were: attending four hours per week or more at the day centre, not having a primary diagnosis of substance abuse, not having comorbidity of dementia or developmental disorder and being able to speak and read Swedish. The potential attendees were informed about the project at a meeting at the day centre. Ethical principles about voluntariness, confidentiality and the attendees' right to withdraw at any time were emphasised. Those who agreed to participate gave their written informed consent to a member of the day centre staff, designated to be a contact person between the research team and the day centre. The selection procedure resulted in 108 participants, 57 from the intervention centres and 51 from the comparison centres. Characteristics of the participants are shown in Table1.

Table 1

Socio-demographic characteristics of the respondents (N = 108)

Characteristics
Age (SD)	46 (11)
Gender: male/female; %	44/56
Civil status; married or cohabiting/single; %	19/81
Having friends; yes/no; %	82/18
Type of housing; %
Own apartment or house without support	69
Own apartment or house with support	26
Sheltered living	5
Educational level; %
Not completed compulsory school	4
Completed compulsory school	31
Completed 6th form college	56
Completed undergraduate studies	10
Diagnostic group; %
Psychoses (F20)	25
Mood and anxiety disorders (F30 + F40)	38
Autism/neuropsychiatric disorders (F80 + F90)	21
Other disorders (mostly F60 or not known)	16

The staff members were also asked to participate by completing one of the questionnaires, reflecting the service's ability to offer meaningful occupations to the attendees and which is further described below. All staff members at all centres agreed, comprising a total of 42 persons, 10 of which were men. Three were 20–35 years old, 20 were 36–50 years old and 18 were 51–65 years old.

Data collection

Research assistants, who were all occupational therapists with great experience from data collection for research projects, met each participant individually in a secluded room at the day centre. The present study was based on the instruments described below.

Background questionnaire and diagnosis

Socio-demographic data, including age, civil status, housing conditions and education, were collected to characterise the sample. Moreover, self-reported diagnosis was requested. Day centres in Sweden do not keep medical records and the attendees are not diagnosed at the centres. A specialised psychiatrist (the third author) coded the self-reported diagnoses according to the ICD-10 system (WHO, 1993). A previous study found self-reported diagnoses to be valid (Eklund & Sandlund, 2012), according to a procedure where different diagnostic groups were compared regarding types of symptoms.

Day centre's ability to generate meaningfulness for the attendees

A specific questionnaire was devised to characterise the centre's ability to generate meaningful occupations for the attendees. It was termed EPM-DC and has shown good psychometric properties according to the Rasch model, such as satisfactory construct validity and good scaling properties (Nilsson et al., 2011). It consists of 60 items, answered on a four-category response scale where a higher score indicates more meaningfulness. The EPM-DC exists in two versions, one for attendees and one for the staff. It was used to reflect if the intervention, aimed at enriching the unit with meaning-generating characteristics, had any effect at the unit level, as perceived by attendees and staff.

Satisfaction with the services received at the day centre

The Client Satisfaction Questionnaire (Larsen, Attkisson, Hargreaves & Nguyen, 1979) was used to assess satisfaction with the services received at the day centre. The original version comprises eight items measuring the clients' satisfaction with the care received and was slightly rephrased to suit the day centre context. A four-point scale was used, with response alternatives ranging from very dissatisfied (= 1) to very satisfied (= 4). A Swedish version was used, shown to have excellent internal consistency (Eklund & Erlandsson, 2013).

Satisfaction with everyday occupations

Satisfaction with everyday occupations in a broad sense was assessed by means of the SDO, which is an interview-based questionnaire covering the areas of work, leisure occupations, domestic tasks and self-care. The original version has nine items and has shown satisfactory psychometric properties in terms of internal consistency, construct validity (Eklund, 2004; Eklund & Gunnarsson, 2008) and test–retest reliability (Eklund & Gunnarsson, 2007). A seven-point scale was used, ranging from extremely dissatisfied (= 1) to extremely satisfied (= 7). For the present study an extended version with 14 items was used to cover more aspects of leisure occupations, domestic tasks and self-care. Cronbach's alpha analysis was performed on the data from all three measurement points, as this version has not previously been tested for psychometric properties, and the coefficients varied between 0.74 and 0.80. This indicates satisfactory internal consistency on par with the original nine-item version, and also with a Danish version of the extended SDO (Eklund & Morville, 2013). This latter study also showed satisfactory construct and criterion validity.

Quality of life

The Manchester Short Assessment of Quality of Life (MANSA) was used to estimate quality of life. The Swedish version was used, which has been found to be psychometrically sound in terms of construct validity and internal consistency (Björkman & Svensson, 2005). MANSA is administered as a structured interview and includes the individual's subjective rating of general life satisfaction as well as satisfaction concerning eleven quality of life domains, such as work, finances, social relations, leisure, housing situation, family relations and health. The ratings were made on a seven-point (1–7) scale, a higher rating indicating a higher level of satisfaction, with the mean rating from the different domains forming an overall quality of life score.

Major events and adherence to the intervention

To check for any major events in the centres and the staff's adherence to the intervention a questionnaire was devised. It was completed individually by staff members. The first part asked for any positive and negative events in the unit, such as a new and better location for the unit, better resources, unexpected or major staff turnovers, or serious illness or accidents affecting the atmosphere at the unit. Any such events were rated according their impact, from 1 (very little impact) to 7 (major impact). These questions were answered by staff at both the intervention units and the comparison units. For the staff in the intervention group, there were also questions regarding the degree to which they adhered to the intervention and implemented the intervention goals set, which varied in number from three to five and the degree to which the staff used the rehabilitation principles taught during the training session that introduced the intervention. All of these questions were rated on a scale from 1 (to a little extent) to 7 (to a great extent). This questionnaire was administered on four occasions at 2, 4, 7 and 14 months after baseline.

In the assessment of the major events, based on qualitative descriptions of the events and the staff ratings of them, an independent researcher rated the impact of positive and negative events for each day centre. This researcher had no knowledge of the study context, or of which units were intervention units or control units. No impact of major events was set at zero. A rating of −3 was set to denote an extremely negative situation regarding negative events, −2 a negative and −1 a somewhat negative situation. Analogously, positive values from 1 to 3 were used to denote a positive situation with respect to major events. The first author of this paper made a corresponding rating, without knowledge of the result from the independent researcher's rating. The ratings agreed perfectly in terms of negative events for all eight day centres and also perfectly in terms of positive events for five of the centres. There was deviation of one scale step for the remaining three. According to this formula for calculating rater agreement: alpha = 1 − (observed disagreement/expected disagreement), proposed by Krippendorff (2013), the alpha value was excellent at 0.93. The independent researcher's rating was used for the analyses.

Data analysis

The data were fairly normally distributed and parametric statistics were used. The χ2 test was used to examine ordinal data. Changes over time while also considering the group factor were analysed by means of repeated measures MANCOVA. Polynomial contrast was used in order to identify any differing trends between the groups at the three measurement points.

Imputation with the subject's mean was made in order to reduce attrition. This was performed in the cases where at least 75% and less than 100% of the items of a scale had been answered. The software used was the SPSS/PASW 20.0 (IBM Corp., Armonk, NY, USA).

Results

Major events and adherence to the intervention

Most of the day centres had experienced some change that was perceived as either a positive or a negative major event by the staff. The same event, such as staff turnover or moving to a new locality, could be perceived as partly positive, partly negative. The independent researcher's ratings of positive events were 0, 1, 1 and 2, respectively for the four intervention centres and 2, 3, 3 and 3, respectively for the four comparison units. This indicates more positive events occurring in the comparison units, and a statistically significant difference was found (P = 0.029). Examples of positive events that occurred to a greater extent in the comparison group were new and appreciated occupations and access to more appropriate premises. The negative events were rated at −2, −1, 0 and 0 in the intervention units at −1, −1, 0 and 0 in the comparison units. No statistically significant difference was indicated (P = 1.0).

The staff's general rating of adherence to the intervention is shown in Table2, together with the degree to which they implemented the intervention goals and how much they utilised the rehabilitation principles learnt. The only significant difference over time concerned the readiness to implement the intervention goals, which was lower at the first measurement. The mean ratings were > 70% of the maximum score for all the adherence variables.

Table 2

Staff ratings of adherence to the intervention; mean (SD)

	At two months	At four months	At seven months	At 14 months	P-value
General fidelity (maximum score 7)	4.8 (0.9)	4.9 (1)	5.1 (1.1)	5.4 (1.2)	0.211
Implementation of intervention goals(maximum score 21)*	12.9 (2.8)	15.8 (2.7)	16.1 (2.72)	16.7 (3.2)	0.006†
Utilisation of rehabilitation principles (maximum score 28)	19.9 (3.8)	21.3 (4.2)	20.7 (4.8)	21.3 (4.3)	0.116

The variable is based on the first three mentioned intervention goals. Only one unit had set more than three goals.

The P-value signifies that the rating at two months was lower than all the other ratings.

Perceived meaningfulness among attendees and staff

There was no difference between the intervention group and the comparison group, regarding change in perceived meaningfulness between the measurements points, for either the attendees' (F = 0.001; P = 0.977) or for the staff (F = 0.795; P = 0.381).

Satisfaction with day centre support, everyday occupations and quality of life

No differences between the groups could be identified regarding change over time concerning satisfaction with the day centre support received (F = 0.876; P = 0.352), satisfaction with everyday occupations (F = 0.876; P = 0.352) or quality of life (F = 0.454; P = 0.502).

The statistics presented in the result section concern a linear trend, but all tests of a quadratic trend were non-significant as well.

Discussion

The day centre staff largely adhered to the tailor-made intervention. Still, the intervention did not result in any differences between the intervention group and the comparison group. This was a non-anticipated finding, but still an important result in relation to the steadily present long-term aim of improving the support to people with psychiatric disabilities. Some possible explanations to the pattern of findings will be discussed.

The intervention was insufficient

This is the first study, to our knowledge, that evaluates an intervention in the day centre context. Very little research exists in this field and the studies that have been performed are qualitative and quantitative descriptions of the attendees and the services (Catty & Burns, 2001; Gahnström-Strandqvist et al., 2003; Tjörnstrand et al., 2011). It is thus impossible to deduce if the non-findings reflect a typical or unique situation. The intervention appears to have been insufficient. It is possible to speculate that an intervention designed to target the centre routines would not impact on the individual level; however, the intervention was not effective at the centre level either. The way the units' meaning-generating features developed over time did not differ between the intervention units and the comparison units, according to the opinions from both attendees and staff. A more individualised approach may still have been beneficial, especially because not all attendees participated in the study. This may negatively have affected the psycho-social climate, known to be important to outcomes of interventions (Eklund & Hansson, 1997). Meaningfulness is also a highly individual phenomenon, which underscores that a more individualised approach in the intervention might have been advantageous.

A Hawthorne effect, i.e. a positive outcome of the mere focus on a group and its undertakings, is likely to have occurred in both groups of attendees. They were subjected to the same amount of interviewing and number of visits from the research group during the period of data collection. A Hawthorne effect has been demonstrated in previous research, particularly for psychological dimensions (Bouchet, Guillemin & Briancon, 1996), and may have obscured any minor effect of the intervention.

The education and workshop package during which the intervention was developed was clearly based on occupational therapy theory and research. However, only few staff members were occupational therapists, and they may not have taken full advantage of the education and workshop package. A study aimed at promoting health among healthy elderly people showed that a generalised social activity group, staffed with non-professionals, was less effective compared to a lifestyle intervention group staffed with occupational therapists (Clark et al., 1997). Thus, further efforts to improve day centre services by adding meaningful occupations should consider the importance of having occupational therapists among the staff. Furthermore, although the importance of allowing the centres to take the lead in developing the interventions was acknowledged (Sundell & Roselius, 2008), the balance between staff and user influence may have been suboptimal. Consumer-run services have been shown to produce good outcomes (Rebeiro et al., 2001) and the attendees in the present study could have been involved earlier on in the development of the intervention. It was developed for the mainstream municipality-run services in the community, and did not aim for a consumer-run alternative, but participation and shared decision-making (Adams & Drake, 2006) are desired components in the municipality-run services as well. Involving the attendees in the one-day training session and the workshops would possibly have resulted in a more powerful intervention.

Another factor is the time frame for the intervention. Without having any results from previous intervention studies in the day centre context, we assumed 14 months would be sufficient time to allow for the intervention to work. This may have been too short a period for the changes to become visible to the attendees. Research has shown that changes in mental health intervention programmes that are recognised by the staff are much less obvious to users (Eklund & Hansson, 2001; Maitra & Erway, 2006).

There were more positive events in the comparison group

More positive events occurred in the comparison group in the present study. Major events may affect the psycho-social atmosphere in the day centre, which has been demonstrated in previous research (Eklund & Hansson, 2001). Many of the reported positive events in the comparison group were also similar to the measures taken to enrich the intervention centres, such as more and varying occupations offered to the attendees. Better prerequisites for the day centre occupations in the comparison group may thus have worked in the same direction as the programme changes in the intervention group.

Methodological shortcomings produced the result

The instruments were in general well-tested, also regarding sensitivity to change. However, the EPM-DC, assessing perceived meaningfulness in day centres, is a recently developed measure, not yet tested for sensitivity to change. It cannot be disregarded that it may have been too blunt to detect changes. Unfortunately, no alternative or complement was available, because this seems to be the first and so far only instrument to characterise day centres with respect to the meaningfulness the attendees can find when engaging in the day centre occupations.

Another methodological reflection concerns the statistical power of the study. We obtained the desired sample size, with some margin, and the length of data collection and the number of day centres included in the study must be seen as strength. However, previous research did not provide a solid basis for the power calculation, and the present study indicates that smaller differences than 0.5 on the SDO should be expected and that the study may have been underpowered. A post-hoc analysis based on the differences in SDO change obtained in the present study indicates a very large sample, > 300 in each group, for the difference to become statistically significant. That difference corresponds to an effect size of 0.23, which is considered small (Wampold, 2001) but still clinically interesting (Bolier et al., 2013). In the light of this post-hoc analysis, the present project may serve as a pilot study for the planning of future studies in the day centre milieu.

Finally, the day centres varied in size, location (urban or rural) and orientation (work or meeting-place orientation) and were therefore stratified and purposefully grouped before the drawing of lots to either condition (intervention or comparison). This was not strict randomisation, but was considered to be the best alternative in this naturalistic study.

Conclusion

It is important to report studies of non-findings and this study can hopefully serve as a basis for future occupation-based intervention studies in day centre contexts. Our suggestions would be to include consumers earlier on and more deeply in the intervention development and to combine changes aimed at enriching the programme with individualised measures. Future research should consider the importance of having an occupational therapist among the staff. Another suggestion would be to expect smaller differences between the intervention and the comparison groups and thus calculate for a larger sample. A cross-over design could also be considered in order to counteract the error variance generated by the day centres' variation in location, size and orientation. Finally, further instrument development is needed as there is currently a lack of measures suitable for the day centre context.