Study protocol for a controlled trial of Strengths Model Case Management in mental health services in Hong Kong.

INTRODUCTION: Although strengths-based models are popular within recovery-oriented approaches, there is still a lack of conclusive research to guide how they should be implemented. A recent meta-analysis confirmed the lack of clarity in how this perspective is operationalised and that fidelity monitoring during the implementation process is lacking. Hence, there is a clear need to evaluate the feasibility of delivering and evaluating a clearly operationalised strengths-based intervention that incorporates fidelity checks to inform more definitive research. This protocol therefore describes a controlled trial of Strengths Model Case Management (SMCM), a complex intervention, for people with severe mental illnesses in Hong Kong. This trial follows the guidelines of the Medical Research Council as a phase 2 trial. Hence, it is a pilot study that tests the feasibility and effectiveness of the model. METHODS AND ANALYSIS: This is a 9-month controlled trial that uses the Kansas Model. Participants and a matched control group are recruited on a voluntary basis, after screening for eligibility. Effectiveness of the SMCM will be measured through outcome measures taken at baseline, the mid-point and at the end of the trial. Outcomes for service users include personal recovery, hope, subjective well-being, psychiatric symptoms, perceived level of recovery features within the organisation, therapeutic alliance and achievement of recovery goals. Outcomes for care workers will include job burnout, organisational features of recovery and perceived supervisory support. With a 2×3 analysis of variance design and a moderate intervention effect (Cohen's d=0.50), a total of 86 participants will be needed for a statistical power of 0.80. ETHICS AND DISSEMINATION: Ethical approval has been obtained from the Human Research Ethics Committee for Non-Clinical Faculties at The University of Hong Kong (HRECNCF: EA140913). TRIAL REGISTRATION NUMBER: Australian New Zealand Clinical Trial Registry (ACTRN)12613001120763. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

First clinical trial that utilises the Kansas Model of Strengths Model Case Management.

This clinical trial couples with fidelity monitoring during implementation.

Primary evidence of feasibility and effectiveness of using a Strengths Model Case Management will be established.

Lack of randomisation.

Dropout rate may be high.

Background

The development and implementation of recovery-oriented, strengths-based approaches are emerging, with more qualitative as well as quantitative views of recovery-oriented practices in general, and there are indications that Hong Kong is taking up speed in implementing recovery-oriented practices. Most notably, the Mental Health Service Plan for Adults states that “[t]he vision of the future is of a person-centered service based on effective treatment and the recovery of the individual” (ref. 1, p.5).1 The strengths perspective has a long philosophical history since it was officially popularised by Saleebey2 (for a recent review on the development of strengths perspective see Rapp and Sullivan3). A distinct and noteworthy feature of the strengths perspective is that it is a highly individualised and inductive concept based on the premise that meanings and reality are constructed through personal narratives.

In recent years, researchers have advocated that the strengths-based approach be applied among people with psychiatric conditions,4–6 and it has gradually evolved into a set of guidelines and tools designed to enhance recovery outcomes for those with both mild and severe psychiatric disabilities.5–7 In the 1980s, the University of Kansas School of Social Welfare developed and synthesised the strengths philosophies and systematically operationalised how it should be implemented. They developed three primary tools of the Strengths Model Case Management (SMCM) and fidelity scales.

The three core elements of SMCM are: (1) strengths assessment, (2) personal recovery plan and (3) group supervision. The strengths assessment appraises the users’ skills, talents and environmental strengths that are important and meaningful to them in the present, as well as in achieving their goals in the past. The personal recovery plan helps service users take small, specific and measureable steps towards a goal until it is achieved. Group supervision increases the supportive environment for direct care workers to help service users around their identified goals.8

The central tenet behind SMCM is to assist service users in identifying strengths and resources, both personal and those available from the environment. Through realising those strengths, users are inspired to achieve their aspirations as they define them. They are also inspired to integrate into the community, thus improving overall quality of life. SMCM is guided by six principles (ref. 6, p.52–62):

People with psychiatric disabilities can recover, reclaim and transform their lives.

The focus is on individual strengths rather than deficits.

The community is viewed as an oasis of resources.

The client is the director of the helping process.

The case worker–client relationship is primary and essential.

The primary setting for our work is the community.

To date, 12 empirical studies have examined the effectiveness of SMCM in mental healthcare settings.4 9–19 In terms of research designs, only one of these studies used a randomised control trial design. The others were quasi-experimental with a pre-post design, between-group comparison or secondary data analyses. Across all of the studies, 18 different outcome measures were used. Out of these 18 outcomes, the most common was rehospitalisation, while others focused on psychosocial outcomes such as education, housing, vocational outcomes and finances.

Most studies have demonstrated that SMCM was effective in improving these service users outcomes—especially in employment—and have reported greater physical and mental health than before they were exposed to SMCM.9–11 13 14 16 A recent study suggested that applying the model with higher fidelity leads to better outcomes for the service users. Fukui et al19 used a total of 14 SMCM teams in nine different community centres and achieved an overall fidelity of 87%. Furthermore, all of these teams demonstrated a significant reduction in hospitalisation and gain in competitive employment. In no study did service users do worse when they received SMCM.

However, the limitations of these studies lie not only in the fact that there are merely a handful of experimental studies, and two of those studies had a particularly high attrition rate. Even so, it was not documented whether those individuals also dropped out of community mental health services they were receiving at the time (over 50% attrition in Modrcin et al,15 and 24% in Macias et al20). There has been no rigorous trial of SMCM coupled with high fidelity scores (ie, implementation monitoring) to study the effectiveness of the intervention.

In addition, whether SMCM has any impact on personal psychosocial outcomes such as subjective well-being, hope and the level of recovery stages still remains an unexplored area to date. It is also unknown whether the application of SMCM has any impact on care workers’ job burnout and perceived level of supervision support. Higher burnout would compromise not only the psychological well-being of care workers but also the quality of care they deliver.21 22 Thus, it is imperative to look into the outcomes of care workers when evaluating a service model in order to establish its long-term effectiveness.23 24

Furthermore, according to a recent meta-analysis, SMCM remains poorly operationalised and inadequately described in all previous studies.25 Drawing on previous findings, it is clear that SMCM needs to be conducted with stringent and well-defined operationalisation of implementation procedures, coupled with fidelity monitoring. Hence, this trial will use the Kansas Model which is a complex intervention26 as detailed in the SMCM intervention manual6 and its associated fidelity scale27 to test the feasibility and effectiveness of SMCM in Hong Kong. The trial is considered to be a phase 2 pilot trial according to the guidelines of British Medical Research Council (MRC) in complex interventions design and evaluation.28 29 This study protocol is reported according to the guidelines of the Template for Intervention Description and Replication (TIDieR).30

Methods

Trial synopsis

This study will be conducted by university researchers, a peer researcher who has experience as a service user, and clinicians from three non-governmental organisations (NGOs) providing residential services for people with severe and persistent mental illness. This is an assessor-blind, 9-month pre-post controlled trial to test the feasibility and effectiveness of SMCM in Hong Kong (figure 1). Outcome measures will be collected at baseline, month 4.5 and month 9. This is justifiable because high fidelity can be achieved within 6 months of the implementation. Furthermore, given the relatively high attrition rates reported in the aforementioned studies (between 24% and 50%), the planned time points will provide us opportunities to closely monitor the activities that may deter people who are at risk of withdrawal.

Figure 1

Diagram showing the design of the study (SMCM, Strengths Model Case Management).

A developmental study or quasi-experimental design is appropriate for the following reasons: (1) SMCM is a complex intervention29 and a novel case management practice in Hong Kong or wider Asia. Hence, it is fitting to run this pioneering trial in Hong Kong which will inform more rigorous trials to be appropriately designed in the future. (2) Although the outcome measures used in this study have been carefully chosen by the project team, they were developed and validated in the West. Thus, it is unclear how they may be applied in the Chinese linguistic and cultural context. The cultural adaptions and understandings of SMCM will be addressed in a separate qualitative study. (3) SMCM fidelity is not established around individual workers but involves a lot of structural (eg, having regular group supervision, field mentoring) as well as cultural (eg, staff's attitude, the languages used) changes in the workplace; therefore, it needs a developmental study to formulate specific strategies to achieve the workplace transformation.

Finally, this study will adhere to international standards such as that of the British MRC.29 Thus, this intervention will pilot sample sizes by estimating recruitment and retention sizes, testing procedures, outcome measures, and effectiveness. All of this information will be useful for a more definitive and rigorous trial in the future.

Objective

The 9-month trial will be launched in Hong Kong with the objective of assessing the feasibility and effectiveness of SMCM. The feasibility investigation will be achieved by documenting recruitment numbers, dropouts and retention in the final wave of data collection. Effectiveness will be assessed in terms of personal recovery, psychosocial outcomes and vocational outcomes for the service users. Care worker outcomes will include job burnout and perceived supervision support.

Qualitative data to complement findings of this current trial will also be gathered. Specifically, during the course of the trial, feedback about the perceived barriers and facilitators of SMCM will be solicited. This will be done to investigate changes in trial participants—if any—in the process of implementation based on their subjective experience. In addition, interviews will be conducted at the end of the trial to document the experience of service users in receiving SMCM. These data will be analysed and reported in a separate qualitative study.

Sample size and statistical power

With a 2×3 analysis of variance design and a moderate intervention effect (Cohen's d=0.50),31 a total of 86 participants will be needed for a statistical power of 0.80. Instead of a smaller effect size, we have estimated a medium intervention effect due to the realistic expectation of a reasonably high fidelity score, which has previously been demonstrated to have a significant positive effect on outcomes.19 Given resource constraints, a total of 160 participants (80 from each group) will be recruited.

Participants

This study will target residential rehabilitation service users because their goals are generally to advance in their recovery stage and achieve community reintegration. In Hong Kong, there are three types of residential rehabilitation services that are provided for people with severe mental illness: (1) Supported hostels provide residential services for those in recovery and who live semi-independently with some assistance from hostel staff. (2) Halfway houses provide residency for those in transition with an aim to improve functioning and achieve reintegration to the community. (3) Long-stay care homes provide rehabilitation services for those that have a chronic but stable condition and are in need of nursing care.

This study will draw participants from these three types of residential facilities operated by three different local NGOs. Over 90% of service users residing in the facilities are diagnosed with schizophrenia spectrum and other psychotic disorders. Within the three participating NGOs, a total of six sites (two of each type of residential facility described above) are involved. Of these six sites, three (one for each type of residential facility) will be implementation sites, and three will be comparison sites. Both long-stay care homes cater to males only, and most of the users from all settings are in mid-adulthood (≥40 years old). Table 1 depicts the recruitment plan.

Table 1

Recruitment plan

	Number of recruitments from each setting			Total number of recruitments (before potential attrition)
Types of setting group	Halfway house	Supported hostel	Long-stay care home
Control group	30 NGO-1*	20 NGO-3	30 NGO-5	80
Intervention group	30 NGO-2	20 NGO-4	30 NGO-6	80

*NGOs 1–6 represent six separate residential settings (of three NGOs) to avoid any across control-intervention group contamination.

NGO, non-governmental organisation.

A number of participants matched on age and gender who are diagnosed with either schizophrenia spectrum, bipolar disorder or other psychotic disorders will be recruited from the same type of residential setting to form a control group after recruitment for the intervention group is completed. The procedure of control group recruitment is identical to the intervention group, of which the procedure is elaborated in the following paragraph. Moreover, social workers, nurses, occupational therapists and programme workers from all the six recruitment sites will be invited to participate as mental health professionals.

Recruitment and sampling of service users

Prior to recruiting participants, formal invitation letters with information on the trial will be distributed to all six implementation sites. A generic study title will be used in the invitation letters (mental health recovery research) for both the intervention and control groups in order to avoid potential knowledge of group membership for both the staff and service users. The recruitment will be overseen by the person in charge of each residence and the project principal investigator (W-SET) according to the inclusion and exclusion criteria. Participants who agree to participate will then be assessed for eligibility according to the criteria listed below to receive SMCM. Those who refuse to participate will continue with their usual rehabilitation.

At least 18 years of age;

Consent to participation;

Is able to read and comprehend Chinese;

Those with a diagnosis of either schizophrenia or bipolar disorder given by the participant's treating psychiatrist;

Is currently a user of mental health services from one of the six participating sites.

Exclusion criteria

Service users will be excluded if they are currently experiencing a crisis or if they have serious mental impairments and thus have difficulty participating in the research in any way, as determined by their care workers.

Recruitment of care worker participants

All mental health staff responsible for the delivery of intervention will be invited to participate in the study. A matched number of care workers from the control sites will be recruited.

Consent of participants

All participants will be briefed about the study objectives, rationale (without disclosing knowledge of their group membership) risks and benefits of joining the study, before administration of the questionnaire. The participants will then be asked to fully review the consent letters given to them. Finally, they will be asked to sign on two copies of the same consent letter, of which one will be returned to the interviewer for record keeping and the other to be retained by themselves.

Intervention groups

The Strengths Assessment and Personal Recovery Plans are used whenever appropriate during the intervention sessions. A copy of the Strengths Assessment and Personal Recovery Plans can be accessed at http://mentalhealth.socwel.ku.edu/fidelity-resources-0. The intervention will consist of regular individual sessions (approximately once every 1–2 weeks), each lasting for 30–60 min, preferably taking place in the community (eg, nearby parks, fast food locations for tea; SMCM principle 6). The SMCM intervention is to be run for the entire course of 9 months.

A typical session consists of natural, hope-inducing, strengths-based conversation between the service user and the care worker. All care workers from the intervention group are adequately trained to deliver SMCM. They received a 2-day training workshop hosted by trainers of SMCM from the University of Kansas (KU), and since 2012, the care workers have attended bi-monthly supervision sessions via internet videoconference with the same trainer from KU, to discuss a few selected pilot cases in preparation for this trial. The purpose of these meetings is to uphold the quality of SMCM work being delivered to the service users through ongoing monitoring and improvement of care workers’ practice skills.

During the interventions, the care workers help the service users identify recovery goals and activities that are meaningful for them in their own recovery. Then they help them achieve these goals by breaking the goals down into achievable steps. Moreover, supervisors review participants’ progress by referring to the information recorded in the Strengths Assessment and Personal Recovery Plan and providing field mentoring and group supervision among teams of care workers (the highest fidelity requires this to be done weekly).

SMCM Fidelity Scale

To ensure the integrity of the intervention, the SMCM Fidelity Scale, developed by the University of Kansas School of Social Welfare, will be administered by a local trainer (who is independent from the recruitment sites) who has more than 10 years of experience in using SMCM jointly with a co-trainer who has personal experience of mental illness. The fidelity review will be conducted prior to start of the trial. The feedback and follow-up training will be given to improve the scores and reach high fidelity. Based on the fidelity protocol, 6-month fidelity will also be measured. High fidelity is achieved when a programme reaches an average of 4 (out of 5-point scale: 1=low fidelity; 5=high fidelity) for the structural items; an average of 4 for the supervision/supervisors items; and an average of 4 for the clinical/service items.27 Precise details of the fidelity scale can be accessed online at http://mentalhealth.socwel.ku.edu/fidelity-resources-0

Control groups

Participants assigned to the control groups will continue the rehabilitation they have been receiving from their respective agencies (treatment as usual (TAU)). This TAU will be the same treatment as those participants in the intervention group have been receiving prior to SMCM. We do not dictate the control treatment because it differs by setting. Typical content of the usual rehabilitation treatment includes regular face-to-face sessions with the care workers, medical appointments and general community activities (eg, outings, lunches).

Interviewers

At the time of the writing of this protocol, four persons who are current mental health service users reaching a state of advanced recovery are employed as part-time research assistants. They will be responsible for administering the questionnaires to research participants. Before fieldwork begins, they will attend a training workshop which will cover five areas: (1) nature of the study; (2) briefing of the questionnaires being used in the current study; (3) research ethics such as the proper handling of sensitive interview data, storage of questionnaires, issue of confidentiality; (4) practical guidelines on how to build rapport and engage research participants; and (5) pointers for handling emergencies or unanticipated incidents (eg, participants were upset by the study). Group memberships of the participating sites will not be disclosed in order to achieve assessor blinding. All of peer researchers will be requested to sign a pledge of confidentiality at the end of the training.

Outcome measures

Feasibility outcomes

According to the guidelines of the British MRC,29 feasibility assessment includes testing procedures for their acceptability, estimating the likely rates of recruitment and retention of participants, and the calculation of appropriate sample sizes. In terms of acceptability of testing procedures, five individuals from a different psychiatric residential setting will be recruited to complete the outcome assessments, and their feedback about the wording of the questionnaires and testing procedures will be solicited. Recruitment and the reasons for dropouts during the trial will be fully documented.

Process evaluation

A qualitative inquisition will be conducted parallel to this trial to delve into the process of implementation of SMCM. Two types of data will be collected. First, during the course of the trial, qualitative feedback will be gathered periodically from the case workers in the participating sites to identify the difficulties or facilitators in implementing and adopting the trial in their respective setting. Second, researchers’ observations in the form of field notes will be analysed. All of these data are integral to the process evaluation.32

Effectiveness outcomes for service users

The hypothesis for the primary outcome is that recovery, as measured with the Maryland Assessment of Recovery in People with Serious Mental Illness (MARS33), will be higher in the SMCM than in the control group at the 9th month postintervention measure controlling for baseline recovery scores and other control variables.

Hypotheses for SMCM on secondary outcomes are that, controlling for differences in baseline (if any), the SMCM group will show better results than the control group at the 9th month post-intervention measure on: (1) subjective well-being, or satisfaction with life, as measured by the Satisfaction with Life Scale (SWLS34); (2) state of hope as measured by the State of Hope Scale (SHS35); (3) psychiatric symptoms as measured by the Brief Psychiatric Rating Scale (BPRS36); (4) perceived level of therapeutic alliance as measured by the Working Alliance Inventory (WAI37); and (5) organisational features of recovery as measured by the Organizational Climate Subscale (OCS38), one of the sections of the Recovery Enhancing Environment Measure (REEM); (6) recovery goals. Control variables will include demographic information as measured using a two-page self-constructed survey.

In this survey, participants will be asked to write down recovery goals in different life domains (eg, social, financial) they set in the previous 3 weeks. Then, for each goal, participants will be asked to rate the progress in achieving such goals on a scale of 1–5, where 1 denotes no progress, and 5 denotes that the goal was achieved. Information about the transition to independent living, competitive employment, further education and rehospitalisation will also be obtained in the survey. All of these instruments have been translated and validated in Chinese, with the exception of the WAI, which will be translated into Chinese in accordance with established guidelines.39

Outcome measures for care workers

Hypotheses for the effect of SMCM on care workers are that the following will show better results than the control group at the 9th month postintervention measure: (1) burnout as measured by the Maslach Burnout Inventory (MBI21); (2) organisation features of recovery as measured by the Organization Climate Subscale (OCS38) and (3) perceived level of supervisory support as measured by the Perceptions of Supervisory Support Scale (PSS40). All of these instruments have been translated and validated in Chinese, with the exception of PSS, which will be translated into Chinese following established guidelines.39

Statistical analyses

The rate of recruitment and retention of participants will be presented using descriptive statistics in order to establish the feasibility of adopting SMCM on a larger scale. Then background information including sociodemographic characteristics and all outcome variables will be summarised based on the implementation conditions. After univariate and multivariate outliers are examined, demographics and outcome scores prior to the intervention will be examined to investigate the equivalency of the group characteristics between the two groups. This is particularly important because the study will not use random assignment. If there are any differences, they will be controlled for in the tests of the main hypotheses. The mixed model approach, also known as multilevel modelling or hierarchical linear modelling will be used to examine the intervention effect (group and time interaction). A significance level of p<0.05 will be used. Mixed model uses maximum likelihood estimation to handle missing data, without the use of ad hoc imputations.41 All statistical analyses will be carried out using JMP Pro V.12.42

Discussion

SMCM is a user-directed, recovery-oriented approach that advocates service users’ autonomy and facilitates recovery as defined by the users themselves. SMCM emphasises users’ own strengths and priorities, reflecting the core values of recovery-oriented practices.43 44 This trial will potentially provide considerable insight into whether SMCM is feasible and effective in psychiatric residential service settings in Hong Kong or the wider non-Western context, responding to the need to promote evidence-based practices in the social work profession. To the best of our knowledge, this trial will be the first of its kind conducted in Asia.

The significance of this trial is twofold. First, this trial will add to our understanding of how to conduct effectiveness studies of strengths-based interventions (operationalised in a form of SMCM along with the fidelity scales) in a Chinese community and help the project team to design a more rigorous evaluative trial for SMCM. The information and knowledge collected from the feasibility study can inform organisational level changes (eg, the running of group supervision, how to best organise field mentoring) within the agencies and at the individual client level.

Second, the results of this trial can establish preliminary evidence as to whether SMCM is useful and beneficial for service users, as well as for the workers providing care for them. However, it is important to note that this trial will only provide a preliminary indication of effectiveness and that its primary purpose is to assess feasibility and acceptability. While it may not provide solid conclusions for SMCM effectiveness in Chinese culture, this pilot study is warranted to build our capacity for a more rigorous testing such as a randomised control trial with active control groups in the future. This study will also help to determine which of the primary and secondary outcomes are likely to be the most relevant for a definitive trial in the future.

This trial has limitations in that it lacks randomisation and may have a high dropout rate. Missing data as a result of dropouts will be handled with an investigation of this bias by comparing characteristics of participants who have completed all outcome measures at all three time points with those who have incomplete data or were lost to follow-up in order to establish predictors for discontinuation. There may also be contamination of the control group, since all the three of the participating NGOs are using the recovery approach in general across their mental health services.

However, the potential contamination is thought to be minimal because participants from the intervention and control groups come from different residential settings, each of which is managed by different staff. Moreover, fidelity checks will be conducted prior to the start of trial, and there will be ongoing monitoring throughout the trial to ensure integrity of SMCM in the experimental sites. Notwithstanding these methodological challenges, the findings and output (eg, appropriate outcome measures to be used, extent of burnout among care providers between the control and intervention groups) from the proposed study will take us significantly closer to both understanding recovery in Chinese people with severe mental illness and designing evidence-based, recovery-oriented psychiatric services through strengths building and empowerment.