Improving Cognitive Skills for People with Mental Illness to Increase Vocational and Psychosocial Outcomes: The Employ Your Mind Program.

Cognitive impairments contribute to difficulty obtaining employment for people with severe mental illness (SMI). We describe an evaluation of a program, Employ Your Mind (EYM), which integrates cognitive remediation with vocational rehabilitation to improve cognitive skills and psychosocial outcomes relevant to employment. Participants with SMI were referred to WISE Employment and completed the six-month EYM program. Assessments of psychosocial functioning, cognition and vocational data were collected at baseline and completion, and additional vocational outcomes were collected at 12-month follow-up. Psychosocial functioning and cognition were compared pre- and post-EYM and vocational outcomes were compared for the year prior to EYM and for the 12-month follow-up for program completers. Thirty-two participants commenced the EYM program and 21 (65.6%) completed it. Completers reported significant improvements in mental wellbeing, quality of life and enhanced overall perceived working ability. Participants also demonstrated significantly enhanced speed of processing. Of the 15 participants who reported vocational outcomes, four (26.6%) were engaged in competitive paid employment in the year prior to EYM commencement and eight (53.3%) in the year following EYM commencement. The results indicate that EYM helps improve cognitive performance, psychosocial outcomes, and work readiness in people with SMI.

Introduction

Severe mental illnesses (SMIs) often lead to psychosocial disability, affecting an individual’s capacity to interact with others, work, and lead ‘a contributing life’ [25]. Cognitive impairments associated with SMIs can include difficulties with memory, concentration, organised thinking, planning, problem-solving, and goal setting [13, 29]. On average, people with psychotic disorders score 1.6 standard deviations below the general population in cognitive ability,this has a significant impact on everyday functioning and ability to undertake everyday activities [23].

The majority of people with SMIs want to work and contribute to society [23]. The mental health benefits of work for this population are well established. A systematic meta-review by Modini et al. [22] found that ‘the role work can play in facilitating recovery from an illness and enhancing mental well-being needs to be highlighted and promoted more widely’ (p. 331). In addition to an income, identified mental health benefits of work for people with SMI include: enhanced structured daily activities, improved self-esteem, feeling a useful member of the community, and provision of social opportunities [9, 22]. Work remains elusive for many, however. Australian data show that only one in four people with a psychosocial disability (25.7%) are employed, compared with 80.3% of those without disability [1]. An important contributing factor to this may be that many modern work roles make complex demands on cognitive and social skills, which are often impaired in individuals with SMIs [23]. Psychosocial rehabilitation programs, particularly those focusing on vocational goals, need to address this issue.

The Cognitive Remediation Experts Working Group [7] define cognitive remediation (CR) as an ‘intervention targeting cognitive deficit (attention, memory, executive function, social cognition or meta-cognition) using scientific principles of learning with the ultimate goal of improving functional outcomes. Its effectiveness is enhanced when provided in a context (formal or informal) that provides support and opportunity for extending to everyday functioning'. CR has been shown to be effective in improving cognitive function in people with SMIs [40]. A review of CR interventions in individuals with SMIs found that computer-based CR exercises are equally as effective when delivered via computer or clinician, with computer-based CR more easily standardised [10]. Vocational rehabilitation can improve everyday living and employment-related skills [3, 14, 19] with the suggestion that the future of CR for individuals with SMI lies in ‘developing effective programs that combine CR and psychosocial and vocational rehabilitation’ [10, p. 4].

The EYM program incorporates core features of CR as identified by Bowie et al. [4]: use of qualified cognitive remediation therapists,cognitive exercise aimed at improving cognitive functioning; procedures to develop problem-solving strategies, and procedures to facilitate transfer to real-world functioning. A recent meta-analysis by Lejeune et al. [17] concluded that programs that incorporate bridging sessions and strategy coaching are ‘more cognitively potent’. The EYM program also incorporates lessons gleaned from the evaluation of the EYM pilot program [21], including use of validated assessments to measure cognitive and psychosocial outcomes, collection of vocational outcomes, implementation by a dedicated multi-disciplinary team of mental health professionals and longer-term follow-up of participant outcomes.

The current study aimed to evaluate the effectiveness of EYM in improving cognitive skills and psychosocial outcomes relevant to employment and community engagement in individuals with SMIs.

Methods

Overview and Design

EYM was delivered by WISE Employment, a not-for-profit employment services provider in Victoria, Australia. The EYM team consists of three masters-qualified occupational therapists, a counsellor, Indigenous mental health worker, and a mental health peer support worker. Evaluation was conducted by St Vincent’s Hospital Mental Health Service. Recruitment took place between March 2019 and October 2020. The study was granted ethics approval from the Human Research Ethics Committee at St Vincent’s Hospital, Melbourne, and all procedures were in line with the Declaration of Helsinki. Informed written consent was obtained from all participants.

Participants

Participants were referred to the EYM program by mental health and other community support services. They were eligible to complete the program if they were over the age of 18, had a diagnosis of a SMI (including affective disorders, psychotic disorders, or PTSD), had an employment or other vocational goal, and were able and willing to participate in groups. Exclusion criteria were: (1) unable to attend sessions at the service and (2) lack of basic English literacy.

Program Outline

EYM was originally designed by Fife Employment Access Trust in Scotland. The six-month program involves one to two sessions per week, combining computer-based CR exercises, individual project work, group reflection/ discussion, bridging exercises, and a work experience placement (see Table 1 for program outline). Bridging exercises, as described by Medalia and Bowie [20], are ‘therapist-led verbal discussion and skills-training activities that help cognitive remediation participants apply what is learned on the restorative exercises to cognitive performance in everyday life’ (p. 66). The CR program used was Happy Neuron [12], a computer-based cognitive training module, which includes exercises on spatial memory, verbal memory, visual memory, executive function, visual and auditory attention, and processing speed. COVID-19 restrictions were introduced in Melbourne, Australia, while recruitment was underway, leading to transitioning of the delivery to online,this was readily effected and did not unduly disrupt program delivery.

Table 1

Outline of Employ Your Mind program phases and components

Phase	Components
Phase 1 One 90-min individual session/week for six weeks	Pre-intervention assessments (see Table 2) VIA Survey of Character Strengths [37] Introduction to program components Presentation and discussion of assessment outcomes Collaborative goal setting and formulation
Phase 2 Two 90-min sessions per week for six weeks	Two sessions per week: First half of session: group work exploring strengths, social cognition, bridging exercises, goals; second half of session: computer-based CR using Happy Neuron Project rationale, topic exploration and choice Group celebration of Phase 2 completion
Phase 3 Two 90-min sessions/week for six weeks	Two sessions per week: First half of session: group work exploring strengths, social cognition, bridging activities, goals; second half of session, CR via computer-based Happy Neuron Project presentation Preparation for work orientation placements Group celebration of Phase 3 completion
Phase 4 One 90-min session/week for six weeks	One session per week: Continuation of group activities supporting work orientation, CR Five work orientation sessions (up to 20 h) Completion of cognitive and psychosocial assessments. Sharing/explanation of final report Review of goals Group celebration of program completion

Table 2

Description of psychosocial outcome measures

Assessment	Description
Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) [34]	The WEMWBS assesses 14 items of positive mental wellbeing on a 5-point Likert-type scale ranging from 1 (none of the time) to 5 (all of the time), with higher scores indicating improved levels of psychological wellbeing
Work and Social Adjustment Scale (WSAS) [24]	The WSAS consists of 5 questions assessing individuals’ perspectives concerning impaired functioning, with responses made on a 0 to 8 scale. 0 indicates no impairment and 8 indicates very severe impairment. The total score is the sum of all responses, with a total maximum score of 40 and higher scores indicative of more severe impairment
Questionnaire about the Process of Recovery (QPR) [26]	The QPR is a 15-item measure developed from service users’ accounts of recovery from psychosis. Each item is scored on a 4-point scale and higher scores are indicative of greater recovery
Assessment of Quality of Life (AQoL-6D) [30]	The AQoL-6D consists of 20 items that assess six QoL dimensions and provides an overall QoL score; higher scores are indicative of better quality of life. The dimensions include: independent living, relationships, mental health, coping, pain and senses
Dialogue about Working Ability (DWA) [28]	The DWA consists of 34 items that assess an individual’s subjective ability to perform work-related activities in five areas: (1) self-awareness, interests and values, (2) roles and habits, (3) physical ability, (4) organisational and problem solving ability and (5) communication and interaction ability. The items are scored on a 5-point Likert scale from 0 (low level) to 4 (high level) of ability and the total score is calculated by summing all scores, with higher scores indicative of increased subjective ability to perform work-related activities

Outcome Measures

Basic demographic and clinical information were collected. Participants self-reported their primary mental health diagnosis, which were classified according to the International Classification of Diseases, 10th revision [39]. Assessments of psychosocial functioning (Table 2) and cognition (Table 3) were collected at commencement (baseline), completion, and at 12-month follow-up from program commencement. Vocational data were collected at program commencement and at 12-month follow-up from program commencement.

Table 3

Description of cognitive tests and neurocognitive domain

Test	Neurocognitive domain
Trail Making Test (Part A) (US [35], Brief Assessment of Cognition in Schizophrenia (BACS) symbol coding [15], Category Fluency Test (animal naming) [32]	Speed of Processing
Hopkins Verbal Learning Test, Revised (HVLT-R) [5]	Verbal Learning
Letter Number Span (LNS) [11],Wechsler Memory Scale-III, Spatial Span subtest (WMS-SS) [38]	Working Memory
Neuropsychological Assessment Battery (NAB) Mazes sub-test [33]	Reasoning and Problem Solving
Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT) [18]	Social Cognition

Psychosocial Outcome Measures

Table 2. details the psychosocial outcomes that were collected.

Cognitive Outcome Measures

Cognitive assessments were selected from the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery (MCCB) [27] for which normative data have been established [16]. Eight of the ten MCCB tests were included as sufficient to evaluate five key neurocognitive domains (see Table 3). The excluded tests were the Continuous Performance Test, Identical Pairs (CPT-IP) [8] and the Brief Visuospatial Memory Test, Revised (BVMT-R) [2]. This selection reduced assessment burden on participants and allowed for more extensive assessment of psychosocial outcomes.

Raw scores for each MCCB test and domain were converted to age and gender corrected t-scores using the MCCB scoring program. Testing was conducted by masters-level examiners who received extensive training in administration of the MCCB using video and group training sessions under the guidance of a psychiatrist. Quality assurance included periodic checks on MCCB administration and scoring practices.

Vocational Outcome Measures

Employment and other vocational data for the preceding 12-months were collected to provide an overview of activity over two time periods: the year prior- and the year post-EYM commencement. Participants were asked to provide details of (1) competitive paid employment, (2) informal paid employment (for example, babysitting), (3) voluntary unpaid work, and (4) study and educational engagement over the previous 12-months.

Statistical Analyses

Data analyses were performed using Statistical Package for the Social Sciences (IBM, SPSS Statistics Version 25). Descriptive and frequency analyses were used to explore demographic and clinical characteristics. Data were assessed for normality and paired t-tests were used to compare cognitive and psychosocial parameters between baseline and post-program for the participants who completed the EYM program. Between-group comparisons of vocational outcomes were conducted using Fisher’s exact test. For all analyses, Bonferroni corrections were applied to account for multiple comparisons and significance was set at p < 0.003 for psychosocial outcomes and p < 0.006 for cognitive outcomes. Missing values were excluded on a list-wise basis.

Results

Thirty-two participants commenced the EYM program. Table 4 provides demographic and clinical characteristics of the entire sample at baseline.

Table 4

Demographic and clinical characteristics

	N (%)
Total	32
Age (yrs; M ± SD)	40.03 ± 12.12
Gender
Male	18 (56.3%)
Female	14 (43.8%)
Country of birth
Australia	13 (40.6%)
Other	18 (56.3%)
Missing	1 (3.1%)
Ethnicity
Aboriginal or Torres Strait Islander	1 (3.1%)
Anglo-Celtic	17 (53.1%)
Asian	3 (9.4%)
African	3 (9.4%)
Other European	4 (12.5%)
Other	4 (12.5%)
Education
Secondary school	8 (25.0%)
Undergraduate	9 (28.1%)
Postgraduate	4 (12.5%)
TAFE	7 (21.9%)
Other	3 (9.4%)
Missing	1 (3.1%)
Current employment
Part-time	6 (18.8%)
Unemployed	21 (65.6%)
Other	4 (12.5%)
Missing	1 (3.1%)
Accommodation
Own house	5 (15.6%)
Board with friends/family	6 (18.8%)
Private rental	9 (28.1%)
Public housing	2 (6.3%)
Supported accommodation	4 (12.5%)
Other	2 (6.3%)
Missing	4 (12.5%)
Lives with
Alone	9 (28.1%)
Partner	3 (9.4%)
Family/friends	12 (37.4%)
Housemates/other boarders	3 (9.4%)
Other	1 (3.1%)
Missing	4 (12.5%)
Primary mental health diagnosis
Schizophrenia, schizotypal and delusional disorders	13 (40.6%)
Mood [affective] disorders	7 (21.9%)
Neurotic, stress-related and somatoform disorders	11 (34.4%)
Disorders of adult personality and behaviour	1 (3.1%)
Taking medication	27 (84.4%)
Community Support services linked
Clinical Mental Health service	24 (75.0%)
Community Mental Health service	15 (46.9%)
Disability Employment service	4 (12.5%)
Australian Disability Enterprise	1 (3.1%)
Other	11 (34.4%)

Twenty-one participants completed the four phases of the EYM program: 15 (71.4%) completed the program in person at WISE employment and 6 (28.6%) completed the program remotely via video teleconferencing due to COVID-19 restrictions. Of the 32 participants recruited, 11 (34.3%) withdrew prior to completion of the program: one (3.1%) due to mental ill health; one (3.1%) relocated interstate; two (6.25%) discontinued due to COVID-19; three (9.4%) were lost to follow-up, and four (12.5%) commenced employment or study. (Note: the two who discontinued due to COVID-19 rejoined the program when restrictions were relaxed but after completion of the evaluation).

Psychosocial Outcomes

Psychosocial and cognitive outcomes are reported for the 21 participants who completed the EYM program. Comparison of psychosocial outcomes between baseline and post-program are reported in Table 5.

Table 5

Psychosocial outcomes pre- and post-EYM program completion

	N	Baseline	Post-EYM program	Mean change	Cohen's d	Significance
		M ± SD	M ± SD	M (95%CI)		p
WEMWBS	18	40.33 ± 11.48	46.89 ± 10.19	6.56 (2.65, 10.46)	0.835	0.003*
AQoL-6D	19
Independent living		67.32 ± 14.34	75.68 ± 17.04	8.37 (2.68, 14.06)	0.709	0.006
Relationships		60.53 ± 18.40	66.84 ± 20.01	6.32 (.05, 12.58)	0.486	0.048
Mental Health		47.84 ± 23.68	67.16 ± 21.02	19.32 (9.81, 28.82)	0.979	< 0.0005*
Coping		45.16 ± 23.67	57.00 ± 18.71	11.84 (4.06, 19.63)	0.733	0.005
Pain		66.84 ± 28.88	78.42 ± 24.55	11.58 (0.47, 23.63)	0.463	0.059
Senses		80.63 ± 13.16	84.21 ± 14.08	3.58 (0.74, 7.89)	0.400	0.098
Total score		61.37 ± 14.63	71.63 ± 14.40	10.26 (5.87, 14.66)	1.125	< 0.0005*
QPR	16	34.50 ± 12.78	41.69 ± 9.09	7.19 (1.32, 13.05)	0.653	0.020
WSAS	20	20.75 ± 11.56	15.15 ± 8.04	-5.60 (-1.30, -9.90)	0.610	0.013
DWA	19
Self-awareness, values		33.74 ± 4.24	39.05 ± 3.21	5.31 (3.17, 7.46)	1.195	< 0.0005*
Roles & habits		33.26 ± 3.43	34.8 ± 3.06	1.58 (0.02, 3.18)	0.475	0.053
Physical ability		15.89 ± 3.05	17.12 ± 2.69	1.21 (0.01, 2.41)	0.487	0.048
Organisation & problem solving		21.37 ± 4.00	23.32 ± 4.12	1.95 (.06, 3.83)	0.498	0.043
Communication		26.58 ± 5.51	29.74 ± 4.38	3.16 (0.83, 5.48)	0.655	0.011
Total score		130.84 ± 15.29	144.05 ± 14.51	13.21 (5.80, 20.62)	0.859	0.001*

EYM, employ your mind; M, mean; SD, standard deviation; WEMWBS, Warwick–Edinburgh mental well-being scale; AQoL-6D, assessment of quality of life-6 dimension; QPR, questionnaire about the process of recovery; WSAS, work and social adjustment scale; DWA, dialogue about working ability

*: indicates significant difference between Baseline and Post-EYM program scores

Participants reported significant increases in positive mental wellbeing (WEMWBS), mental health QoL and overall QoL, as well as significantly increased self-awareness, interests and values, and enhanced overall perceived working ability (as assessed by the DWA). Improvements were also seen in additional subscales of the AQoL (independent living, relationships, coping) and DWA (physical ability, organisation and problem solving, communication), increased work and social adjustment, as well as increased self-reported recovery, but these did not remain significant after correction for multiple comparisons.

Cognitive Outcomes

Participants completed an average of 13.05 ± 10.66 computer-based CR hours over the course of the EYM program. Cognitive outcomes are reported below in Table 6 for the participants who completed the EYM program. Participants also engaged in group discussion, reflection, strategy coaching, and bridging exercises. It is important to note that while this additional activity is considered instrumental in facilitation of transfer to real-world functioning, the time spent engaged in these activities was not collected as part of the evaluation.

Table 6

Cognitive outcomes pre- and post-EYM program completion

	N	Baseline	Post-EYM program	Mean change	Cohen's d	Significance
		M ± SD	M ± SD	M (95%CI)		p
Speed of processing
TMT-A	12	47.17 ± 14.75	54.42 ± 17.88	7.25 (− 0.44, 14.94)	0.600	0.062
BACS	12	45.42 ± 9.29	48.58 ± 11.34	3.17 (− 3.17, 9.51)	0.317	0.295
Category fluency test	19	44.21 ± 10.05	53.95 ± 16.08	9.74 (3.34, 16.13)	0.734	0.005*
Verbal learning
HVLT-R	19	44.53 ± 10.57	50.26 ± 12.59	5.74 (0.24, 11.24)	0.503	0.042
Working memory
LNS	19	45.47 ± 15.17	52.89 ± 16.63	7.42 (1.69, 13.15)	0.624	0.014
WMS-SS	12	54.25 ± 7.40	57.42 ± 11.05	3.17 (− 4.21, 10.54)	0.273	0.365
Reasoning and problem solving
Mazes	12	50.42 ± 11.08	55.42 ± 9.35	5.00 (0.61, 9.39)	0.742	0.029
Social cognition
MSCEIT	19	62.89 ± 17.55	67.68 ± 13.75	4.79 (2.35, 9.34)	0.507	0.040

EYM, Employ your mind; M, mean; SD, standard deviation; TMT-A, trail making test (part A); BACS, brief assessment of cognition in schizophrenia HVLT-R, Hopkins verbal learning test, revised; LNS, letter number span (LNS); WMS-SS, Wechsler memory scale-III, spatial span subtest; MSCEIT, Mayer–Salovey–Caruso emotional intelligence test

*: indicates significant difference between Baseline and Post-EYM program scores

A significant increase in speed of processing was observed in the Category Fluency Test between baseline and post-EYM. There were also increases in verbal learning (HVLT-R), working memory (LNS), reasoning and problem solving (Mazes) and social cognition (MSCEIT), yet these did not remain significant after correction for multiple comparisons.

Vocational Outcomes

Fifteen participants completed assessment of vocational outcomes for the preceding year at baseline and 12-months following EYM commencement (see Table 7).

Table 7

Vocational outcomes for year preceding and year following EYM commencement

	Year prior to EYM commencementN (%)	Year following EYM commencementN (%)	Fisher's Exact p value
Vocational outcome			0.634
Competitive job	4 (26.6%)	8 (53.3%)
Informal job (eg, babysitting, gardening)	4 (26.6%)	2 (13.3%)
Volunteer role	4 (26.6%)	3 (20.0%)
Studying	3 (20.0%)	4 (26.6%)
No employment	4 (26.6%)	2 (13.3%)

There was no significant change in the proportion of participants engaged in vocational activities between the year prior to and year following EYM commencement. The proportion of participants in competitive employment doubled from four (26.6%) to eight (53.3%) in the year following EYM commencement.

Discussion

This study aimed to evaluate the effectiveness of the Employ Your Mind (EYM) program in improving cognitive skills and psychosocial outcomes relevant to employment and community engagement in individuals with SMIs. The evaluation incorporated the recommendations made in the pilot EYM evaluation [21] including the utilisation of a selection of tests from the MCCB, the inclusion of validated assessments of mental well-being and quality of life, collection of vocational outcomes and longer term follow up. In addition, the program was delivered by a team directly employed by WISE Employment, which allowed for closer adherence to the core features of CR as identified by Bowie et al. [4] through ongoing training, professional development and supervision. Under these more stringent conditions, our study confirmed the positive results of the pilot program evaluation.

There were significant improvements in category fluency—an assessment of speed of processing—and measures of mental well-being, quality of life, and perceived working ability, despite the small sample size and limited statistical power. Moreover, improvements in these outcomes demonstrated medium to large effect sizes. There were trends towards improvement across all psychosocial measures (mental wellbeing, quality of life, recovery, work and social adjustment, perceived working ability) and several cognitive domains including speed of processing, verbal learning, working memory, reasoning and problem solving, and social cognition. The trend towards more competitive employment—despite the impact of a strict and extended COVID-19 lockdown in Melbourne during 2020—is also encouraging. These results are consistent with increasing evidence that CR has valuable and sustained benefits for people with SMI with vocational goals [31, 36, 42].

It is noteworthy that the evaluation results were obtained with an average time of 13.05 ± 10.66 h engaged in computer-based CR (using the Happy Neuron program) over the course of the EYM intervention. This amount of time is significantly less than the recommended hours (20–40 h) previously reported in the literature [4]. The Expert Working Group on Cognitive Remediation advised that 'intensive training is ideal to produce meaningful effects' whilst acknowledging there are as yet no published comparative 'dosing' studies of CR [4, p. 51]. Dosage captured by the Happy Neuron program does not include the valuable time spent engaged in group and individual strategy coaching, bridging, and reflection/ discussion, all of which are key components of CR used in the EYM program. In subsequent evaluations, it will be important to document this time in more detail so that dosage can be more accurately reported. The length of intervention, however, is consistent with other CR programs. A meta-analysis by Wykes et al. [41] found the average length of treatment to be 32.2 h provided across 16.7 weeks. More recently, in their meta-analysis of CR for negative symptoms of schizophrenia, Cella et al. [6] identified effective programs provided 24 to 100 h of therapy time across periods ranging from 12 weeks to two years. EYM provides 63 h of individual and group intervention and up to 20 h of volunteer work experience over the course of six months. These results are consistent with other research that demonstrates improvement in cognitive and psychosocial functioning, when provided in the context of rehabilitation [19, 41]. Further investigation would establish if positive outcomes might be obtained with fewer hours of computer based CR activities integrated into vocational rehabilitation.

Limitations

A particular challenge was the facilitation of EYM during the COVID-19 lockdown restrictions in 2020, which were strict and long-lasting in Melbourne. With recruitment affected for six months and no face-to-face contact, program delivery and the ability of participants to gain employment were all impacted. This was mitigated by adaptation of the program to telepractice (using computer-based CR and videoconferencing). Despite the effect of workplace restrictions, it is notable that a number of participants were nevertheless able to find employment. Future evaluation of the program should include a larger sample size, controlled design, and investigation of the effect of dosage, particularly in the context of a vocational rehabilitation setting.

Conclusion

The results of this evaluation strengthen confidence in the value of the EYM program to improve cognitive and psychosocial outcomes related to vocational and community engagement. A larger trial to improve understanding and effectiveness of the program, especially relating to the CR dose–effect relationship, is warranted.