Effects of web-based stress and depression literacy intervention on improving work engagement among workers with low work engagement: An analysis of secondary outcome of a randomized controlled trial.

OBJECTIVE: The purpose of this randomized, controlled trial was to examine the effects of a psychoeducational information website on improving work engagement among individual workers with low work engagement, where work engagement was measured as a secondary outcome.
METHODS: Participants were recruited from registered members of a web survey site in Japan. Participants who fulfilled the eligibility criteria were randomly allocated to intervention or control groups. Immediately after the baseline survey, the intervention group was invited to study a psychoeducational website called the "UTSMed," which provided general mental health literacy and cognitive behavioral skills. Work engagement was assessed by using the Utrecht Work Engagement Scale at baseline, 1-, and 4-month follow-ups for both intervention and control groups. An exploratory analysis was conducted for a subgroup with low (lower than the median scores) work engagement scores at baseline.
RESULTS: A total of 1,236 workers completed the baseline survey. In the low work engagement subgroup, a total of 313 and 300 participants were allocated to an intervention and control group, respectively. In the high work engagement subgroup, 305 and 318 participants were allocated to an intervention and control group, respectively. The program showed a significant effect on work engagement (t = 1.98, P = 0.048) at the 4-month follow-up in the low work engagement subgroup, with a small effect size (d = 0.17).
CONCLUSION: A web-based psychoeducation resource of mental health literacy and cognitive behavioral skills may be effective for improving work engagement among individual workers with low work engagement.

RCT Entities:   Population Interventions Outcomes

Introduction

Recently, occupational health research has focused on positive health outcomes[1)]. One of the important positive health outcomes in the working population is work engagement, which is defined as a positive, fulfilling, work-related state of mind and is measured using three dimensions: vigor, dedication, and absorption[2],[3)]. Because it is known that work engagement is positively associated with quality of life of employees (i.e., job and family satisfaction) and productivity[4],[5)], work engagement has been recognized as an important health outcome in occupational health.

Although there is increasing research aimed to develop interventions to improve work engagement, most randomized, controlled trials (RCTs) failed to detect a significant intervention effect on increasing work engagement[4]-[7)]. For example, the RCTs that included lifestyle change programs such as a physical fitness program[4],[6)], mindfulness-based intervention[5)], or career management skill workshops[7)], failed to show a significant improvement in work engagement. On the other hand, other RCTs, which included changing the physical environment[8)] or online mindfulness-based intervention[9)], showed significant intervention effects for only absorption or vigor, but not dedication So far, only one RCT showed a significant intervention effect of an internet-based cognitive behavioral therapy (iCBT) program on improving work engagement[10)]. Theoretically, work engagement is affected by psychological resources such as self-efficacy, optimism, and positive perception regarding one's job[11],[12)]. Cognitive Behavioral Therapy (CBT) could improve those psychological resources[13)]. Thus, a CBT-based intervention may also be effective in improving work engagement.

However, major limitations exist in the dissemination of these CBT interventions: the programs require that professionals be well trained in CBT[14]-[16)]; time required for sessions, expensive cost for professionals, and stigma toward mental health care are other barriers to implementing a CBT program[17)]. A low intensity web-based intervention that is still effective in improving work engagement would be a more desirable alternative. As a less expensive, more easily administered, and potentially more accessible intervention than conventional psychological interventions, a web-based psychoeducational intervention could be such a desirable alternative[18],[19)]. In previous RCTs, a psychoeducational website for depression aiming at increasing knowledge of mental health and providing CBT skills was shown to be effective in reducing depressive mood among community residents with high levels of depression[20],[21)]. In these studies, the website was effective in improving depressive symptoms similar to an iCBT program[20],[21)]. A psychoeducational information website may be effective on increasing work engagement, as well.

It might be important to have workers with low work engagement as a primary target of work engagement interventions. Workers with low work engagement usually have poor mental and physical health, as well as poor job satisfaction and poor work performance[22]-[25)]. On the other hand, recent research has shown that workers with high work engagement may not receive greater health-promoting effect compared to these with its medium level[26],[27)]. A three-year cohort study reported a U-shaped association between work engagement and major depressive episode (MDE) among workers[27)]: the onset of MDE was greater in both groups with high and low work engagement, compared with a group with medium level of work engagement. The other previous one-year cohort study reported that workers with high and medium levels of work engagement showed similarly lower levels of high-sensitivity C-reactive protein (hs-CRP) levels, a risk factor for cardiovascular diseases among workers[26)]. Too high levels of work engagement may not be health promoting, or may even have an adverse effect on health. Thus, a primary aim of a work engagement intervention should be to increase work engagement among workers with low work engagement; it may not be needed to improve work engagement among workers who already have high work engagement.

The purpose of this RCT was to examine the effects of a psychoeducational information website program, originally developed for improving depression in individual workers[28)], on increasing work engagement among workers with low work engagement, where work engagement was measured as a secondary outcome. We also analyzed the effect of this website program among workers with high work engagement and compared these results with those of workers with low work engagement.

Methods

Trial design

Data for this study were collected in a previous RCT examining the effects of a web-based psychoeducation intervention on improving depressive symptoms among workers[28)]. The Research Ethics Review Board of the Graduate School of Medicine/Faculty of Medicine, the University of Tokyo approved the study procedures (no. 3083-1).

Participants

Participants were recruited from registered members of a web survey site, which registered over one million members in Japan. The inclusion criteria at the baseline survey were: (1) aged 20 to 60 years at the time of study entry and (2) currently employed. There were no exclusion criteria in the present study. The site provides participants with 30 tokens (equivalent to 30 Japanese yen) for completing a web-based questionnaire on each occasion. The aims and procedures of the study were fully explained on a webpage and consent was obtained from a respondent when he or she completed a baseline questionnaire.

Interventions

Participants in the intervention group were provided access to an information website for stress and depression named The University of Tokyo website for Stress Management and Education on Depression; UTSMed (http://mental.m.u-tokyo.ac.jp/utsmed/)[28)]. Briefly, the website consisted of about 90 pages, with around 800 Japanese characters per page. The UTSMed included psychoeducational information about depression (e.g., symptoms, diagnosis, treatment, and mechanisms of depression), stress (e.g., learning about stress and daily tips for coping with stress)[29)], and cognitive behavioral therapy (CBT) for self-help (e.g., cognitive restructuring skills, assertiveness skills, and problem-solving skills)[30]-[35)]. A self-help worksheet to practice each CBT skill was provided on the website. This website was accessible anywhere the internet is available, at no cost.

While the UTSMed was not specifically designed for increasing work engagement, this psychoeducational information website may be effective in increasing work engagement because work engagement is theoretically affected by psychological resources[11],[12)]. A CBT-based intervention may also be effective on improving work engagement. In this UTSMed program, a CBT component was developed to provide some special features, as follows: (1) Users can learn any part of the content they want at any time; (2) The content has been explained by using actual cases of workers (e.g., a depressed worker who has a problem with his work, boss, or interpersonal relation at work); (3) The CBT contents, especially cognitive restructuring and problem solving, were arranged so that a user can learn the basics of CBT step-by-step; (4) Examples of completed CBT worksheets were prepared to help users understand and work by themselves; and (5) Daily tips for coping with stress (e.g., sleep, alcohol, and exercise) were also prepared.

CBT components of the UTSMed

The UTSMed includes five components of cognitive behavioral skills: (1) self-case formulation based on the cognitive behavioral model and self-monitoring, (2) cognitive restructuring, (3) assertiveness training, (4) problem solving techniques, and (5) relaxation training. In component 1, the user learns about a cognitive behavioral (CB) model consisting of five parts: situations, thoughts, emotions, behaviors, and physical feelings[36)] and a self-case formulation based on this model. This component becomes the basis for the other four components. In component 2, the user learns about Beck's cognitive model and the cognitive restructuring skills based on this model. The model postulates that an individual's mood and behavior are affected by his/her automatic thoughts, which are shaped by dysfunctional schemas[30)]. The cognitive restructuring technique is one of the standard cognitive approaches of CBT and is utilized to change an automatic negative thought into an actual, realistic, and flexible thought[31)]. After a self-guided psychoeducational lesson about Beck's cognitive model, the user learns how to change an automatic negative thought into an actual thought through six steps. In steps 1 and 2, the user listens to a lecture on the cognitive ABC model (Activating/Actual event, Belief, and Consequence)[31)] and how to identify the automatic thoughts that cause a negative mood. Using a self-guided worksheet, the user selects a particular situation that triggers an automatic thought. In steps 3-5, the user tries to reconsider the rationale behind the automatic thought, seek alternative thinking, and replace the automatic thought with rational thinking. In step 6, the user can check the change in their mood compared with initial cognitive restructuring. In component 3, the user learns about assertiveness, which is typically defined as the legitimate and honest expression of one's personal rights, feelings, beliefs, and interests without violating or denying the rights of others[32)]. In order to communicate assertively, the Describe, Express, Specify, and Choose or Consequence (DESC) script is used[33)]. In component 4, the user learns about a problem-solving technique based on problem-solving therapy. Problem-solving therapy is a CB intervention that focuses on training adaptive problem-solving attitudes and skills[34)]. In component 5, the user learns relaxation techniques such as a breathing method, progressive muscle relaxation method, and autogenic training.

Intervention and control group conditions

Immediately after the baseline survey, the intervention group was invited to access UTSMed. They were allowed to access the UTSMed site within 4 months after the baseline survey. At 3 months after the baseline, they were reminded by e-mail to look at the UTSMed site. Participants in the control group were asked to complete the baseline and follow-up surveys. They were not invited to the UTSMed site nor sent a 3-month reminder. However, they were allowed to access any other website on stress and depression.

Outcome in this study

Work engagement was assessed using the short form of the Japanese version of the Utrecht Work Engagement Scale (UWES)[37)]. The UWES consists of three subscales (i.e., vigor, dedication, and absorption), comprising nine items in total. Items are scored on a 7-point scale ranging from zero (never) to six (always). A total score was measured using a web-based self-report questionnaire and calculated from all nine items at baseline, 1-month follow-up, and 4-month follow-up.

Demographic characteristics

Demographic data, such as age, gender, marital status, occupation, and education were collected at baseline survey.

Sample size

A minimum sample size of 587 in each group was determined to detect an effect size of 0.20 for depressive symptoms, at an alpha error rate of 0.05 (two-tailed) and a beta error rate of 0.20, with an expected dropout rate of 30%[28)]. A previous RCT to examine the effects of an iCBT program on improving work engagement among workers yielded a Cohen's d of 0.16 at 6-month follow-up[10)]. An estimated post hoc power (1-beta) was 0.38 if the effect size was 0.16, assuming that alpha was less than 0.05 (two-tailed), and 70% of the initial 300 respondents for each of the intervention and control groups for participants with low work engagement completed the follow-up, using the G*Power 3 program[38],[39)].

Randomization

Participants who fulfilled the inclusion criteria were randomly allocated to intervention or control groups, regardless of their baseline levels of work engagement. A simple randomization was conducted. Participants were randomized into the intervention and control groups upon entry using a computer-generated random number. Randomization procedures (i.e., sequence generation and allocation concealment mechanism) were conducted by the web survey company.

Statistical analysis

The present study aimed to examine the intervention effect among participants with low work engagement. A subgroup with low work engagement was defined as participants having a score of UWES at baseline lower than the median. An exploratory analysis was conducted for this subgroup, as well as for a subgroup with high work engagement, consisting of the remaining participants for comparison.

As a primary analysis, a mixed model for repeated measures conditional growth model analysis was conducted to estimate fixed effect of a group [ (intervention and control) × time (baseline, 1-month and 4-month follow-ups) ] interaction as an indicator of intervention effect. Intention-to-treat analysis was conducted. The level of significance was 0.05 (two-tailed). The MIXED procedure in SPSS Statistics 21.0 (IBM Corp., USA) was used.

As a sensitivity analysis, a mixed model for repeated measures analysis of variance model analysis (ANOVA) was conducted to estimate fixed effect of a group [ (intervention and control) × time (baseline and 1-month or 4-month follow-up) ] interaction as an indicator of intervention effect at each follow-up.

In addition, the effect sizes and the 95% confidence intervals (CI) for UWES were calculated using Cohen's d only among those who completed the questionnaire at baseline and at follow-up. The values of 0.2, 0.5, and 0.8 are generally interpreted as being suggestive of small, medium, and large effects, respectively[40)]. For comparison, Cohen's d was also calculated for depression measured by using Beck Depression Inventory-II (BDI-II).

As a process evaluation, rate (percentage) of reading each of the contents of the UTSMed was calculated among participants in the intervention group, including both low and high work engagement groups. Participants were asked "How many contents of the UTSMed did you read?" and were told to respond on a 5-point scale ranging from 0 (little) to 4 (all). Also for each content, participants were asked "Did you read the content of...," with a dichotomous response option of 0 (no) or 1 (yes).

Results

Recruitment

Recruitment and the baseline survey were conducted in November 2010. The recruitment stopped after the number of participants for each group exceeded 600. Both groups were surveyed with a web questionnaire at 1-month (December 2010) and 4-month follow-up (March 2011). The participant flowchart is shown in Fig. 1. A total of 1,236 workers completed a baseline survey. Participants were randomly allocated to an intervention or a control group (N = 618 for each). Each group was divided into two groups of participants with low levels of work engagement (2.56 or lower in scores of UWES) and high levels of work engagement (greater than 2.56 in scores of UWES).

Fig. 1.

participant flowchart

Baseline characteristics

Demographic characteristics of participants in the intervention and control groups stratified for subgroups with low and high levels of work engagement are presented in Table 1. In both subgroups, participants who were allocated in the control group tended to be male and married at a higher rate than those of the intervention group. In the whole sample, most participants were males, professionals or clerical, and received university or higher education.

Table 1.

Baseline characteristics of participants in intervention and control groups in each group.

		Low work engagement (UWES<2.56)						High work engagement (UWES≥2.56)
		Intervention (N=313)			Control (N=300)			Intervention (N=305)			Control (N=318)
		N/mean	(%/SD)		N/mean	(%/SD)		N/mean	(%/SD)		N/mean	(%/SD)
Note: UWES, Utrecht Work Engagement Scale.
Gender (men)		196	(62.6)		224	(74.7)		208	(68.2)		242	(76.1)
Age (years)		38.0	(8.5)		39.1	(8.5)		39.9	(9.4)		40.9	(8.8)
Marital status (married)		154	(49.2)		167	(55.7)		170	(55.7)		205	(64.5)
Occupation
	Manager	40	(12.8)		30	(10.0)		57	(18.7)		71	(22.3)
	Professional	86	(27.5)		98	(32.7)		76	(24.9)		80	(25.2)
	Clerical	111	(35.5)		86	(28.7)		90	(29.5)		81	(25.5)
	Production	37	(11.8)		36	(12.0)		38	(12.5)		43	(13.5)
	Sales	34	(10.9)		44	(12.7)		42	(13.8)		36	(11.3)
	Others	5	(1.6)		6	(2.0)		2	(0.7)		7	(2.2)
Education
	High school	80	(25.6)		71	(23.7)		74	(24.3)		87	(27.4)
	Some college	73	(23.3)		69	(23.0)		55	(18.0)		66	(20.8)
	University	142	(45.4)		140	(46.7)		161	(52.8)		147	(46.2)
	Graduate school	18	(5.8)		20	(6.7)		15	(4.9)		18	(5.7)
Work engagement (scored by UWES)		1.4	(0.7)		1.5	(0.7)		3.3	(0.6)		3.4	(0.8)

Effects of the intervention on work engagement

Table 2 shows the means and standard deviations of the UWES at baseline, 1-month, and 4-month follow-ups in the intervention and control groups separately for the subgroups with low and high work engagement. Table 3 shows the estimated intervention effects of the UTSMed on the UWES on the basis of the mixed-model analyses, also separately for the two subgroups. In the low work engagement subgroup, the UTSMed showed a significant pooled intervention effect on the UWES (t = 1.98, P = 0.048). In a sensitivity analysis, the UTSMed also showed a significant intervention effect on the UWES at 4-month follow-up, while the effect size was small and not significant (d = 0.17, 95% CI: -0.01 to 0.34). In the high work engagement subgroup, the UTSMed failed to show a significant intervention effect on the UWES. However, the effect size at 1-month follow-up was statistically significant (d = 0.17, 95% CI: 0.004 to 0.34) for this subgroup.

Table 2.

Average scores of work engagement at baseline, one- and four-month follow-up in each group.

	Low work engagement at baseline (UWES<2.56)								High work engagement at baseline (UWES≥2.56)
	Intervention				Control				Intervention				Control
	N	Mean	(SD)		N	Mean	(SD)		N	Mean	(SD)		N	Mean	(SD)
Note: UWES, Utrecht Work Engagement Scale; F/U, follow-up.
Baseline	313	1.4	(0.7)		300	1.5	(0.7)		305	3.3	(0.6)		318	3.4	(0.8)
1-month F/U	272	1.7	(0.9)		269	1.8	(0.9)		259	3.1	(0.9)		290	3.2	(0.9)
4-month F/U	241	1.9	(0.9)		259	1.8	(0.9)		240	3.1	(0.9)		280	3.2	(1.0)

Table 3.

Effect of the UTSMed on work engagement in each groups.

	Low work engagement at baseline (UWES<2.56)						High work engagement at baseline (UWES≥2.56)
	Estimates of fixed effects (95% CI)	t	p		Cohen's d (95% CI)		Estimates of fixed effects (95% CI)	t	p		Cohen's d (95% CI)
Note: UWES, Utrecht Work Engagement Scale.* A mixed-model for repeated measures ANOVA model analyses was conducted.** A mixed-model for repeated measures conditional growth model analyses was conducted.
1 months*	–0.01 (–0.13 to 0.11)	–0.19	0.85		–0.04 (–0.20 to 0.13)		0.12 (–0.01 to 0.25)	1.78	0.08		0.17 (0.004 to 0.34)
4 months*	0.15 (0.02 to 0.29)	2.27	0.02		0.17 (–0.01 to 0.34)		–0.01 (–0.15 to 0.14)	–0.07	0.94		0.00 (–0.17 to 0.17)
Pooled**	0.08 (0.0009 to 0.16)	1.98	0.048				0.00 (–0.08 to 0.08)	0.03	0.97

For depression as an outcome, in the low work engagement subgroup, changes of BDI-II score (standard deviation) from baseline to follow-ups were -1.0 (6.6) in the intervention group and -2.1 (6.3) in the control group at 1-month follow-up, and -2.5 (7.1) in the intervention group and -2.8 (7.5) in the control group at 4-month follow-up. The UTSMed showed a significant and weak adverse intervention effect (d = 0.17, 95% CI: 0.002 to 0.34) on the BDI-II at 1-month follow-up, while the effect was not significant at 4-month follow-up (d = 0.04, 95% CI: -0.14 to 0.21). In the high work engagement subgroup, changes of BDI-II score (standard deviation) from baseline to follow-ups were -1.1 (5.2) in the intervention group and -0.9 (5.8) in the control group at 1-month follow-up, and -1.7 (5.9) in the intervention group and -0.8 (5.8) in the control group at 4-month follow-up. The intervention effect was not significant at any of the follow-ups (d = -0.05, 95% CI: -0.22 to 0.12 at 1-month follow-up; d = -0.16, 95% CI: -0.33 to 0.02 at 4-month follow-up).

Process evaluation

Among respondents (N = 481) of the intervention group at the 4-month follow-up, 23.7% in the low work engagement subgroup and 27.1% in the high work engagement subgroup read all or most of the contents of the UTSMed; 37.3% in the low work engagement subgroup and 36.3% in the high work engagement subgroup read some of the contents. In more detail, in terms of the self-case formulation section, 54.4% of participants in the low work engagement subgroup and 59.2% in the high work engagement subgroup read the contents. In the cognitive restructuring section, 47.5% in the low work engagement subgroup and 50.0% in the high work engagement subgroup read the contents. In the assertiveness training section, 39.4% in the low work engagement subgroup and 45.8% in the high work engagement subgroup read the contents. In the problem solving section, 40.7% in the low work engagement subgroup and 45.5% in the high work engagement subgroup read the contents. In the relaxation training section, 46.9% in the low work engagement subgroup and 53.7% in the high work engagement subgroup read the contents. Log data when individual participants accessed the UTSMed were not collected.

Discussion

The present RCT examined the effects of a psychoeducation website, the UTSMed, for improving work engagement at 1- and 4-month follow-ups among workers grouped into low and high work engagement subgroups in Japan. For the low work engagement subgroup, the UTSMed showed a significant intervention effect on work engagement, but with a small effect size of 0.17 at the 4-month follow-up. For the high work engagement subgroup, the UTSMed showed a marginally statistically significant intervention effect size on work engagement only at the 1-month follow-up (Cohen's d = 0.17). Therefore, a web-based psychoeducation approach may be effective in improving work engagement for workers who have low work engagement.

To our knowledge, this is the first RCT that has demonstrated a positive effect of a psychoeducational information website on improving work engagement, specifically targeting workers with low work engagement. This result is consistent with a previous RCT that reported that an iCBT program was effective in improving work engagement among healthy workers[10)]. Cognitive behavioral therapy components of the UTSMed could be effective for improving work engagement, presumably by increasing psychological resources theoretically associated with work engagement[11],[41)]. While the effect size of UTSMed on work engagement was small (d = 0.17), the public health impact can be large, when considering the great accessibility and minimal cost of this kind of low-intensity, self-help, psychoeducational information intervention program. The program is particularly useful, because the effect was clearer for a subgroup with low work engagement, which is considered to enjoy the merit of improved work engagement without a concern for a possible adverse health effect of too much work engagement. In our previous report, we failed to show a significant intervention effect of UTSMed on improving depression in a group of participants who did not visit a clinic for their mental health problem[28)]. In the present study, the UTSMed did not show a significant effect on improving depressive symptoms in any subgroup; rather, the program slightly increased depression of the participants at 1-month follow-up in the subgroup with low work engagement, possibly because of increased awareness for stress[28)]. Thus, the effect of UTSMed on improving work engagement is less likely to be explained by improvement of depression. The mechanism of UTSMed improving work engagement may be independent of depression. However, in a previous study, it was reported that improving depression partly mediated improvement of work engagement in a six-session web-based CBT training[10)]. Interactions between these two outcomes (i.e., work engagement and depression) relating to a CBT program may depend on the intensity of the program and the target population.

The UTSMed showed an intervention effect on improving work engagement which gradually increased with time in the low work engagement subgroup. On the other hand, the high work engagement subgroup showed a significant and small increase in work engagement at 1-month follow-up (d = 0.17), which disappeared at 4-month follow-up. There was almost no difference in access to the contents of UTSMed during the follow-up between the subgroups with high and low work engagement. In the low work engagement subgroup, i.e. workers with lower vigor, dedication, and absorption state, participants may take more time to put knowledge into practice. In the high work engagement population, the UTSMed may have a short-term effect on maintaining work engagement, but the effect does not seem to remain for a longer period. A possible reason is a ceiling effect. It is possible that the UTSMed is too low intensity to improve work engagement among participants who already had high work engagement. A higher intensity intervention and/or an intensive follow-up may be needed to improve work engagement for a longer period among employees with high work engagement. Further research is required to determine if it is beneficial to improve work engagement among people with high work engagement.

Only about a quarter of the intervention participants were found to read content on the UTSMed. The low rates of reading the content may limit the intervention effect. In addition, there was almost no difference among the contents of the program associated with components of CBT in terms of the proportion being read by the intervention participants. Thus, from the current study, it was unclear which content component was more associated with improved work engagement. Further study is needed to examine the effectiveness of each of the components of CBT on improving work engagement.

Limitations

This study has several limitations. First, the present study did not use a stratified, permuted-block, randomization into the low or high levels of work engagement subgroups at baseline. Instead, participants were divided into two subgroups after a simple randomization. The sample may be biased between the intervention and the control groups in each of the two subgroups. Second, all participants were recruited from registered members of a web survey site in Japan. Participants were supposed to have experience using a PC. Workers who had extremely low levels of work engagement (i.e. workers who did not even have an energy to use PC) may not be included in this study. Therefore, generalization of the present findings to the working population is limited. Third, the intervention group had a greater rate of dropping out from the study. Dropouts may have caused an attrition bias, particularly if participants in the intervention group had higher levels of motivation and were more likely to browse the UTSMed.

Acknowledgments: We appreciate the help of the following persons in completing this project: Hiromi Tanigawa and Mitsuyo Kuroda.

Conflicts of interest: None.