Effect of the National Stress Check Program on mental health among workers in Japan: A 1-year retrospective cohort study.

OBJECTIVES: This retrospective cohort study evaluated the impact of the Stress Check Program, a recently introduced national policy and program aimed at reducing psychological distress among Japanese workers.
METHODS: A baseline survey was conducted from November 2015 to February 2016, the period when Japan began enforcing the Stress Check Program. A one-year follow-up survey was conducted in December 2016. In the follow-up survey, two exposure variables were collected: having taken the annual stress survey, and experiencing an improvement in the psychosocial work environment. Psychological distress was assessed using the Brief Job Stress Questionnaire (BJSQ) at baseline and 1-year follow-up. The two exposure variables were used to define four groups: "Neither", "Stress survey (SS) only", "Psychosocial work environment improvement (WI) only", and "Both". BJSQ results were analyzed using repeated measures general linear modeling (GLM).
RESULTS: The study included 2,492 participants: 1,342 in the "Neither" group, 1,009 in the "SS only" group, 76 in the "WI only" group, and 65 in the "Both" group. Overall time-group interaction effects were not significant. The "Both" group showed significantly greater improvements in psychological distress than the "Neither" group (p = 0.02) at the 1-year follow-up, although the effect size was small (d = -0.14).
CONCLUSIONS: Combination of the annual stress survey and improvement in psychosocial work environment may have been effective in reducing psychological distress in workers, although the effect size was small.

Introduction

Occupational stress is considered a major risk factor for a wide range of health outcomes[1]-[5)]. Several policies and programs to prevent occupational stress and improve the psychosocial work environment have been proposed[6],[7)]. Countries such as the UK[8],[9)], several European countries[10)], and Canada[11)] have already developed and implemented national policies and programs to prevent occupational stress. However, there have been few epidemiological evaluations of these national policies and programs. One such study used national survey data to evaluate the effects of the Management Standard Approach implemented by the UK on the psychosocial work environment between 2004 (before the Management Standard was introduced) and 2010[12)], and found significant improvements in worker preparedness for change and managerial support.

Following passage of the Partial Amendment of the Industrial Safety and Health Act in 2014, a new occupational health policy called the Stress Check Program was started in Japan on Dec. 1, 2015. The Stress Check Program focuses on primary prevention of mental health problems rather than the screening of mental disorders, combined with efforts to improve the psychosocial work environment. The program has three main objectives: (a) to increase participants' awareness of psychological stress by providing an opportunity to participate in an annual stress survey; (b) to prevent stress-related diseases by providing a physician's interview to workers with high stress; and (c) to improve the psychosocial work environment based on group analysis of data collected by the stress surveys. Goals (a) and (c) focus on primary prevention, while goal (b) focuses on secondary prevention. The law mandates that the Stress Check Program be applied to all workplaces in Japan with 50 or more employees. Employers must provide employees a chance to participate in the stress survey and receive the results, and provide a physician's interview if requested by an employee with high stress. While not mandated, employers are also encouraged to analyze the data from the stress survey on a team basis, and improve the psychosocial work environment based on the analysis (e.g., reducing work hours, improving ways to work, and improving communication in the workplace). According to a report to the government in 2017, the Stress Check Program was implemented in 82.9% of the workplaces subject to the program[13)]; 78.0% of the employees in these workplaces participated in the stress survey; and 78.3% of the workplaces conducted a team-based analysis of the stress survey data. However, no study has investigated the effectiveness of the Stress Check Program in improving mental health (i.e., the psychological distress of Japanese workers).

The Stress Check Program includes two main components for the primary prevention of mental health problems among workers[14)]. The first is an annual stress survey which aims to decrease the risk of mental health problems in workers by increasing their awareness of their own stress through periodic surveys and feedback. The second is the analysis of group stress survey results to identify work-related stressors, followed by active efforts to improve the psychosocial work-environment. However, evidence for the effectiveness of the annual stress survey is weak. A previous randomized controlled trial (RCT) reported that providing workers with their stress survey results was not effective in reducing psychosocial stress[15)]. In addition, another cluster RCT also reported that online screening and personalized feedback on mental health were not effective in improving mental health among workers[16)]. In contrast, evidence based on well-designed intervention studies provides some support for the efficacy of activities to improve the psychosocial work-environment. Previous systematic reviews and a systematic meta-review reported that organization-level, participatory employee interventions designed to tackle a variety of psychosocial factors, as well as processes and procedures required to accomplish work tasks, was effective in improving workers' mental health in the workplace[17]-[19)]. For instance, a previous cluster RCT showed that participatory interventions for workplace improvements based on employee stress surveillance and workplace risk evaluations were effective in reducing psychological distress among workers[20)]. In addition, a non-randomized controlled trial reported that work environment interventions significantly improved depressive symptoms among workers in departments with high proportions of employee participation in the intervention workshop (i.e., 50% or more)[21)]. In addition, a previous cross-sectional study based on a nationally representative survey found that improvement of the work environment was associated with less stress at work[22)]. Furthermore, the previous cluster RCT also showed that a participatory psychosocial work environment improvement activity had a significant effect in improving work performance among workers[20)]. Nevertheless, another cluster RCT study failed to show that applying a participatory physical and psychosocial intervention to balance employee demands and resources improved work performance in the absence of stress surveillance[23)]. Based on this evidence, it seemed likely that feedback on the results of the stress survey would be ineffective in improving psychological distress and work performance, but that experiencing improvements in psychosocial work environment based on stress survey results might be effective in improving psychological distress and the work performance of participating employees.

To evaluate the impact of the newly introduced national Stress Check Program in Japan on improving psychological distress and work performance of participating workers, we performed a retrospective cohort study based on a large-scale cohort of workers surveyed before and one year after the implementation of the Program. We focused on two components of the Stress Check Program: participation in the stress survey and experiencing improvement in the psychosocial work environment. We analyzed this self-reported data to determine whether workers who only participated in the stress survey or workers who both participated in the survey and experienced improvement in the psychosocial work environment showed improvements in psychological distress and work performance compared to workers who did not participate in the Program.

Methods

Study design

This 1-year retrospective cohort study was conducted from 2015 to 2016. Baseline surveys were taken from November 2015 to February 2016, during the implementation of the Stress Check Program in Japan. A 1-year follow-up survey was conducted in December 2016. At the 1-year follow-up survey, exposure variables (i.e., implementation of the Stress Check Program in the workplace and improvement in the workplace environment) were collected. The Research Ethics Review Board of the University of Tokyo, Graduate School of Medicine (No. 10856) approved the study procedures. The aims and procedures of the study were explained on a webpage and consent was obtained from the respondent when he or she completed the baseline questionnaire. The study conformed to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist, which is used to improve the quality of reporting of observational studies[24)].

Participant recruitment

Participants were recruited from registered members of a web survey site, which has registered over one million members in Japan. Of these, a stratified random sample of 5,000 registrant workers were recruited in the baseline survey. Participants were sampled from eight strata according to two factors: gender (male and female) and age (18-29, 30-39, 40-49, and 50 or older). Inclusion criterion of the baseline survey were workers living in Japan. Exclusion criteria were workers who were (1) currently unemployed or (2) part-time employees. The site provides participants with 80 tokens (equivalent to 80 Japanese yen) for completing a web-based questionnaire on each occasion.

Exposure variables

Exposure to the annual stress survey

An original scale was used to ask about participants' experience of the annual stress survey. The scale included one item with response options of yes, no or unknown. Participants were asked about their exposure to the annual stress survey with the following question: "According to the Partial Amendment of the Industrial Safety and Health Act in 2014, every workplace with 50 or more employees is mandated to provide the employees with the opportunity of assessing their stress levels once a year after December 2015 in Japan. This is called the Stress Check Program. Have you received the stress survey in your workplace?" The respondents who answered yes were categorized into a "received" group and the respondents who answered no or unknown were categorized into a "not received" group. These data were only collected in the follow-up survey because the Stress Check Program had not been started at the time of the baseline survey. The scale has not yet been validated.

Exposure to psychosocial work environment improvement

An original scale was used to ask whether participants had experienced improvement in the psychosocial work environment based on the Stress Check Program. The scale included one item with yes, no or unknown options. Participants were asked whether they had experienced psychosocial work environment improvement activities with the following question: "Has a psychosocial work environment improvement activity been conducted in your workplace after the Stress Check Program?" The respondents who answered yes were categorized into a "conducted" group, and the respondents who answered no or unknown were categorized into "not conducted" group. These data were also collected only at the follow-up survey. The scale has not yet been validated.

Outcome variables

Psychological distress

Psychological distress was measured using the Brief Job Stress Questionnaire (BJSQ)[25)], comprising 15 items assessing irritation (3 items), fatigue (3 items), anxiety (3 items), and depression (6 items). All items were measured on a four-point Likert scale ranging from 1 (never) to 4 (almost always). The total possible scores range from 15 to 60. High scores indicate a high degree of perceived psychological distress. Acceptable reliability and validity was demonstrated in previous studies[25]-[27)]. These data were collected in the baseline and follow-up surveys.

Work performance

Work performance was assessed using one item from the WHO Health and Work Performance Questionnaire (HPQ)[28)]. Respondents were asked to rate their overall work performance during the past 4 weeks. The item was scored on an 11-point scale ranging from 0 (worst possible performance) to 10 (best possible performance). High scores indicate a high degree of perceived work performance. These data were collected in the baseline and follow-up surveys.

Potential confounders

The following potentially confounding covariates were collected: demographic characteristics including gender (male or female), age (18-29, 30-39, 40-49, or 50+in years), occupation (managers and professionals; clerical, sales, and service; or production), company size (less than 50, 50-99, 100-299, 300-999, 1000-4999, or 5000+), exposure to a voluntary stress survey provided by their company prior to enforcement of the Stress Check Program (no, yes, or unknown), and whether a job change had occurred in the previous year at the 1-year follow-up survey (no or yes).

Statistical Analysis

In this study, four groups were defined based on the two exposure variables which were reported in the 1-year follow-up survey. First was the "Neither" group, which consisted of participants who reported that they had experienced neither the annual stress survey nor the psychosocial work environment improvement activity. Second was the "Stress survey (SS) only" group, which consisted of participants who reported that they had only experienced the annual stress survey. Third was the "Psychosocial work environment improvement (WI) only" group, which consisted of participants who reported that they had only experienced a psychosocial work environment improvement activity. This group may have included participants who experienced psychosocial work environment improvement activities independent of the Stress Check Program, such as reduction of working hours based on the company's work-life balance policy. Fourth was the "Both" group, which consisted of participants who reported that they had experienced both the annual stress survey and a psychosocial work environment improvement activity.

For primary analysis, a general linear modeling (GLM) repeated measures test was conducted to estimate interaction effect of time (baseline and 1-year follow-up) × group (Neither, SS only, WI only, and Both). The analysis was adjusted to account for socio-demographic characteristics and potential confounders (gender, age, occupation, company size, a stress survey prior to baseline, and a job change in the past year). An a priori comparison was made between the "Neither" group with the other three groups ( "Both", "SS only", and "WI only" ). Statistical calculations were performed with SPSS Statistics 22.0 (SPSS Inc, Chicago, IL). The effect sizes and 95% confidence intervals (95% CIs) were calculated using Cohen's d. Values of 0.2, 0.5, and 0.8 are generally interpreted as being suggestive of small, medium, and large effects, respectively[29)].

Results

Participant Flowchart

Fig. 1 shows the participant flowchart in this study. Participants were recruited from registered members of a web survey site in Japan with 5,152 respondents. Of these, 1,261 were excluded because they were part-time employees. The remaining 3,891 participants were included in this study. The 1-year follow-up survey was completed by 2,581 (66.3%) of these participants. Of this group, 89 participants met the exclusion criteria and were excluded from the study (69 participants were unemployed, and 20 worked part-time). Therefore, 2,492 participants were included in the final analysis (64.0% of the baseline participants). Of the final participants, 1,342 were in the "neither" group, 1,009 were in the "SS only" group, 76 were in the "WI only" group, and 65 were in the "both" group. The reasons for dropping out of the study were not assessed.

Fig. 1.

Participant flow diagram

Participant Characteristics

Demographic characteristics are presented in Table 1. In all groups, more than half of the participants were in clerical or sales occupations. Participants in the "Neither" group tended to be 50 years of age or older (32.0%), in a company with fewer than 50 employees (44.9%), and to have not experienced a voluntary stress survey provided by their company prior to enforcement of the Stress Check Program (84.1%). In the "SS only" group, participants tended to be male (67.8%). In the "WI only" group, participants tended to be 40-49 years old (38.2%). In the "Both" group, participants tended to be male (83.1%), and to have already experienced a voluntary stress survey provided by their company prior to enforcement of the Stress Check Program (60.0%).

Table 1.

Baseline characteristics of participants in each of the four groups.

		Neither (n=1342)		SS only (n=1009)		WI only (n=76)		Both (n=65)
		n	%	n	%	n	%	n	%
Gender
	Male	783	58.3	684	67.8	47	61.8	54	83.1
	Female	559	41.7	325	32.2	29	38.2	11	16.9
Age
	18-29	225	16.8	177	17.5	9	11.8	11	16.9
	30-39	331	24.7	271	26.9	16	21.1	19	29.2
	40-49	357	26.6	308	30.5	29	38.2	20	30.8
	50+	429	32.0	253	25.1	22	28.9	15	23.1
Occupation
	Manger or professional	333	24.8	338	33.5	21	27.6	19	29.2
	Clerical or sales	742	55.3	514	50.9	41	53.9	33	50.8
	Service	27	2.0	20	2.0	3	3.9	1	1.5
	Production	239	17.8	136	13.5	11	14.5	12	18.5
	Unknown	1	0.1	1	0.1	0	0.0	0	0.0
Company size
	Less than 50	603	44.9	85	8.4	7	9.2	5	7.7
	50-99	164	12.2	110	10.9	8	10.5	5	7.7
	100-299	156	11.6	184	18.2	16	21.1	7	10.8
	300-999	142	10.6	195	19.3	6	7.9	17	26.2
	1000-4999	113	8.4	195	19.3	19	25.0	11	16.9
	5000+	134	10.0	220	21.8	16	21.1	17	26.2
	Unknown	30	2.2	20	2.0	4	5.3	3	4.6
Stress check before baseline
	No	1128	84.1	563	55.8	39	51.3	23	35.4
	Yes	184	13.7	426	42.2	33	43.4	39	60.0
	Unknown	30	2.2	20	2.0	4	5.3	3	4.6
Job change in the past year(collected at 1-year follow-up)
	No	1229	91.6	957	94.8	69	90.8	57	87.7
	Yes	113	8.4	52	5.2	7	9.2	8	12.3

Effects of the Stress Check Program

Table 2 shows the means and standard deviations of the outcome variables in each of the four groups at baseline and at the 1-year follow-up. Table 3 shows the estimated effects of the Stress Check Program on the outcome variables based on the GLM. The overall interaction effect was not significant for either outcome variable. However, the "Both" group showed significantly greater improvement of psychological distress compared with the "Neither" group (p = 0.02) at the 1-year follow-up, although the effect size was small (d = -0.14). However, the Stress Check Program had no significant effect on HPQ scores.

Table 2.

Average scores of psychological distress (BJSQ) and work performance (HPQ) at baseline and at the 1-year follow-up in each of the four groups.

	Psychological distress			Work performance
	Mean	SD		Mean	SD
“Neither” group (n=1342)
Baseline	31.3	10.6		5.8	1.8
1-year follow-up	31.3	10.5		5.8	1.9
“SS only” group (n=1009)
Baseline	31.7	10.3		5.9	1.9
1-year follow-up	31.4	10.5		5.8	1.8
“WI only” group (n=76)
Baseline	31.0	10.7		5.7	2.0
1-year follow-up	31.0	11.4		5.3	2.0
“Both” group (n=65)
Baseline	29.7	9.3		6.0	1.9
1-year follow-up	28.2	9.8		6.1	1.9

Table 3.

Effect of the Stress Check Program on psychological distress and work performance

	Pooled*			Contrast (vs. "Neither" group)
				"SC only" group					"WI only" group					"Both" group
	F	p		Estimate**	SE	p	Cohen's d		Estimate**	SE	p	Cohen's d		Estimate**	SE	p	Cohen's d
* GLM adjusted for gender, age, occupation, company size, stress check before baseline, and a job change within the past year.** Simple contrast estimate. SE, standard error.
Psychological distress	0.39	0.76		-0.05	0.42	0.91	-0.03		-0.41	1.07	0.70	0.00		-2.75	1.16	0.02	-0.14
Work performance	1.23	0.30		-0.01	0.07	0.86	0.01		-0.31	0.19	0.10	-0.18		0.23	0.21	0.27	0.09

Discussion

This retrospective study first examined the improvement in psychological distress in Japanese workers one year after enforcement of the Stress Check Program, a recently introduced policy and program intended to prevent occupational stress. This study is one of the few efforts made to assess the impact of a national policy and program for prevention of occupational stress on worker health[12)]. We found that the "Both" group experienced a significant improvement in psychological distress compared with the "Neither" group at the 1-year follow-up, with a small effect size. However, the "SS only" group failed to show significant improvement in psychological distress compared with the "Neither" group. No significant improvement of work performance was observed in any of the comparisons. Our results indicate that although the Stress Check Program as a whole was not effective in reducing worker psychological distress, combining the annual stress survey with improvement in the psychosocial work environment can effectively reduce psychological distress.

To our knowledge, this study is the first to demonstrate significant beneficial effects of combining the annual stress survey and psychosocial work environment improvement according to the Stress Check Program in Japan. Psychological distress was significantly improved in the "Both" group compared with the "Neither" group, consistent with previous systematic reviews[17]-[19)]. The improvement in the "WI only" group was not significant compared with the "Neither" group. This result is inconsistent with a previous national representative cross-sectional study that reported that improvement in the work environment effectively reduced the number of occupational stressors[22)]. Interventions to improve the psychosocial work environment were often conducted based on a prior assessment of psychosocial factors at work. The BJSQ[25)] recommended for the Stress Check Program[14)] measures several aspects of the psychosocial work environment (e.g., job demands and worksite support) in addition to stress reactions. The use of a stress survey that provides a more precise assessment of the psychosocial work environment may result in implementation of more effective workplace improvements, resulting in better outcomes. The findings may also be attributable to a psychological effect, in that workers may have a greater sense of ownership over the process of psychosocial work environment improvement if it is based on their own responses to the stress survey, which may enhance the outcome of the action. The results also indicate that work environment improvement activities were not effective if the participation rate of employees at the targeted worksite was low (less than 50%)[21)]. It is possible that work environment improvement activities conducted without a stress survey may have a less effective starting condition (i.e., a low participation rate). The improvement effect in the "SS only" group was not significant compared with the "Neither" group, which is also consistent with previous studies[15],[16],[19)]. A previous meta-review reported that screening was only effective in improving depression outcomes when followed by a specific treatment (i.e., cognitive behavioral intervention) or a systematic program (i.e., telephone outreach and care management)[19)]. Receiving their stress survey results in the absence of additional support may not lead workers to develop coping behaviors, and it may be necessary to provide employees with an appropriate response to the results of the stress survey (i.e., psychosocial work environment improvement activity).

In this study, none of the Stress Check Program components, or even the combination of the stress survey and work environment improvement, significantly improved work performance among participants. Only two well-designed intervention studies have examined the effect of participatory psychosocial work environment improvement activities on increasing work productivity or performance, and the results of these studies were inconsistent[20],[23)]. One cluster RCT reported that a participatory intervention for workplace improvements based on a stress surveillance and risk evaluation of their workplace was effective in improving job performance[20)]. The other cluster RCT reported that a participatory physical and psychosocial intervention to balance worker demands and resources without any stress surveillance was not effective[23)]. Notably, the former study used an intensive intervention program including pre-intervention stress surveillance for hazard identification, supervisor training, a worker-participatory planning workshop, and planned follow-up workshops to monitor the process of planned activities[20)]. The work environment improvement efforts in this study are likely to be less intensive, which may be a reason for the non-significant effect. In the "WI only" group, the work performance of participants actually deteriorated compared to the "Neither" group, although this decrease was not statistically significant. Participation in a work environment improvement activity may lead to an increase in additional work tasks or overtime work. It may also be difficult for participants to find and implement effective actions to improve their work environment without feedback from the stress survey results, which may lead to ineffective actions and decreased work performance. It remains unclear whether the Stress Check Program or any of its components is effective in improving work performance.

Limitations

This study has several limitations. First, the study used a retrospective design, and the exposure variables were collected in the follow-up survey; thus, a causal relationship cannot be accurately inferred. Additionally, the findings of this study may be biased by study design. Second, all participants were recruited from registered members of a web survey site in Japan. These participants may be more likely to be interested in social research and be willing to cooperate with the survey. Therefore, generalization of our findings to the working population is limited. Third, all outcomes were measured by self-reporting, which may be affected by the participants' perception or situation. For instance, participants may give questionnaire answers that are perceived to be socially desirable. Fourth, study participants had not been given a strict definition of a psychosocial work environment improvement activity based on the Stress Check Program. Therefore, participants may have had different understandings about what the psychosocial work environment improvement activity based on the Stress Check Program entailed. Fifth, 682 (27.4%) of the participants in this study had experienced a voluntary stress survey provided by their company before the enforcement of the Stress Check Program, and thus may have already experienced improved psychological distress or work performance. Therefore, the observed intervention effect for psychological distress and work performance may be somewhat underestimated, although this potential confounder was adjusted for using the GLM analysis. Sixth, this study did not assess the details of the participant's psychosocial work environment improvement activity, and the content and quality of the improvement activity may differ among participants.

Policy Implications

These findings have possible implications for the Stress Check Program. First, the study suggests that the combination of an annual stress survey and a psychosocial work environment improvement activity is effective in improving worker psychological distress. However, the work environment improvement following the Stress Check Program is not mandated. As seen in this study, the implementation of work environment improvement is still limited, while group analysis of the data from stress surveys is reported to be popular[13)]. To improve the effectiveness of the Stress Check Program on reducing the psychological distress of workers, psychosocial work environment improvement based on a group analysis of data collected by stress surveys should be more strongly encouraged. Second, this study provided a unique opportunity to evaluate the effect of a newly introduced national policy and program by establishing a cohort of workers before the introduction of the program using an Internet sample. If participants had not been assessed prior to implementation, it would be difficult to determine the effectiveness of the program. Therefore, this type of study may be a good model for effectively evaluating national policy and programs at low cost, while allowing that caution is required in generalizing study findings to all workers in the nation.

Acknowledgments: The present study was supported by the Health and Labor Sciences Research Grant 2015-2017 (H27-Rodo-Ippan-004) from the Ministry of Health, Labour, and Welfare, Japan. The sponsor of the study had no role in the study design, data collection, data analysis, data interpretation, or writing of the report. The authors had access to the data in the study and the final responsibility of submitting the paper.

Conflicts of interest: None declared.

Authors' contributions

Experiments were conceived and designed by KI, YA, KW, AT, AS, AI, HH, YO, TY, EY, and NK. Experiments were performed by KI, YA, and NK. Data was analyzed by KI and NK. Reagents, materials, and analysis tools were contributed by KI, YA, and NK. The paper was written by KI and NK. All authors reviewed the manuscript.