Improving work functioning and mental health of health care employees using an e-mental health approach to workers' health surveillance: pretest-posttest study.

BACKGROUND: Mental health complaints are quite common in health care employees and can have adverse effects on work functioning. The aim of this study was to evaluate an e-mental health (EMH) approach to workers' health surveillance (WHS) for nurses and allied health professionals. Using the waiting-list group of a previous randomized controlled trial with high dropout and low compliance to the intervention, we studied the pre- and posteffects of the EMH approach in a larger group of participants.
METHODS: We applied a pretest-posttest study design. The WHS consisted of online screening on impaired work functioning and mental health followed by online automatically generated personalized feedback, online tailored advice, and access to self-help EMH interventions. The effects on work functioning, stress, and work-related fatigue after 3 months were analyzed using paired t tests and effect sizes.
RESULTS: One hundred and twenty-eight nurses and allied health professionals participated at pretest as well as posttest. Significant improvements were found on work functioning (p = 0.01) and work-related fatigue (p < 0.01). Work functioning had relevantly improved in 30% of participants. A small meaningful effect on stress was found (Cohen d = .23) in the participants who had logged onto an EMH intervention (20%, n = 26).
CONCLUSION: The EMH approach to WHS improves the work functioning and mental health of nurses and allied health professionals. However, because we found small effects and participation in the offered EMH interventions was low, there is ample room for improvement.

RCT Entities:   Population Interventions Outcomes

Introduction

Working as a nurse involves dealing with a range of potential workplace stressors, such as psychological and emotional demands [1]. Unsurprisingly, mental health complaints including burnout, posttraumatic stress, anxiety, and depression are quite common in nurses [2]. Impaired mental health can have adverse effects endangering the health and safety of the nurses themselves but also of their patients [3,4]. Consequently, it seems worthwhile to pay preventive attention to the mental health and work functioning of nurses.

One promising preventive tool may be a mental module for workers' health surveillance (WHS). WHS can be used to identify and treat health complaints relevant to work and it should be an essential component of programs aimed at the protection of employees [5]. In The Netherlands, employers are obliged to periodically offer employees the opportunity to undergo a medical examination targeted toward preventing or limiting the risks for the employees' work-related health. A job-specific assessment should be applied to tailor the interventions to detected work functioning impairments [6]. Attention has been paid to the occupational hazards of health care employees, but WHS targeting work functioning as well as mental health of nurses and allied health professionals has not been studied in this way.

We have developed a self-help e-mental health (EMH) approach to a mental module for WHS, consisting of online screening on impaired work functioning and impaired mental health followed by personalized online feedback and online tailored advice combined with access to self-help EMH interventions. Online screening offers a practical and efficient method to screen for self-reported impaired work functioning and impaired mental health. Several EMH interventions are available as subsequent interventions. Self-help EMH interventions may offer some advantages over face-to-face health care, e.g., they can be followed in a self-chosen time and place and at one's own pace. Research has shown that unguided self-help EMH interventions have positive outcomes for a variety of mental health aspects [7,8]. However, their effects on work functioning have not been studied in a specific working population such as nurses and allied health professionals. Moreover, EMH interventions have thus far only been offered as stand-alone interventions for a specific common mental disorder. In our study, we offer a choice of EMH interventions, tailored to the specific complaints as indicated by the individual's screening results.

We have studied the EMH approach to WHS in a randomized controlled trial (RCT) [9]. Because we applied a waiting-list design for the control group (i.e., the control group was enrolled in the EMH approach to WHS after completion of the trial), we had the opportunity to study the pre- and posteffects of the EMH approach in a larger group of participants. For this study, we have focused on three outcomes: impaired work functioning, stress complaints, and work-related fatigue. We have chosen stress complaints and work-related fatigue as outcome measures because we consider them to form a generic and encompassing measure of mental health in health care employees. In this study, we addressed the following research question: Does our EMH approach to WHS improve work functioning, stress, and work-related fatigue in hospital nurses and allied health professionals?

Materials and methods

Study design and participants

The study participants originated from two study arms of a previous RCT performed in 2011 [10]. The study population for this trial was formed by all nurses, including surgical nurses and anesthetic nurses, and allied health professionals (such as physiotherapists and radiotherapists) employed at one academic hospital in The Netherlands. Because the trial regarded a preventive study, participants were included if they were not, or were not expecting to be, on sick leave for > 2 weeks at baseline. The Medical Ethics Committee of the Academic Medical Center Amsterdam, Amsterdam, The Netherlands approved the study. All participants gave their written informed consent prior to taking part. Participation rate of the RCT at baseline was 32%.

In the original EMH approach group, 178 participants filled out the baseline questionnaire and were offered the intervention. Participants from the control group who filled out the last follow-up questionnaire 6 months after baseline were offered the same intervention as the original EMH approach group. They were also asked if we could contact them again in future. Those who agreed (n = 117) were invited to complete an extra follow-up questionnaire 3 months after the last RCT follow-up questionnaire. The original EMH approach group and the original control group were then combined to study the effect of offering an EMH approach to WHS in a pre-post design. The scores that were used for this single-group pretest–posttest study are shown in Fig. 1. The outcome measures were assessed using online questionnaires.

Fig. 1

Schematic representation of the scores that were used as pretest and posttest scores. B, baseline; EMH, e-mental health; N, number of participants who filled out the pretest questionnaire and were invited for filling out the posttest questionnaire; T0, pretest scores; T1, posttest scores; 3 mo, follow-up after 3 months; 6 mo, follow-up after 6 months; 9 mo, follow-up after 9 months (only assessed in the original control group).

Thus, in total, 295 participants filled out the pretest questionnaire and were invited for the posttest questionnaire. Forty-three percent (n = 128) also filled out the posttest questionnaire.

Intervention

The intervention consisted of three parts. (1) Participants were screened on impaired work functioning (seven subscales) and impaired mental health (stress, work-related fatigue, risky drinking behavior, depression including suicide risk, anxiety including panic disorder, and posttraumatic stress disorder) using the pretest questionnaire (see Gärtner et al [10] for information on the instruments and cutoff points that were used). (2) All participants received automatically generated personalized feedback on screening results, onscreen and by e-mail. (3) The personalized feedback was followed by online tailored advice, consisting of an invitation to follow an EMH intervention and (if applicable) the receipt of an onscreen educational leaflet with advice per subscale on how to improve work functioning. In Table 1 an overview is given of the algorithm used for tailoring the advice.

Table 1

Algorithm used for tailoring the advice based on screening results

Screening on impaired work functioning	Screening on ≥ 1 mental health complaints	Tailored advice
Negative	Negative	Invitation to follow EMH intervention Psyfit.
Positive	Negative	Receipt of onscreen educational leaflet with advice to improve work functioning.Invitation to follow EMH intervention Psyfit.
Negative	Positive	Invitation to follow ≥1 EMH intervention(s); offer based on specific symptoms and work-relatedness of symptoms.∗
Positive	Positive	Receipt of onscreen educational leaflet with advice to improve work functioning.Invitation to follow ≥1 EMH intervention(s); offer based on specific symptoms and work-relatedness of symptoms.∗

EMH, e-mental health.

Participants were mostly offered a choice of 2 or 3 EMH interventions to leave room for personal preferences.

The EMH interventions that were used are self-help interventions on the Internet aimed at reducing specific mental health complaints or enhancing wellbeing: (1) Psyfit [11], aimed at enhancing mental fitness; (2) Strong at work [12], aimed at gaining insight into work stress and learning skills to cope with it; (3) Color Your Life [13], aimed at tackling depressive symptoms; (4) Don't Panic Online [14], aimed at reducing panic symptoms for subclinical and mild cases of panic disorder; and (5) Drinking Less [8], aimed at reducing risky drinking behavior.

The interventions are mainly based on the principles of cognitive behavioral therapy and combine a variety of aspects, e.g., providing information and advice, weekly assignments, and a forum to get in contact with others with similar complaints. The EMH interventions were developed as stand-alone interventions by the Trimbos Institute (Netherlands Institute of Mental Health and Addiction, Utrecht, The Netherlands) at an earlier stage. In separate trials, Psyfit, Colour your Life, and Drinking Less have had positive outcomes [7,8,15,16].

Few participants in the original EMH approach group had logged onto an EMH intervention [17]. We decided to slightly adapt the wording of the personalized feedback and the information about the offered EMH interventions for the original control group, to encourage following an EMH intervention.

Measures

Impaired work functioning was measured with the total score of the Nurses Work Functioning Questionnaire (NWFQ) [3], using the six final subscales [18]: cognitive aspects of task execution and general incidents, causing incidents at work, avoidance behavior, conflicts and irritations with colleagues, impaired contact with patients and their family, and lack of energy and motivation. The NWFQ has 47 items with a total standardized sum score range of 0–100.

Stress was measured with the distress subscale of the Four-Dimensional Symptoms Questionnaire (4DSQ) [19,20]. The 16-item questionnaire uses a five-point response scale (0 = no, 4 = very often) and has a total score range of 0–32.

Work-related fatigue after working time was measured using the need for recovery subscale of the Dutch Questionnaire on the Experience and Evaluation of Work (QEEW) [21,22]. The 11-item questionnaire with dichotomous response categories (yes, no) has a total score range of 0–11 and a standardized score range of 0–100.

For all outcomes, higher scores indicate a higher level of impairment.

Statistical analyses

All participants with a score on pre- and posttest were analyzed. We also performed subgroup analyses with participants who had logged onto an EMH intervention. Paired t tests were used to determine if there were statistically significant differences between scores on pretest and posttest. The significance level was set at α = 0.05.

Furthermore, we analyzed the size of the effects to determine their relevance. For impaired work functioning, we calculated the relative change scores of individuals at posttest compared to their pretest score. Individuals with a relative improvement on their NWFQ total score of ≥ 40%, which is the minimal important change (MIC) value of the NWFQ total scale [23], were defined as relevantly improved. For stress and work-related fatigue, we calculated Cohen d [24] by determining the mean difference between the pretest score and the posttest score, divided by the pooled standard deviation. For Cohen d, a score of 0.2–0.5 can be considered a small effect, 0.5–0.8 a medium effect, and > 0.8 a large effect [24].

All analyses were carried out using IBM Corp. IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp; 2010.

Results

In Table 2 [25-27], the participant characteristics are shown. Most participants were female and employed as a nurse, and had a permanent position at the hospital. Three quarters of the participants screened positive at pretest on impaired work functioning or impaired mental health, or both.

Table 2

The characteristics of the participants analyzed pretest

Descriptive	n	(%)	Mean	SD
Sex (N = 128)
Female	99	(77)
Age, y (N = 128)			40	12
Occupation (N = 128)
Nurse	89	(70)
Nurse practitioner	12	(9)
Allied health professional	27	(21)
Y of experience (N = 127)			11	11
Working hours per wk according to contract (N = 126)			31	6
Type of contract (N = 127)
Permanent position	118	(93)
Fixed-term contract	8	(6)
Other	1	(1)
Impaired work functioning (above cut-off; N = 128)∗
Work functioning impairments (red score on ≥ 1 subscales and/or orange score on ≥ 3 subscales) [10]	75	(59)
Impaired mental health (above cutoff; N = 128)
Impaired overall mental health (above cutoff of ≥ 1 of the 6 mental health aspects)	73	(57)
Stress, above cutoff (≥ 11) [25]	27	(21)
Work related fatigue, above cutoff (≥ 54.5) [26,27]	40	(31)
Screened positive on impaired work functioning∗ and/or impaired mental health	97	(76)

Work functioning is presented here including the subscale impaired decision-making, as it was included in the pretest screening [10].

All participants received online feedback on their personal screening results. The 75 participants (59%) who screened positive on impaired work functioning received the onscreen educational leaflet with advice on how to improve their work functioning. All participants were offered access to EMH interventions: 55 participants (43%) screened negative on impaired mental health and were offered access to Psyfit, whereas 73 participants (57%) screened positive and were offered access to EMH interventions tailored to their screening results. Twenty percent (n = 26) of the participants logged on at least once to an EMH intervention. Three of these participants logged on to two EMH interventions. Twenty-three participants logged on to Psyfit, four logged on to Strong at Work, and two logged on to Color Your Life. Nine participants followed an EMH intervention to some extent: eight partly followed Psyfit and one partly followed Color Your Life. No one completed an EMH intervention.

As shown in Table 3, the score on impaired work functioning decreased from 12 to 11 after the intervention of screening, personalized online feedback, and online advice plus access to EMH interventions (p = 0.01). Thirty percent of the participants had relevantly improved work functioning at posttest. When looking specifically at the subgroup of participants who had logged onto an EMH intervention, the decrease in score on impaired work functioning is also statistically significant (p = 0.04).

Table 3

Scores on pre- and posttest, mean difference, paired t test results, percentage whose work functioning improved with at least the minimal important change (% ≥MIC), and effect sizes

		Pretest		Posttest		Pretest–Posttest
	n*	M	SD	M	SD	Mean diff	t	p	≥MIC, n (%)	ES (95% CI)
Total group of participants
Impaired work functioning (NWFQ, 0–100)	128	12.1	8.85	10.6	8.78	1.4	2.67	0.008	38 (30)†
Stress (4DSQ, 0–32)	127	6.1	6.21	5.2	6.55	0.9	1.65	0.102		0.14 (−0.11–0.38)
Work-related fatigue (QEEW, 0–100)	126	34.8	29.41	29.4	30.79	5.4	3.02	0.003		0.18 (−0.07–0.43)
Complying subgroup‡
Impaired work functioning (NWFQ, 0–100)	26	13.0	9.37	11.3	9.64	1.8	2.13	0.043	7 (27)
Stress (4DSQ, 0–32)	26	6.8	6.69	5.3	5.90	1.5	1.12	0.273		0.23 (−0.31–0.78)
Work-related fatigue (QEEW, 0–100)	25	40.4	32.15	34.2	32.60	6.2	1.72	0.098		0.19 (−0.37–0.74)

4DSQ, Four-Dimensional Symptoms Questionnaire; CI, confidence interval; ES, effect size; M, mean; MIC, minimal important change; NWFQ, Nurses Work Functioning Questionnaire; QEEW, Dutch Questionnaire on the Experience and Evaluation of Work; SD, standard deviation.

The n varies due to missing values on the outcomes.

Total n = 125, because three participants had scored 0 on pretest and thus a relative improvement could not be calculated.

Including only participants who logged on at least once in an e-mental health intervention.

The score on stress decreased between pre- and posttest (from 6 to 5), but this difference was not statistically significant (p = 0.10). The effect size was < 0.2; thus, the effect does not seem meaningful. In the subgroup who logged onto an EMH intervention, the score regarding stress also decreased nonsignificantly (from 7 to 5, p = 0.27). However, in this subgroup the effect size was 0.23, indicating a small effect.

Regarding work-related fatigue, the score significantly decreased after the intervention (from 35 to 29, p < 0.01). In the subgroup of participants who logged onto an EMH intervention, the score also decreased, although nonsignificant. The effect size in both the total group and the subgroup was < 0.2.

Regarding all three outcomes, the subgroup of participants who had logged on at least once to an EMH intervention scored worse than the total group, both at pretest and at posttest. However, they also showed a larger improvement over time.

Discussion

A mental module for WHS, consisting of online screening on impaired work functioning and impaired mental health followed by personalized online feedback and online tailored advice combined with access to self-help EMH interventions, led to a statistically significant improvement of work functioning and work-related fatigue and had a small meaningful positive effect on stress in nurses and allied health professionals.

In this study, the outcomes of interest were work functioning and mental health of nurses and allied health professionals. The effects of WHS regarding these outcomes have not been studied before. This is an important approach, because ultimately occupational health care aims to keep employees functioning well and as healthy as possible.

We used the waiting-list control group of our previous RCT to enlarge the group of participants of the EMH approach to WHS, and especially the group of participants who had participated at pretest as well as posttest. By doing so, we were able to increase our certainty about the effects of the mental module for WHS on several outcomes. Furthermore, because we studied the effects in participants who had a score on pretest as well as posttest, our results are not obscured because of the intention-to-treat principle and thus show the maximum effect that the mental module for WHS has had on those employees who decided to participate in our study. From our previously performed RCT, we know that the control group (who had not yet received the intervention at that time) improved over time [9]. However, a reduction of complaints is not uncommon in RCT control groups and in our study could have stemmed from filling out questionnaires on work functioning and mental health, making people aware of their mental health state even if they did not receive feedback on screening results and subsequent interventions.

Because only few of the participants in the original EMH approach group logged onto an EMH intervention [17], we slightly adapted the wording of the personalized feedback and the information about the offered EMH interventions for the original control group, for encouragement to follow an EMH intervention. Using post hoc analyses, we did not find statistically significant differences between the original EMH approach group and the original control group on pre- and posttest.

All participants received personalized online feedback and an offer to follow one or more EMH interventions. We found a statistically significant improvement of work functioning, and the work functioning of 30% of the participants in this study had relevantly improved after 3 months. In addition, we found a statistically significant improvement of work-related fatigue, although the size of this effect did not seem meaningful because the effect size was < 0.2. Regarding stress, we also found a small, but not statistically significant, improvement over time. Therefore, it seems that the WHS has an effect on work-related outcomes, but not on the more general outcome of stress. Participants who screened positive on impaired work functioning not only received access to one or more EMH interventions, but also an onscreen educational leaflet with advice on how to improve their work functioning. Although we are not sure whether these participants read and applied the advice, it might have generated a positive effect on the work-related outcomes.

As stated earlier, the work functioning of 30% of the participants in this study had relevantly improved after 3 months, comparable to what we found in the EMH approach group (30%) and the control group (32%) in the RCT that we performed previously. After 6 months compared to baseline, this percentage was still comparable in the control group (30%), but a larger percentage of the EMH approach group (40%) had improved their work functioning [9]. It might take longer than 3 months to achieve a clinically relevant improvement in the majority of health care employees. It is conceivable that it takes time to apply advice to work activities and discuss it with a supervisor.

The subgroup of participants who had logged on at least once to an EMH intervention scored worse than the total group on all outcomes, both at pre- and posttest. However, they also showed a larger improvement over time, indicating that people who felt relatively worse or received online feedback indicating more impairments were more inclined to try the EMH intervention that they were offered and possibly benefitted more from it. However, the EMH interventions were not followed by many participants, because only 26 employees who fully participated in this pretest–posttest study logged onto an intervention. In this subgroup, we found a small positive effect on stress; although it was not statistically significant, it can be considered a meaningful effect. Therefore, to target stress, logging onto an EMH intervention seems a valuable addition to the mental module for WHS. However, the low compliance to the EMH interventions should be addressed. Several reasons may underlie the low compliance. First, because our study regarded a preventive setting, we think that the perceived need of our participants was insufficient to motivate them to log onto and follow an EMH intervention. This idea is supported by findings in other studies [28,29]. Second, some of the participants (n = 5) reported problems with logging onto the interventions due to technical problems and/or inadequate computer skills, which might have posed a problem for more participants. A third explanation might be that the channeling from the personalized online feedback toward the EMH interventions might not have been attractive enough to encourage participants to follow an EMH intervention. Finally, we offered several fitting EMH interventions to leave room for personal preferences. Most participants had to decide whether they wanted to follow an EMH intervention, but also determine which one to use. This might have made the option to not participate more attractive.

Although the EMH interventions were only followed by a small number of participants and only to a small extent, we did find positive results. It seems plausible that undergoing screening and receiving personalized online feedback plus tailored advice when needed form an intervention in itself, as it might encourage thoughts about one's own work functioning and mental health and to seek help if needed.

To ensure that health care employees stay mentally healthy and well-functioning, preventive strategies should be put in place to identify impaired work functioning or impaired mental health early and to offer tailored support. We recommend setting up and periodically performing WHS for nurses and allied health professionals. If available, occupational health services form a natural platform for WHS. To improve the mental module, we suggest exploring the possibilities of blended care by incorporating guidance by a health care provider, to increase compliance and to check whether the EMH intervention is of added value for the specific employee. Additionally, we recommend adapting the screening to a stepwise method, starting with a general and short screening. The health care provider could perform further specific screening (if needed) and provide feedback. Furthermore, more attention is needed to increase the attractiveness of the EMH interventions, e.g., by applying elements of persuasive design.

In summary, an EMH approach to WHS improves work functioning and mental health of nurses and allied health professionals. However, because effects were small and participation in the EMH interventions was low, there is ample room for improvement.

Conflicts of interest

The e-mental health interventions used in this study are stand-alone interventions which were developed by the Trimbos Institute at an earlier stage. However, neither the authors working at the Trimbos Institute (LB and OS) nor the institute itself derive financial income from the interventions. The other authors declare no conflicts of interest.