Internet-based stress management for distressed managers: results from a randomised controlled trial.

OBJECTIVE: The aim of this randomised controlled trial (RCT) was to evaluate the efficacy of a guided internet-based stress management intervention (iSMI) among distressed managers compared with a attention control group (AC) with full access to treatment-as-usual.
METHOD: A total sample of 117 distressed managers, mainly employed in the healthcare, IT, communication and educational sector, were randomised to either iSMI (n=59) or an AC group (n=58). The iSMI consisted of eight modules including cognitive behavioural stress management and positive management techniques. Participants received a minimal and weekly guidance from a psychologist or master-level psychology student focusing on support, feedback and adherence to the intervention. Self-report data were assessed at pre, post and 6 months after the intervention. The primary outcome was perceived stress (Perceived Stress Scale-14). The secondary outcomes included mental and work-related health outcomes.
RESULTS: Participants in the iSMI intervention reported significantly less symptoms of perceived stress (d=0.74, 95% CI 0.30 to 1.19) and burnout (d=0.95, 95% CI 0.53 to 1.37) compared with controls, at postassessment. Significant medium-to-large effect sizes were also found for depression, insomnia and job satisfaction. Long-term effects (6 months) were seen on the mental health outcomes.
CONCLUSION: This is one of the first studies showing that iSMIs can be an effective, accessible and potentially time-effective approach of reducing stress and other mental-related and work-related health symptoms among distressed managers. Future studies are needed addressing distressed managers and the potential of indirect effects on employee stress and satisfaction at work. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

RCT Entities:   Population Interventions Outcomes

Work-related stress has become one of the major challenges of modern working life, yet access to treatment is limited.

Managers frequently report high work intensity, which may result in both individual and employee stress.

Meta-analysis has provided support for the efficacy of internet-based stress management interventions; however, few studies conducted addressing managers.

The present study provides further evidence of the short-term and long-term effects of internet-based stress management interventions.

Internet-based stress management interventions could be an effective, accessible and potentially time-effective approach for distressed managers.

Background

Work-related stress has become one of the major challenges of modern working life. Today, every fourth employee, within the European Union, has experienced work-related stress during most of their working time.1 Prolonged exposure to stress has been associated with severe health implications, for example, coronary disease, lowered immune functioning, inflammation, impaired memory, premature ageing of genes2–4 and contributes to the development of many psychiatric disorders, such as anxiety, depression and insomnia.5 6

In some occupational sectors the prevalence of stress is higher. These are typically found in occupations with high job demands/effort and low decision latitude/reward imbalance,7 8 for example, healthcare, social services, education and transportation. Managers frequently report high job demands and work intensity,9 10 which may result in chronic stress and even burnout. In a recent study,11 managers, in both private and public sectors, displayed high prevalence of insomnia (woman=23%; men=15%), exhaustion (woman=10%; men=5%) and depression (woman=5%; men=3%). The prevalence of insomnia, depression and burnout was nearly twice as common among female managers, compared with male managers. Furthermore, both male and female first-line and middle managers reported higher prevalence of insomnia, depression and burnout compared with top management and CEOs, were perceived autonomy are believed to be the mediating factor between different levels of management.11

Stress and ill health does not only affect managers themselves. Nyberg et al 12 13 have shown that stress and poor managerial leadership can affect the health (eg, ischaemic heart disease) of their subordinates, which calls for effective interventions addressing both stress management and health-promoting tactics. The use of systematic positive management tactics in occupational settings has been associated with improved employee performance,14 lower levels of stress and increased well-being in employees.15 However, to the best of our knowledge, no previous study has examined the efficacy of a combined stress management and positive management intervention for distressed managers.

During the last four decades, psychological interventions have been developed in order to increase the individual’s psychological resources and resilience to stress.16 Meta-analyses17 18 have provided support for the efficacy of stress management interventions (SMI) in working populations, and SMIs based on cognitive behavioural treatment (CBT) approaches have yielded the largest effect sizes (d=1.16). Despite the well-documented prevalence of stress and the efficacy of SMIs, only a small percentage of distressed employees actually receive interventions from health professionals.19 This clarifies the need to further develop and evaluate iSMIs that are accessible to the working population.

van der Hek and Plomp20 early addressed the need for effective SMIs that are comprehensive, coordinated, accessible, user-friendly, non-stigmatising, safe and cost-effective. Since the late 1990s, a growing body of literature supports the efficacy of internet-based cognitive behavioural interventions (iCBT), for example, depression, anxiety and insomnia.21 22 A major advantage of iCBT is that it is accessible and it has also been shown to be cost-effective.21 Since our first randomised controlled trial (RCT) on a internet-based SMI (iSMI) in 2003,23 a growing body of studies has evolved evaluating the efficacy of iSMIs. In a recent meta-analysis,24 including 26 studies (n=4226), for example, the mean effect size for stress was Cohen’s d=0.43 and small effects were found for depression (Cohen’s d=0.34) and anxiety (Cohen’s d=0.32). Moreover, subgroup analyses revealed that guided interventions (Cohen’s d=0.64) were more effective than unguided interventions, and medium-to-long interventions (5–8 weeks) showed to be more effective than shorter or longer ones, and interventions based on third wave CBT yielded the largest effect sizes. The results from the meta-analysis24 also indicate that iSMIs can have sustained effects on stress reduction up to 6 months.

However, to our knowledge, only one study has previously examined the efficacy of an iSMI targeting managers. Ly  et al 25 evaluated a iSMI smartphone application for middle managers at medium-sized and large-sized companies (>50 employees) in Sweden. The authors reported small-to-moderate effect sizes (Cohen’s d range 0.41–0.50) on perceived stress compared with a wait-list control group  after a 6-week programme.

The main purpose of this study was to evaluate the efficacy of iSMI programme in a self-referred sample of distressed managers. The primary objective was to test whether participating in the iSMI programme would lead to significant improvements on stress, and secondary mental health-related and work-related outcomes compared with attention control group (AC).

Method

Design

This study was a controlled trial in which 117 participants were randomised into one of two conditions, the iSMI or the AC group. Estimates of power and sample size were based on our previous RCT trials23 25 including n=63 and n=73 participants and with mean effects ranging from d=0.41–0.62. Sample size was set to 120 participants. This sample size was estimated with the aim of detecting an effect size of Cohen’s d=0.41 or higher. Self-reported outcome assessments were collected at pretreatment, post-treatment (8 weeks) and at a 6-month follow-up (figure 1). Participants who met the study criteria and provided informed consent were randomly allocated by an independent researcher using an internet-based random generator (www.randomizer.org). The independent researcher received a list of anonymous identification numbers of all participants (n=117) and coaches (n=6). All participants and coaches were randomised at the same time in a 1:1 proportion, allocating n=59 to the iSMI and n=58 to the AC group and 9–10 participants per coach and group. This procedure ensured that blinding was effected during randomisation. The Ethical Committee of Linkoping University, Sweden, approved all procedures involved in the study (reference number 2012/353–31).

Figure 1

Flow of study participants. AC, attention control; iSMI, internet-based stress management intervention; ITT, intention-to-treat; PSS-14, Perceived Stress Scale; MADRS-S, Montgomery Åsberg Depression Rating Scale-self-assessment; SMBQ, Shirom-Melamed Burnout Questionnaire.

Participants

Participants were first-line and middle managers who had volunteered for the project. To be eligible for the study, participants had to fulfil the criteria for an adjustment disorder described in the subdivision F43 Reaction to severe stress, and adjustment disorders of the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD-10),26 established via telephone interviews using the Mini International Neuropsychiatric Interview,27 additional criteria from ICD-1026 and national diagnostic guidelines.28 In addition to an adjustment disorder, participants had to fulfil the following criteria: (i) a minimum age of 18 years, (ii) mastering Swedish, (iii) have access to a computer or a tablet computer with internet access, (iv) currently be working as a first-line or middle manager and (v) not be on full-time sick leave. Mild to moderate forms of Diagnostic and Statistical Manual (DSM) axis-I diagnosis29 were accepted as comorbid conditions, as long as these were considered to be secondary to the primary adjustment disorder.

Participants were excluded from the study if they (i) recently had participated in a stress management programme, (ii) were CEO/top management, (iii) currently were suffering from bipolar disorder, psychosis, post-traumatic stress syndrome (PTSD), eating disorder, substance abuse, severe forms of depression, an anxiety disorder or a DSM axis-II personality disorders,29 (v) were showing suicidal ideation based on item 9 of the Montgomery Åsberg Depression Scale-Self Rated.30 Participants with scores of 3 or more were excluded and received information via email regarding appropriate help. Participants taking medication (eg, antidepressants or  sleep medication) were not excluded from the study but were requested to keep their medication constant during the study period.

Procedure

The study was conducted in a university setting, which included the researchers and the treatment platform hosted by the university. Participants were recruited through advertisement and articles in regional and national newspapers and labour organisation magazines. Detailed information and application to the study was presented on the project’s homepage (www.istress.se). After initial registration using personal email address, potential participants received an ID number and were asked to (i) provide written informed consent, (ii) complete online screening questionnaires (see  Primary and secondary outcome measures) and participate in a diagnostic interview over the telephone. After the interview, participants were randomised. Participants of the iSMI group received access to the iSMI immediately after randomisation and participants of the AC group received access to iSMI after the 6-month follow-up.

Study conditions

Intervention

The iSMI programme was based on our previous RCTs23 25 on iSMI. For the present iSMI, we added contemporary CBT techniques (including third wave CBT) such as behaviour activation, exposure, emotion regulation, values, acceptance and methods from recovery from work training,31 boundary tactics in the work-home interface32 and positive management.33

The programme was designed to facilitate change in three dimensions; the individual, the work-environment and the work-family interface. This was translated into five consecutive areas: (i) education, for example, stress responses, stress physiology, (ii) stressors, for example, action plan to reduce work-environmental stressors, (iii) balance, for example, applied relaxation, physical exercises, boundaries in the work-home interface, (iv) exposure, for example, assertiveness, perfectionism and worry and (v) positive management, for example, positive and corrective feedback. In addition to the main areas of the programme, optional modules were included with the aim of individualising the programme. As an example, participants with insomnia were encouraged to work on sleep management.

The iSMI consisted of 8 weekly modules. Each module contained text (about 10–15 regular pages), exercises, worksheets, images, examples, audio-files and video-files and homework exercises. The participants were informed that the programme would take about 2–3 hoursper week to complete.

The first week (introduction), the participants received information about the layout of the programme and they formulated their individual goals (eg, “I would like to be more assertive”). The first week also contained information and exercises on work-environmental stressors (eg, workload, pace, social support) and how to manage those stressors. The work on stressors continued until the fourth week of the programme. The last part of the first week covered information on stress physiology and stress symptoms and how to use a daily stress record.

The second week, the participants were introduced to different recovery techniques31 and applied relaxation,34 which continued throughout the programme.

The third and fourth week, participants did exercises related to the work-home interface and values-based action skills applied from Acceptance Commitment Therapy35 and behavioural activation.36 The participants were also introduced to the CBT principle exposure and how to apply it in a stress-related area, including assertiveness/boundaries, perfectionism, procrastination or excessive worry.

Between weeks 5 and 8, the work on balance and exposure continued. In addition to the main stress management areas, participants could choose to work with an optional chapter on sleep management, physical activity and/or time management. During the last 4 weeks, participants worked on positive management tactics or more specifically, how to apply positive reinforcement as a managerial skill.33 The modules on positive management were distributed over 3 weeks including (i) positive feedback, (ii) corrective feedback and (iii) negative feedback. During the final week (week 8), participants summarised their work on both stress management and positive management in a maintenance plan.

Participants in the iSMI received weekly personalised written feedback via email from a coach on the exercises they had completed. The coaches were psychologists and supervised master-level psychology students who were trained to perform feedback according to a standardised manual. The aim of the feedback was to provide support and encouragement, monitor homework assignments and adherence to the intervention. iSMI group had full access to treatment-as-usual.

Attention control group

To limit the risk of attentional effects, which has been debated in the use of wait-list control groups,37 an AC group was developed to provide a similar attentional focus (eg, weekly mail contact, homework assignment) as to the iSMI group. Using an AC is considered to be a more conservative approach.37 Participants assigned to the AC group were given a brief weekly text module (half a regular page), via the web platform, about a stress-related topic (eg, stressors, recuperation or feedback). They were then invited to discuss each topic, anonymously, with other participants in a moderated web forum. Compared with the iSMI, the AC was a passive learning experience and did not involve any behavioural modification, worksheets or exercises. Equivalent to the iSMI group, participants in the AC group received weekly homework assignments, consisting of statements about stress. When submitted, participants in the AC group received minimal and standardised feedback from their coach (eg, “Thank you for your feedback! The next text module and topic for discussion are now available.”). AC group had full access to treatment-as-usual.

Primary outcome measure

Perceived stress scale

Perceived stress was measured with the 14-item version of the Perceived Stress Scale (PSS-14),38 translated into Swedish.39 The PSS-14 is designed to measure the degree to which situations in one’s life are appraised as stressful. The Swedish version of PSS has an internal consistency reliability (Cronbach’s alpha) of 0.82 and split-half reliability estimate of 0.84.39

Secondary outcome measures

Mental health

Shirom-melamed Burnout Questionnaire

The Shirom-Melamed Burnout Questionnaire (SMBQ)40 41 is a 22-item scale (graded 1–7) used to assess different aspects of chronic stress and burnout syndrome (physical fatigue, cognitive weariness, tension and listlessness). This scale correlates significantly42 with other well-established questionnaires measuring burnout, for example, Maslach Burnout Inventory.43 The SMBQ has an internal consistency reliability (Cronbach’s alpha) of 0.92.40

Montgomery Åsberg Depression Rating Scale

We used the Montgomery Åsberg Depression Rating Scale-self-assessment, MADRS-S44 to measure symptoms of depression. MADRS-S consists of nine items measuring different symptoms of depression and each symptom is rated on a 6-point scale. The instrument has good reliability45 and has been validated as an internet-based measure.46 In a comparative study,30 the MADRS-S correlated highly (r=0.87) with the Beck Depression Inventory,47 indicating acceptable convergent validity.

Insomnia Severity Index

The Insomnia Severity Index (ISI)48 is a seven-item self-report questionnaire that measures individuals perceptions of their insomnia and the severity of problems with delayed sleep onset, sleep maintenance and early morning awakenings, as well as the level of satisfaction with current sleep patterns, interference with daily functioning and worry or SMI related to the sleep problem. ISI exhibits adequate internal consistency measures (Cronbach’s alpha=0.74), and is a sensitive measure to detect changes in perceived sleep difficulties.48 It has previously been validated as an internet-based measure.49

Alcohol Use Disorders Identification Test

The Alcohol Use Disorders Identification Test (AUDIT)50 was used to assess potential alcohol dependence or abuse. The AUDIT was only collected at baseline for screening purposes. In a study of the psychometric properties of the Swedish version of the AUDIT, both the internal and test–retest reliabilities were satisfactory.51

Work-related health

Work Experience Measurement Scale

The Work Experience Measurement Scale (WEMS) is an instrument measuring the experience of work from a health resource perspective.52 WEMS consists of 32 items measuring job satisfaction in five different domains (supportive work conditions, internal work experience, autonomy, time experience, management, process of change) on a 6-point scale. Cronbach’s alpha on the WEMS has been reported to be in the interval of 0.85–0.96.52

Absenteeism, presenteeism and healthcare consumption

Absence from work, presenteeism and healthcare consumption was measured according to the Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry (TiC-P).53 TiC-P has been used in several studies for economic evaluations of healthcare consumption and productivity loss in mental health.53 Absenteeism and presenteeism were both measured at baseline as the self-rated amount of days during the past month absent or present at work while being physically or mentally ill. Healthcare consumption was calculated by the number of visits to healthcare professionals (GP, occupational healthcare, psychologist, etc) during the last month.

Additional measures

Utility (content) and user-friendliness (access, layout) of each module was assessed using two self-design questions, answered on a 7-point scale (1=low satisfaction; 7=high satisfaction).

Statistical analysis

All analysis followed the Consolidated Standards of Reporting Trials statement for non-pharmacological trials. Statistical analyses were conducted following the intention-to-treat principle (ITT) using SPSS V.22 (IBM, Armonk, New York, USA). Reported p values are two-sided with a significance level of 0.05. We used the multiple imputation (MI) procedure to impute missing sum scores for participants who did not complete the post or 6-month follow-up assessments. MI is considered to be a conservative approach for analysing incomplete datasets as it takes into account the uncertainty due to missing information.54 In MI, predictors are defined and used for estimations of the missing values. We used all pre, post and 6-month follow-up values of all outcome measures as well as age, gender and educational level as predictors. Means, SD and SE of effect sizes were pooled from five sets of imputations.

The effects of group on primary and secondary outcome measures of the ITT and completers-only datasets were analysed using analysis of covariance (ANCOVA), with pre-assessment values as covariate. We report p values and Cohen’s d for the within-group and between-group effect size and corresponding 95% CI. According to Cohen, d=0.2 can be considered a small effect, d=0.5 a medium and d=0.8 a large effect.

Results

As depicted in figure 1, 37 participants were excluded, most because of a score of <25 on the PSS-14 and <2.75 on the SMBQ. A total of 117 participants were randomised to either the iSMI (n=59) or the AC group (n=58). A total of 93 participants (79%; iSMI n=42; AC n=51) responded to the postassessment questionnaires and 82 participants (70%; iSMI n=38; AC n=44) completed data in the 3 months follow-up measurement. No statistical significant differences were found on any baseline measures or characteristics, between participants who completed all questionnaires and participants who failed to return one or more questionnaires.

Baseline characteristics

Table 1 presents the baseline characteristics of the study participants. On average, the participating managers were working 7.1 hours (SD=4.9) of overtime per week, had 8.26 years (SD=7.6) of experience as a manager and 20.4 subordinates (SD=22.0). Thirty-four participants (29%) had previous experience of any SMI or psychotherapy. Only one participant was on sick leave (part-time).

Table 1

Baseline characteristics

Characteristics	All participants (n=117)	iSMI (n=59)	AC (n=58)
Sociodemographic characteristics
Age (years), M (SD)	46.9 (8.3)	46.9 (8.5)	46.8 (8.2)
Gender, female, n (%)	78 (67)	42 (72)	36 (63)
Married or cohabiting, n (%)	105 (90)	52 (89)	53 (90)
University level education, n (%)	106 (91)	55 (94)	51 (88)
Work characteristics
Working hours per week, M (SD)	39.1 (2.5)	38.8 (3.0)	39.3 (2.0)
Hours of overtime per week, M (SD)	7.1 (4.9)	7.0 (4.2)	7.2 (5.5)
Years working as a manager, M (SD)	8.26 (7.6)	8.41 (7.7)	8.10 (7.6)
Number of subordinates, M (SD)	20.4 (22.0)	18.5 (4.8)	22.4 (27.5)
On sick leave (part-time), n	1	0	1
Work sectors, n (%)
Health and social	39 (33)	21 (35)	18 (31)
IT and communication	16 (14)	7 (12)	9 (15)
Educational	15 (13)	9 (15)	6 (10)
Wholesale and economy	11 (10)	5 (9)	6 (11)
Manufacturing	9 (8)	3 (6)	6 (10)
Public administration	9 (7)	5 (8)	4 (7)
Others	18 (15)	9 (15)	9 (16)
Disorder and comorbid characteristics, n (%)
F43.2 Adjustment disorder	74 (63)	34 (58)	40 (69)
F43.8 Other reactions to severe stress	25 (21)	14 (24)	11 (19)
F43.9 Reaction to severe stress, unspecified	18 (15)	11 (19)	7 (12)
F51.0 Non-organic insomnia	67 (57)	40 (68)	27 (47)
F41.1 Generalised anxiety disorder	10 (9)	2 (4)	8 (14)
F33.x Recurrent depressive disorder	10 (9)	3 (5)	7 (12)
F41.0 Panic disorder	9 (8)	6 (10)	3 (5)
F32.x Depressive episode/depression	8 (7)	6 (10)	2 (3)
F40.1 Social phobia	4 (3)	3 (5)	1 (2)
Non-comorbid	41 (35)	16 (27)	25 (43)
Previous experience, n (%)
Previous SMI or psychotherapy	34 (29)	20 (34)	14 (24)
First-time help seeker	83 (71)	39 (66)	44 (76)

AC, attention control group; IT, information technology; M, mean; n, number; SMI, internet-based stress management intervention.

Changes in perceived stress

The iSMI group showed lower scores on the primary outcome PSS-14 at post-treatment (T2; F=12.70, p<0.001). A large effect size was observed (d=0.74; 95% CI 0.30 to 1.19). Table 2 shows the means and SD for all outcome measures separately for pretreatment, post-treatment and 6 months follow-up (T3). Table 3 displays the results of the ANCOVA for all outcome measures separately for T2 and T3.

Table 2

Means and SDs for the ITT sample (iSMI: n=59; AC: n=58) at pretreatment (T1) and post-treatment (T2), and 6 months follow-up (T3)

Outcome	T1				T2*				T3*
	iSMI		AC		iSMI		AC		iSMI		AC
	M	SD	M	SD	M	SD	M	SD	M	SD	M	SD
Primary outcome
PSS-14 (0–56)	33.22	5.27	33.09	5.34	24.39	5.95	29.11	6.70	22.52	7.22	26.61	6.80
Mental health
SMBQ (1–7)	4.47	0.88	4.46	0.77	3.27	0.95	4.16	0.94	3.42	1.16	3.87	1.18
Physical fatigue	4.25	0.96	4.23	0.96	3.22	1.08	3.87	1.12	3.22	1.31	3.61	1.27
Cognitive weariness	4.61	1.06	4.53	0.98	3.88	1.23	4.28	1.21	3.67	1.25	3.76	1.12
Tension	5.04	1.07	4.91	0.85	4.11	1.24	4.53	1.09	3.87	1.27	4.27	1.28
Listlessness	4.28	1.35	4.44	1.20	3.31	1.43	4.06	1.22	3.05	1.42	3.77	1.50
MADRS-S (0–54)	15.71	6.35	15.76	4.99	9.49	5.11	14.64	6.78	10.64	7.17	12.59	6.57
ISI (0–28)	11.97	4.87	11.10	5.54	8.26	4.78	9.95	5.04	8.37	4.37	10.35	5.84
Work related
WEMS (32–192)†	123.66	20.41	124.47	18.89	136.68	19.56	124.61	21.45	128.60	23.12	126.20	21.12
Supportive work conditions (1–6)†	4.02	0.83	4.04	0.71	4.38	0.75	4.15	0.79	4.23	0.95	4.27	0.79
Internal work experience (1–6)†	4.76	0.78	4.71	0.88	4.98	0.74	4.68	0.90	4.84	0.93	4.80	0.94
Autonomy (1–6)†	4.03	1.06	4.02	0.94	4.19	1.07	4.30	0.80	4.18	0.88	4.26	0.70
Time experience (1–6)†	2.15	1.10	2.11	1.00	2.59	1.28	2.39	1.22	2.61	1.25	2.61	1.22
Management (1–6)†	3.64	1.01	3.87	0.90	4.05	0.93	3.65	1.07	3.78	1.12	3.72	1.00
Process of change (1–6)†	3.76	1.27	3.71	1.15	4.23	1.17	3.67	1.25	3.89	1.22	3.74	1.27
Absenteeism (TiC-P)‡	2.00	1.15	3.81	2.98	4.43	3.69	2.70	1.95	3.50	2.32	4.25	0.96
Presenteeism (TiC-P)‡	4.54	3.75	4.35	4.20	4.04	2.18	5.26	5.09	4.35	3.44	5.57	5.21
AUDIT (0–40)	3.42	2.21	3.81	2.53	–	–	–	–	–	–	–	–

*Missing data imputed by multiple imputation.

†Higher scores indicate better outcomes.

‡In relation to the previous month.

AC, attention control; PSS-14, Perceived Stress Scale; SMBQ, Shirom-Melamed Burnout Questionnaire; MADRS-S, Montgomery Åsberg Depression Rating Scale-self-assessment; ISI, Insomnia Severity Index; ITT, intention to treat; iSMI, internet-based stress management intervention; WEMS, Work Experience Measurement Scale; AUDIT, Alcohol Use Disorders Identification Test; TiC-P, Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry.

Table 3

Results of the ANCOVAs and Cohen’s d for the primary and secondary outcome measures (ITT sample) at post-treatment and 6 months follow-up

Outcome	T2† between-groups effect		T3† between-groups effect
	Cohen’s d (95% CI)	ANCOVA‡ F(1,114)	Cohen’s d (95% CI)	ANCOVA‡ F(1,114)
Primary outcome
PSS-14 (0–56)	0.74 (0.30 to 1.19)	12.70***	0.59 (0.14 to 1.04)	2.17**
Mental health
SMBQ (1–7)	0.95 (0.53 to 1.37)	23.71***	0.39 (0.21 to 0.99)	1.44
Physical fatigue	0.59 (0.17 to 1.01)	8.36**	0.30 (0.14 to 0.75)	2.47
Cognitive weariness	0.57 (0.15 to 0.99)	7.04**	0.50 (0.05 to 0.94)	4.71*
Tension	0.36 (0.05 to 0.78)	5.02*	0.31 (0.13 to 0.76)	3.88
Listlessness	0.33 (0.08 to 0.74)	2.96	0.08 (0.36 to 0.52)	0.98
MADRS-S (0–54)	0.86 (0.25 to 1.46)	9.70*	0.29 (0.16 to 0.74)	1.74
ISI (0–28)	0.34 (0.07 to 0.76)	4.85*	0.39 (0.01 to 0.78)	6.17*
Work related
WEMS (32–192)§	0.59 (1.01 to 0.17)	11.37***	0.11 (0.56 to 0.34)	0.31
Supportive work conditions (1–6)§	0.30 (0.72 to 0.13)	4.45*	0.05 (0.44 to 0.53)	0.04
Internal work experience (1–6)§	0.36 (0.78 to 0.07)	5.77*	0.04 (0.52 to 0.44)	0.15
Autonomy (1–6)§	0.12 (0.31 to 0.54)	0.23	0.10 (0.38 to 0.58)	0.60
Time experience (1–6)§	0.16 (0.58 to 0.27)	0.86	0.00 (0.48 to 0.48)	2.58
Management (1–6)§	0.40 (0.83 to 0.02)	10.51**	0.05 (0.54 to 0.43)	0.66
Process of change (1–6)§	0.46 (0.89 to 0.03)	9.54**	0.12 (0.60 to 0.36)	0.30
Absenteeism (TiC-P)¶	0.62 (0.45 to 1.70)	1.59	0.37 (1.70 to 0.96)	0.37
Presenteeism (TiC-P)¶	0.29 (0.83 to 0.24)	1.21	0.28 (0.91 to 0.36)	0.78
AUDIT (0–40)	–	–	–	–

Significance levels used: *p<0.05, **p<0.01; ***p<0.001.

†Missing data imputed by multiple imputation.

‡Controlling for pretreatment scores (T1).

§Higher scores indicate better outcomes.

¶In relation to the previous month.

ANCOVA, analysis of covariance; PSS-14, Perceived Stress Scale; SMBQ, Shirom-Melamed Burnout Questionnaire; MADRS-S, Montgomery Åsberg Depression Rating Scale-self-assessment; ISI, Insomnia Severity Index; ITT, intention to treat; WEMS, Work Experience Measurement Scale; AUDIT, Alcohol Use Disorders Identification Test; TiC-P, Trimbos and Institute of Medical Technology Assessment Cost Questionnaire for Psychiatry.

Secondary outcome analysis

As shown in table 3, the ANCOVA of nearly all outcome measures showed significant differences between the two groups (p<0.05). Participants in the iSMI reported significantly lower levels of perceived stress, burnout, depression and insomnia severity, at post-treatment, compared with the AC participants. In addition, the iSMI showed significantly higher satisfaction with work, with regard to support, internal work experience, management and process of change. However, no significant differences were detected on outcomes of absenteeism (T2, p=0.23; T3, p=0.55) and presenteeism (T2, p=0.28; T3, p=0.38).

Since a majority of the participants fulfilled the criteria for non-organic insomnia (iSMI=70%; AC=45%), an additional analysis was conducted selecting those cases. Analysis revealed higher pre-assessment mean scores for both groups (iSMI M=15.26; AC M=15.27) and the between-group effect sizes were larger at T2 (d=0.48; iSMI M=9.58; AC M=11.74) and T3 (d=0.84; iSMI M=8.78; AC M=12.60) compared with the previous non-selected analysis.

Long-term efficacy

At the 6 months follow-up, both groups were relatively stable or continued to improve on the primary and secondary outcome measures (tables 2 and 3). Between-group analysis revealed smaller effect sizes compared with those at T2. ANCOVA comparing the participants in the iSMI and AC group at T3 showed a medium effect size with regard to the primary outcome PSS-14 (d=0.59; 95% CI 0.14 to 1.04).

Completers-only analysis

Completers-only analysis revealed comparable effects for the primary outcome at T2 (n=93; F(1,90)=11,86, p<0.001; d=0.72; 95% CI 0.28 to 1.17) and T3 (n=82; F(1,79)= 7.92, p<0.01; d=0.61; 95% CI 0.14 to 1.07). Results from the secondary outcomes analyses were similar to those found in the ITT analyses (data not shown).

Adherence

Participants completed on average 5.83 (iSMI=5.69 and AC=5.93) out of 8 weeks and in all, 13% of the participants (iSMI=6 and AC=9) dropped out early (<2 weeks) in the study. The main reason for dropping out was lack of time due to high workload. Analyses showed no significant differences on any outcome or demographic variable between those who dropped out early and those who continued throughout the programme.

Client satisfaction

After completing each module/week (1–8), participants in the iSMI group rated the utility (content) and user-friendliness (eg, access, layout) of each module on a 7-point scale (1=low satisfaction; 7=high satisfaction). Utility was given an average of 5.79 for all eight modules/weeks (SD=0.67; range=5.50–5.80) and user-friendliness, 5.10 (SD=0.96; range=4.77–6.00). Modules on exposure in different stress-related areas (eg, assertiveness, perfectionism and procrastination) and feedback (positive and negative) received the highest rating and physical exercise and applied relaxation was rated lowest on both utility and user-friendliness.

Coaching

The iSMI coaches were asked to measure and register time spent per week with each participant, for example, email feedback regarding the progression of the programme and homework assignments. Coaches spent significantly less time t(90)=−7.28, p<0.001 in relation to the AC group (M=3.98 min, SD=1.76) when compared with the iSMI group (M=16.65 min; SD=11.96). Additionally, the participants were asked questions about the relationship/alliance with their coach. Overall, 81% in the iSMI group experienced the relationship as pleasant to very pleasant, compared with 55% in the AC group.

Discussion

Results from the present study add to the growing body of studies showing that iSMIs can be an accessible and effective way of reducing work-related stress. In line with our expectations, we found that the guided iSMI effectively reduces symptoms of perceived stress in a sample of distressed managers. Analyses also indicate effects on other mental health and work-related outcomes. Long-term effects were seen on several of the mental health-related outcomes (perceived stress, insomnia and cognitive weariness). However, effects on the work-related outcomes were not sustained at the 6 months follow-up.

The effects found in the present study were larger compared with those found in our previous iSMIs trials (d≈0.60),23 25 however within CI of guided interventions (Cohen’s d=0.64, 95% CI 0.50 to 0.79) presented in a recently published meta-analyses on iSMI.24 Compared with regular face-to-face CBT-based SMIs, d=0.68 to 1.16,17 our effects are lower on stress-related measures (perceived stress, d=0.74, burnout, d=0.95) but comparable on other mental health outcomes, for example, anxiety and depression. However, it is important to emphasise that face-to-face SMIs are delivered with substantial support from health professionals, about 60 min in face-to-face compared with M=16.65 min/week in the present iSMI. Furthermore, in previous trials wait-list control groups have consecutively been used. We used an AC group, which is considered to be a more conservative approach.

Results on the primary and secondary outcome measures were sustained or continued to improve in both groups at the 6 months follow-up. Comparing long-term effects are limited due to a lack of iSMI and SMI studies including such assessment. Thus, some studies55 have provided evidence of sustained effects, and the present study adds to that evidence. Interestingly, the AC group continued to improve on most outcomes at the 6 months follow-up, resulting in lower between-group effects. This may be interpreted as an effect of spontaneous recovery. Spontaneous improvement is common in control groups.23 Moreover, both conditions had full access to treatment-as-usual and given that the participants in the present study were managers, a resourceful group (eg, 91% university-level education), it is probable that spontaneous improvement would be higher in this group.

In the present study, participants improved significantly on the work-related health outcomes, such as perceived support, internal experience, management and process of change (organisational change). However, no significant differences were found on the subscales of autonomy and time experience and additionally, the effects on the work-related health outcomes were not sustained at the 6 months follow-up. The results on the work-related health outcomes are difficult to interpret, because we have no data that supports causality between the subscales and other outcomes, and other contextual work factors. It is probable that the results on the work-related health outcomes are indirect effects, mediated by the stress reduction.

This study has the following imitations. First, the present trial was faced with attrition and non-adherence. As mentioned, 21% did not complete the postassessment questionnaires and 30% did not provided data on the 6 months follow-up. Additionally, participants completed on average 5.69 weeks (71%) in the iSMI and 5.93 (74%) weeks in the AC group. However, the attrition and adherence rates in the current trial were in line with previous iSMI trials,23 56 and within limits of acceptable drop-out rate (20% for short-term and 30% for long-term follow-up) according to guidelines.57 Considering these observations on attrition and adherence, together with the use of MI, which is considered to be a conservative approach,54 the results seem nevertheless reliable.

Second, it is likely that the participants were more motivated than the average population. Several factors may have contributed, including, recruitment was carried out through an open self-referral process and did not involve any specific company or organisation, 91% of the participants had a university-level educational background, and a diagnostic criterion (F43) were used for inclusion, selecting highly distressed individuals. These circumstances could constitute a threat to the generalisability of the results. We used a self-referral process considering that the participants would be more inclined to participate, confidant that their managers, colleagues and subordinates were unaware of their participation. Moreover, previous SMI and iSMI trials have frequently included non-clinical populations. Hence, a diagnostic criterion was used to ensure that a clinical population was included in the study. Future studies are therefore needed, evaluating different populations and recruitment strategies.

Third, unfortunately we did not collect any data from subordinates and their perception of their managers on, for example, perceived stress and feedback. Since recruitment was carried out through an open self-referral process, there was, to our knowledge, no confidential way of collecting data from subordinates. Presumably, future studies using different recruitment approaches (eg, within a company or organisation) could gain some insights on employee perception after managerial participation in an iSMI.

Fourth, we did not collect any reliable data on treatment compliance (such as degree of homework completion or active use of the iSMI). We did store information about the number of logins and login time. However, we could not discriminate between active and passive users (eg, signed in but not active). In future studies, we want to collect information from both subordinates and on homework compliance and assess the active use of the iSMI.

In conclusion, this is one of the first studies showing that iSMIs can be an effective, accessible and potentially time-effective approach of reducing stress and other mental-related and work-related health symptoms among distressed managers. Future studies are needed to address distressed managers and the potential of indirect effects on employee stress and satisfaction at work.