Stress at work: Self-monitoring of stressors and resources to support employees.

High levels of stress at work may have serious consequences for employee functioning and mental health. By providing employees with an easily accessible instrument to regularly evaluate stressors and resources, employee self-monitoring and guidance to support can be accommodated. We evaluated an online self-monitoring tool Brief Assessment of Stress and Energy (BASE). Through their organization, 139 railway emergency services employees were invited to complete BASE and six wellbeing measures. We assessed BASE in two ways: using multiple regression analysis (N = 102, 73.4%), as well as by telephone follow-up interviews during which experts and respondents evaluated the BASE outcome (N = 67, 65.7%). Explained variances of BASE on the six wellbeing measures ranged between 26.6% and 49.9%. Telephone interviews confirmed the BASE outcome. The results indicate that BASE is associated with several measures of wellbeing and accurately refers respondents to counseling. This study shows that BASE is a promising instrument to encourage employees to self-monitor stressors and resources and identify those who need counseling.

High levels of stress at work can have serious consequences for employee functioning and mental health (International Labor Organization, 2016). Various theoretical models explain how events in the (work) environment generate stress and stress responses (Bakker & Demerouti, 2017; De Lange et al., 2003; Folkman & Lazarus, 1984; Ganster & Rosen, 2013; Halbesleben et al., 2014; Karasek et al., 1998). Stressors can be defined as aspects that lead an individual to appraise their environment as exceeding their resources and threatening their wellbeing (Folkman & Lazarus, 1984). This translates to work aspects that cause stress and strain for an employee (Bakker & Demerouti, 2017). In addition to the influence of stressors, these models also include resources, emphasizing their importance in the stress process. According to Hobfoll et al. (2015) “resources are loosely defined as objects, states, conditions, and other things that people value” (Hobfoll et al., 2015, p. 2). In the work context, resources are aspects of work that motivate employees and buffer against stressors (Bakker & Demerouti, 2017). Research has shown that resources are a key component of occupational stress (Westman et al., 2005) and losing resources is a strong predictor of negative psychological outcomes (Hobfoll et al., 2015). As Hobfoll et al. (2015) state, having resources is crucial to build resilience. It is therefore important to support employees in acquiring and maintaining resources that may enhance resilience. Early detection of resources loss can contribute to the prevention of stress and a decrease in employee functioning (Westman et al., 2005).

In addition to resources, personal characteristics – aspects related to resilience and the perception of control and impact on one's environment (Bakker & Demerouti, 2017; Barbier et al., 2013) – also play a role in the stress process. Research has shown a reciprocal association between personal resources, job resources and work engagement (Bakker & Demerouti, 2017; Barbier et al., 2013; Xanthopoulou et al., 2007).

Various occupational stress screening instruments exist (Faragher et al., 2004; Hicks et al., 2010; Inoue et al., 2014; Karasek et al., 1998). However, most of these instruments only focus on complaints or do not include positive aspects of work. In addition, they do not provide direct feedback to the employee or have to be interpreted by a professional. In effort to address these issues, we developed and evaluated an online self‐monitoring tool; Brief Assessment of Stress and Energy (BASE). BASE can be used on a regular basis to self‐monitor levels of stressors and resources. Four specific characteristics distinguish BASE from other instruments. First, BASE does not focus on psychological complaints (e.g. burnout symptoms) but on daily occupational factors (e.g. inadequate facilities or support from colleagues) that can cause stress or give energy, and includes personal characteristics (e.g. being able to switch easily between tasks). Second, BASE is an online and short instrument that employees can complete within five minutes, making the instrument more accessible and easy to use. Third, BASE provides direct feedback regarding stressors, resources and personal characteristics with relevant follow‐up information, encouraging self‐monitoring, reflection, and seeking support. Fourth, BASE can be tailored to the organization, enhancing implementation of follow‐up support within BASE.

We evaluated BASE among railway emergency services personnel in the Netherlands. This high‐risk occupational group deals with organizational stressors and typically faces a variety of work‐related critical incidents, such as (attempted) railway suicides, (fatal) accidents, violence, aggression or exposure to hazardous materials. The aims of this study were to: (1) assess the level of wellbeing of Dutch railway emergency service personnel; (2) examine the association between BASE and several wellbeing measures and (3) evaluate BASE's ability to refer respondents to counseling.

METHODS

Sample characteristics

We invited 139 railway emergency services employees to participate in the study, 102 (73.4%) completed the survey in Dutch. In our sample, the mean age was 47 years (SD = 10.9), mean tenure was eight years (SD = 8.3), 93.1% was male, 88.2% was married or living with a partner and 80.4% had children. Respondents rated their current level of functioning with a mean score of 7.7 (range: 3–10).

As suggested by Osborne, (2013), we investigated individual cases to detect systematic answering patterns, such as identical answers on all items of the different measures. We found one case with an abnormal answering pattern and recoded the scores on the Depression, Anxiety and Stress Scale (DASS‐21), the PTSD Checklist for DSM‐5 (PCL‐5) and the Resilience Evaluation Scale (RES) as missing. Results of BASE and the six wellbeing measures are presented in Table 1. Respondents scored average on BASE stressors and high on resources and personal characteristics. Respondents reported low levels of burnout, depression, anxiety and stress and PTSD symptoms, and high work engagement, social support and psychological resilience.

TABLE 1

Mean scores of BASE and wellbeing measures

Measure	N	M a	SD b	Range
Stressors (BASE)	102	2.05	0.51	1.06–3.44 c
Resources (BASE)	102	3.61	0.55	1.80–4.90 c
Personal characteristics (BASE)	102	4.13	0.40	2.71–5.00 c
Burn‐out symptoms (MBI‐GS)	102	1.13	1.13	0.00–5.11 d
Work engagement (UWES)	102	4.72	1.05	1.33–6.00 d
Depression, anxiety and stress (DASS‐21)	101	0.28	0.32	0.00–1.43 e
PTSD symptoms (PCL‐5)	100	0.32	0.42	0.00–2.55 f
Social support (SSL‐12)	102	2.79	0.49	1.42–4.00 g
Psychological resilience (RES)	100	3.17	0.47	1.44–4.00 f

Abbreviations: BASE, Brief Assessment of Stress and Energy; DASS‐21, Depression, Anxiety and Stress Scale; MBI‐GS, Maslach Burnout Inventory‐General Survey; PCL‐5, PTSD Checklist for DSM‐5; RES, Resilience Evaluation Scale; SSL‐12, Social Support List; UWES, Utrecht Work Engagement Scale.

Mean.

Standard deviation.

Maximum range: 1–5.

Maximum range: 0–6.

Maximum range: 0–3.

Maximum range: 0–4.

Maximum range: 1–4.

Brief Assessment of Stress and Energy (BASE)

Employees were offered a comprehensive support program that included BASE, telephone interviews and a face‐to‐face counseling session. Employees received an invitation to complete BASE every three months. Upon completion, respondents received direct personal feedback, accompanied by the color outcome green or orange. Green is indicative of low levels of stressors and high levels of resources and personal characteristics. Based on a green outcome, no further action is advised. Orange reflects an indication of higher levels of stressors and/or lower levels of resources and personal characteristics. The advice states that the respondent will receive telephone follow‐up.

The items of BASE originate from a study within the Dutch police organization which consisted of a literature review, qualitative interviews and pilots, and a survey among 480 police employees. The Job‐Demands Resources model was used as a framework to design the study in the police context (Gouweloos‐Trines et al., 2014). We used 26 (out of 28) relevant items for the railway context, that were further adapted by incorporating existing research within the railway organization (Krommendijk, 2016) and discussing the items in a group interview with five employees. We added seven items specific to the railway work context. This resulted in a 33 item BASE (see Appendix 1 for details in Supplementary Material). BASE consists of three scales: stressors, resources and personal characteristics. Stressors were measured with items related to aspects of work or home that can cause stress for railway emergency services personnel. Resources were measured with items regarding aspects of work that give energy. Personal characteristics were measured with items relating to individual or contextual features that support employees with their work performance.

Procedure

This study concerns the first pilot measurement of the comprehensive support program. The researchers attended several regular team meetings to inform employees about the program and the study, and to answer any questions. It was emphasized that participation was voluntary and anonymous.

BASE was administered online from January 16 until February 16, 2018. Two automatic reminders were sent during a 30 day period, one after 14 days and one last‐minute reminder after 29 days. As part of the pilot measurement, BASE was supplemented by six measures to assess the overall level of wellbeing and to evaluate BASE. The following measures were added: the Maslach Burnout Inventory–General Survey (MBI‐GS), the Utrecht Work Engagement Scale (UWES), the Depression Anxiety Stress scale (DASS‐21), the PCL‐5, the Social Support List (SSL‐12) and the Resilience Evaluation Scale (RES), see Appendix 2 for details in Supplementary Material. Later measurements of the program did not include these additional questionnaires but only BASE. Respondents were presented with their BASE outcome after completing all measures.

Telephone follow‐up interviews with respondents who scored above cut‐off took place between January and March 2018. Experts employed by an organization specialized in work‐related psychological trauma in high‐risk occupations conducted the interviews (see Appendix 3 for details in Supplementary Material). Prior to starting BASE, respondents could indicate that they wished to be excluded from telephone follow‐up.

The Medical Ethical Committee of the Amsterdam University Medical Center exempted this study from formal review (W17_365 # 17.425). Written informed consent was obtained, in accordance with the European General Data Protection Regulation.

Algorithm and telephone interview

One aim of BASE was to refer employees to counseling in case of high stressors and/or low resources and personal characteristics. The algorithm was intentionally sensitive; respondents were included with only minor levels of complaints on BASE, MBI‐GS, DASS‐21 and PCL‐5. Respondents scoring above the cut‐off scores on any of the BASE subscales, or MBI exhaustion or cynicism, or on any of the DASS‐21 subscales or on the PCL‐5, received an orange outcome and telephone interview.

Cut‐off scores for BASE were based on the outcomes of the study with Dutch police. High scores were defined by scores in the upper 25% of stressors (mean score ≥ 2.50), or in the lower 25% of both resources (mean score ≤ 3.66) and personal characteristics (mean score ≤ 4.09). The combination of high stressors or low resources and personal characteristics has been based on several studies that have shown that various job resources can buffer the impact of various job demands on negative outcomes (Bakker & Demerouti, 2007, 2014; Xanthopoulou et al., 2007). For the newly added items, cut‐off scores were defined as scoring three on four items or scoring four or five on two items. MBI‐GS cut‐off scores were set at average complaints or worse on exhaustion (mean score ≥ 0.99) or on cynicism (mean score ≥ 0.49). DASS‐21 cut‐off scores were set at mild symptoms or worse on depression (≥9) or anxiety (≥7) or stress (≥14). Each item on the PCL‐5 rated as two (moderately) or higher was treated as a symptom endorsed. Cut‐off scores were set at 1 B item, or 1 C item, or two D items or 2 E items (Weathers et al., 2013).

During the interview, experts and respondents discussed the BASE outcome to assess the respondent's perception of the BASE outcome. During the interview, experts asked respondents regarding perceived stressors and resources, and their preference for receiving counseling. The expert gave advice about referral to counseling, irrespective of the respondent's results. The outcome of the interview was based on the interaction between the expert and the respondent. If the respondent wished to receive counseling they could, even if the expert did not advise it. The experts reported the discussion and outcome on a standardized form, including their expert opinion and advice.

Statistical analyses

We evaluated the internal consistency reliability of the BASE scales with inter‐item correlations, corrected item‐total correlations and Cronbach's alpha. Corrected item‐total correlations were computed to assess whether item scores regarding stressors, resources and personal characteristics were associated with overall scores of the three scales.

To explore the association of BASE with the wellbeing measures, we conducted multiple regression analyses. We performed separate regression analyses with each of the measures as dependent variables and BASE scales as independent variables. Diagnostic statistics (standardized residuals, Cook's distance, average leverage, Mahalanobis distance and covariance ratio) were used to examine extreme cases (Field, 2013; Osborne, 2010). We also assessed the assumptions for ordinary least squares (OLS) regression of linearity, normality, homoscedasticity and multicollinearity with visual inspection of the data (Field, 2013).

To assess BASE's ability to accurately refer employees to counseling, we first categorized respondents into three groups based on their scores on the MBI‐GS, DASS‐21 and PCL‐5 and the telephone interview outcome. Group one concerned respondents who scored below the cut‐off on all three measures (group label below cut‐off). Group two included respondents who scored above the cut‐off on one of the three measures and were not referred to counseling (group label no counseling). Group three concerned respondents who scored above the cut‐off on one of the three measures and were referred to counseling (group label counseling). We computed the BASE score by summing all item scores for stressors, resources and personal characteristics (first reverse scoring the resources and personal characteristics items); thus, high scores reflect high stressors, low resources and low personal characteristics. We compared the BASE score between groups with one‐way between‐groups analysis of variance (ANOVA). We assessed the assumption of equal variances with Levene's test. All statistical analyses were conducted using SPSS.

RESULTS

Association between BASE and measures of wellbeing

The internal consistency reliability results are presented in Table 2. Internal consistency reliability can be considered good when most inter‐item correlations are in the range of 0.15–0.50 (moderate magnitude) and Cronbach's alpha for the scale is > 0.80 (Clark & Watson, 1995). Corrected item‐total correlations >0.20 are recommended for including an item in a scale (Streiner et al., 2015).

TABLE 2

Internal consistency reliability analysis (N = 102)

BASE scale	Inter‐item correlations range (mean)	Corrected item total correlations range (mean)	Cronbach's alpha
Stressors (16 items)	0.005–0.627 (0.259) a	0.357–0.631 (0.467)	0.847
Resources (10 items)	0.106–0.628 (0.357) b	0.327–0.656 (0.547)	0.846
Personal characteristics (7 items)	−0.008–0.521 (0.243) c	0.242–0.594 (0.402)	0.689

75% recommended range.

82.22% recommended range.

61.91% recommended range.

Regarding the stressor scale, 75.0% of the inter‐item correlations were in the recommended range. Cronbach's alpha coefficient was 0.85. This indicates good internal consistency. Corrected item‐total correlations for this scale ranged between 0.36 and 0.63 with a mean of 0.47, indicating high item scores were associated with high scores on the overall stressor scale.

Of the resources scale, 82.22% of the inter‐item correlations were in the recommended range. Cronbach's alpha coefficient was 0.85. This indicates good internal consistency. Corrected item‐total correlations for this scale ranged between 0.33 and 0.67 with a mean of 0.55, indicating high item scores were associated with high scores on the overall resources scale.

In regard to the personal characteristics scale, 61.91% of the inter‐item correlations were in the recommended range. Cronbach's alpha coefficient was 0.69. This indicates acceptable internal consistency. Corrected item‐total correlations for this scale ranged between 0.24 and 0.59 with a mean of 0.40, indicating high item scores were associated with high scores on the overall personal characteristics scale. Cronbach's alpha of all scales could not be improved by deleting any items.

In the regression analysis, we examined extreme cases with diagnostic statistics. For several cases, the standardized residuals were equal or greater than 3 and the average leverage was more than three times as large. Therefore, we considered these cases as unreliable. As a result, one case was recoded as missing on all measures. Additionally, two cases on the MBI‐GS, two cases on the UWES, one case on DASS‐21, four cases on the PCL‐5 and one case on SSL‐12 were treated as missing in the analysis. All assumptions for OLS regression were met, except for the assumption of homoscedasticity that was violated in the models with burn‐out (MBI‐GS), depression anxiety and stress (DASS‐21) and PTSD (PCL‐5). When the homoscedasticity assumption is violated, Hayes and Cai (2007) recommend employing the heteroskedasticity‐consistent standard error (HCSE) estimator of OLS parameter estimates. This estimates the standard errors without assuming homoskedasticity. We used the RLM macro for SPSS (Darlington & Hayes, 2016) to employ the HC4 estimator in all models (Hayes & Cai, 2007).

The significant F‐statistics in Table 3 indicate that BASE was associated with all measures of wellbeing. The explained variance (R 2) ranged between 26.6% and 49.9%. BASE explained most variance on burnout (49.9%) and work engagement (49.6%). The standardized regression coefficients indicate that higher stressors were significantly related to higher burnout symptoms, depression, anxiety and stress and PTSD symptoms. Higher resources were significantly related to higher work engagement and social support and lower burnout. Higher personal characteristics were significantly related to higher work engagement, social support and psychological resilience and to lower depression, anxiety and stress and PTSD symptoms.

TABLE 3

Ordinary least squares (OLS) regression analysis with BASE and wellbeing measures, using standard error estimates not assuming homoscedasticity (HC4 )

BASE Scales	Measures	B	SE HC4	β	p	F	p	R 2
	Burn‐out (N = 100)					19.449	<0.001	0.499
Stressors		0.433	0.121	0.370	<0.001
Resources		−0.801	0.212	−0.471	<0.001
Personal characteristics		−0.251	0.348	−0.071	0.474
	Work engagement (N = 100)					25.664	<0.001	0.496
Stressors		−0.184	0.104	−0.166	0.079
Resources		0.836	0.163	0.517	<0.001
Personal characteristics		0.660	0.305	0.197	0.033
	Depression, anxiety and stress (N = 100)					11.303	<0.001	0.363
Stressors		0.327	0.087	0.403	<0.001
Resources		−0.205	0.123	−0.177	0.097
Personal characteristics		−0.469	0.204	−0.194	0.023
	PTSD symptoms (N = 96)					13.305	<0.001	0.362
Stressors		0.294	0.089	0.360	0.001
Resources		−0.189	0.121	−0.166	0.122
Personal characteristics		−0.587	0.188	−0.254	0.002
	Social support (N = 101)					10.646	<0.001	0.310
Stressors		0.128	0.084	0.178	0.129
Resources		0.341	0.127	0.324	0.009
Personal characteristics		0.827	0.233	0.381	<0.001
	Psychological resilience (N = 99)					12.596	<0.001	0.266
Stressors		−0.004	0.061	−0.009	0.944
Resources		−0.041	0.077	−0.058	0.594
Personal characteristics		0.792	0.149	0.537	<0.001

Heteroskedasticity‐consistent standard error (HCSE) estimator of OLS parameter estimate, HC4.

Expert opinion in telephone interview

Based on the cut‐off scores of the MBI, DASS‐21 and PCL‐5, 67 (65.7%) of the 102 respondents could be included in the analysis. Four respondents were excluded because they gave no informed consent to be included, one respondent did not complete the PCL‐5 and one respondent could not be reached after five attempts. This resulted in 61 (59.8%) respondents in the analysis.

Eighteen respondents received counseling and 45 respondent did not. Experts reported various reasons why respondents did not receive and/or want counseling, such as no reported problematic complaints or only frustrations regarding the organization, having sufficient resources, support and coping mechanisms. In addition, a few respondents indicated they had received counseling or therapy in the past.

We conducted a one‐way between‐groups analysis of variance (ANOVA) to investigate whether the BASE score differed between the three groups: below cut‐off (N = 23), no counseling (N = 45) and counseling (N = 18). The results showed there was a statistically significant difference in BASE score between the groups: F (2, 83) = 28.99, p < 0.001. Post‐hoc comparisons using the Tukey HSD test indicated that the BASE score of the counseling group was significantly higher (M = 80.0, SD = 12.57) compared to the no counseling group (M = 70.29, SD = 10.29, p < 0.002) and the below cut‐off group (M = 56.52, SD = 6.71, p < 0.001). This significant difference indicated that respondents with the highest BASE scores also received counseling, thereby confirming BASE's outcome.

DISCUSSION

The goal of this study was to evaluate BASE – a self‐monitoring tool that aims to identify high stressors and/or low resources in employees and refer them to counseling. We demonstrated that BASE was associated with wellbeing and subsequent referral to further counselling was accurate. BASE can be considered a promising self‐monitoring instrument for Dutch railway emergency services personnel.

A number of specific outcomes warrant further discussion. First, BASE stressors displayed a stronger association with negative wellbeing compared to positive. The reversed was true for BASE resources. This is in line with other studies that found that positive and negative aspects of work predict different (mental) health outcomes (Bakker & Demerouti, 2007, 2017; Schaufeli & Bakker, 2004). Second, BASE personal characteristics was significantly associated with psychological resilience, consisting of RES subscales self‐confidence and self‐efficacy. This is in line with other studies that also have related personal characteristics to resilience, including self‐efficacy (Barbier et al., 2013; Bonanno, 2021; Connor & Davidson, 2003; Denckla et al., 2020; Van der Meer et al., 2018; Xanthopoulou et al., 2007). No association was found between BASE resources and psychological resilience. This could be due to BASE resources including items focusing on support at work – while BASE personal characteristics contains items in reference to support from friends and family. Apparently, psychological resilience is more closely related to support in the personal surroundings. Nevertheless, our findings suggest that strengthening both resources and personal characteristics is beneficial to employees, considering their significant relation to different measures of wellbeing. Support and recognition from supervisors and colleagues after a potentially traumatic event are crucial to one's wellbeing (Olff, 2012).

It could be argued that organizations have a moral, economic and legal obligation to support optimal employee functioning and mental health. BASE is part of a comprehensive support program that could be offered to employees regularly. This would allow to detect problematic levels of stressors and/or resources and offer support to employees before effects become chronic. Implementing this stepwise approach could thus contribute to optimal functioning and mental health. Additionally, the program may also instigate a cultural change within organizations in which colleagues feel more at ease to share potential issues. Since perceived peer support is related to lower levels of distress, a supportive work context is beneficial to both employees and organizations (Gouweloos‐Trines et al., 2017).

Some limitations to our study must be considered. Our study was conducted with railway emergency services personnel and further research is needed to learn whether our results translate to other professions. Furthermore, the study is cross‐sectional and based on self‐report. BASE and the wellbeing measures were administered at the same time, therefore common method variance may inflate the relationships found between BASE and the wellbeing measures. We tried to counteract this by not showing respondents their BASE outcome until they completed all measures. Other practical considerations also had an effect on this study's design. For instance, only respondents with the orange BASE outcome were included in telephone interview to limit the burden on respondents with no complaints. Though the algorithm included the wellbeing measures and was intentionally sensitive to include respondents with even the most minor complaints, exact numbers of true positives and false positives could therefore not be computed. Lastly, gender specific observations are impossible since our sample was predominantly male (93.1%).

Our study has several strengths. It adds to the evidence base of preventive monitoring tools at the employee level that aim to structurally assess employee wellbeing. It provides the evaluation of a method that could contribute to the prevention of reduced employee functioning and mental health problems. The high response rate is not only indicative for enthusiasm among respondents, but also provides representative results for the population. Lastly, by including expert opinion in assessing if BASE was able to correctly refer employees, a real‐life evaluation step was added to the research design.

We recommend future research to evaluate BASE in different occupational settings, to assess the influence of stressors and resources on employee functioning and mental health. In addition, BASE's cost‐effectiveness could be determined in longitudinal studies. Lastly, when BASE is provided on a regular basis it encourages employees to monitor themselves over time. The effect of this self‐monitoring on both the individual as well as on the organizational culture could be investigated.

In sum, the results showed that BASE is a promising instrument that is able to accurately identify and refer railway emergency services personnel with high stressors and/or low resources. Psychosocial support guidelines accentuate the importance of detecting those with concerning levels of distress (Creamer et al., 2012; Te Brake & Duckers, 2013). At the same time, it is clear that guidelines cannot provide in the day‐to‐day implementation of their recommendations. Therefore, a gap exists between guidelines and practice (Te Brake & Duckers, 2013). This gap can only be closed by an organizational culture free of mental health stigma, supportive leadership and peer support, timely detection and available care. Our results showed that BASE can be used for early detection in the intended population, an important step in bridging the gap between guidelines and practice.

CONFLICT OF INTEREST

This research has been conducted by the independent research center ARQ Centre of Expertise for the Impact of Disasters and Crises within ARQ National Psychotrauma Centre. The ARQ Institute of Psychotrauma (ARQIVP) has an interest in the use of this self‐monitoring tool for their practice and were involved in part of the research. The authors state that ARQIVP has had no influence on the outcomes of this study.

Supplementary Material

Click here for additional data file.