A training intervention for supervisors to support a work-life policy implementation.

BACKGROUND: Effective policy implementation is essential for a healthy workplace. The Ryan-Kossek 2008 model for work-life policy adoption suggests that supervisors as gatekeepers between employer and employee need to know how to support and communicate benefit regulations. This article describes a workplace intervention on a national employee benefit, Family and Medical Leave Act (FMLA), and evaluates the effectiveness of the intervention on supervisor knowledge, awareness, and experience with FMLA.
METHODS: The intervention consisted of computer-based training (CBT) and a survey measuring awareness and experience with FMLA. The training was administered to 793 county government supervisors in the state of Oregon, USA.
RESULTS: More than 35% of supervisors reported no previous training on FMLA and the training pre-test revealed a lack of knowledge regarding benefit coverage and employer responsibilities. The CBT achieved: (1) a significant learning effect and large effect size of d = 2.0, (2) a positive reaction to the training and its design, and (3) evidence of increased knowledge and awareness regarding FMLA.
CONCLUSION: CBT is an effective strategy to increase supervisors' knowledge and awareness to support policy implementation. The lack of supervisor training and knowledge of an important but complex employee benefit exposes a serious impediment to effective policy implementation and may lead to negative outcomes for the organization and the employee, supporting the Ryan-Kossek model. The results further demonstrate that long-time employees need supplementary training on complex workplace policies such as FMLA.

Introduction

Work-life policy

With an increasing number of employees dealing with caregiving and other family responsibilities, the adoption and support of work-life policies at the workplace have increased in importance [1]. Work-life policies are employer-sponsored benefits or working conditions to balance work and nonwork demands (e.g., sick and vacation leave, maternity leave, retirement benefits, daycare, insurance coverage) and to avoid work-life conflicts, when roles within and outside the workplace are overwhelming to an employee or interfering with one another.

Two of the largest US nationwide surveys interviewing employers (The US National Study of the Changing Workforce—NSCW) and employees (US National Study of Employers—NSE) reflect a workforce in need of balancing work and life demands. Retention and helping employees to manage work and family life are, according to the NSE, next to productivity, job satisfaction and commitment, the main reasons for employers to adopt work-life initiatives [2]. The NSCW reveals that the proportion of women and of men in the US workforce are nearly equal (51% men and 49% women), 78% of couples are in dual-earning families, and 35% of employees have elder care responsibilities. Balancing work and life is not easy; 45% of employees with families reported work-family conflicts. This is substantially higher than 25 years ago when the number was 34% [3]. The NSCW also shows that supportive work-life policies and practices lead to employees reflecting more positive work and life outcomes such as higher job satisfaction, commitment, and retention; fewer work-life conflicts, and better mental health [3-6] benefiting both the employee as well as the organization. Yet, the NSE reveals that only 27% of US employers surveyed reported efforts to inform employees about assistance for balancing work and family demands.

The US “Family and Medical Leave Act” as a work-life benefit

Our study evaluated a workplace intervention, investigating the awareness, knowledge, and usage of a US work-life benefit, the Family and Medical Leave Act (FMLA) of 1993. This leave is designed to help employees balance family and work demands by allowing up to 12 weeks of unpaid but job-protected leave from work over 12 months for their own or their families' serious medical needs, including time off to care for a new child. This leave allows a flexible work schedule and is one means to prevent work-life conflicts and to increase employees' productivity, job satisfaction, and retention [7,8]. FMLA includes maternity leave, a globally recognized benefit offered by 190 countries worldwide [9]. It also offers leave for serious medical health issues and to care for family members with serious medical health issues. More than 35 million employees took leave under FMLA in the first 7 years since its enactment [10] and it is widely required of organizations (employers with at least 50 employees and all governmental organizations). In 2000, 11% of private establishments and all public-sector employers in the US were covered under FMLA, a total of 58% of the US workforce [10,11]. However, employers do not always communicate details to the employee [12]. Approximately 41% of employees at these FMLA-covered establishments reported that they had not heard of FMLA and 49% of employees at these covered establishments reported not knowing whether the FMLA applied to them personally [11]. There is a lack of training and education on this law for supervisors [13,14]. In 2000, 14.5% of eligible employees elected to not take leave when needed [11] due to financial constraints (as FMLA is unpaid leave), fear of job loss, seniority, or job advancement. Organizational culture such as the appearance of not being committed to the work and job demands are additional reasons [11,12]. The employer needs to provide information on FMLA but also support and assist the employee in using the leave.

FMLA regulations are complex. Human resource (HR) personnel, the acknowledged experts for FMLA leave and usually the ones who grant and track the leave for their organization [13], report struggling with administering the law and providing consistent information [14]. Definitions such as “serious health condition” are unclear and frequent changes to improve and clarify these regulations challenge FMLA's administration. Another challenge for organizations is that, next to the federal US law, several states (California, Connecticut, Hawaii, Maine, Minnesota, New Jersey, Oregon, Rhode Island, Vermont, Washington, Wisconsin, and the District of Columbia) add their own state version of the law [16]. Differences between federal and state law can include differences in employer/employee eligibility, leave time, notification, and benefit coverage [see Table 1 for an example comparison between FMLA and Oregon Family Leave Act (OFLA)]. The OFLA is more generous than the federal law because it lowers the eligibility criteria for employees; it also covers domestic partners and grandparents as eligible family members, and it includes time off for a sick child (not a serious health condition). Covered employers must comply with the federal or state provision that provides the greater benefit to their employees. It is also a federal requirement to have the FMLA posters visibly posted in the workplace. These posters contain basic information on the leave laws, including eligibility and leave time [17].

Table 1

Differences between the federal Family and Medical Leave Act (FMLA) and Oregon Family Leave Act (OFLA)

	FMLA	OFLA
Employer requirement	50 employees	25 employees
Health coverage continuation	Yes	No – not required
Employment requirement	12 mo and 1250 hr	180 d (6 mo) and 25 h/wk (exception parental leave)
Eligible leave reasons	Serious health condition leavePregnancy disabilityParental leaveMilitary leave	Additionally includes: Care for sick child
Leave time	1 time of 12 wk in 12 mo	Up to 3 times of 12 wk in 12 months possible for the following reasons and only in this order:– 1st, 12 wk of pregnancy disability – 2nd, 12 wk of parental leave – 3rd, 12 wk for a sick child
Definition: Serious health condition	Detailed requirements (e.g., two or more treatments: a second “in-person” visit)	Same requirements, but less specific (e.g., only: two or more treatments)
Covered family member	Child <18 y Child ≥18 y, incapable of self-careSpouse and parent	Additionally includes:Child ≥18 y (not disabled)Parent-in-lawSame-sex domestic partnerGrandparent/grandchildren
Parental leave time	6 wk for each parent	12 wk for each parent

Challenges: Effective policy implementation on supervisor level

Implementing a complex work-life policy such as FMLA and following its regulations can be challenging. Ineffective implementation due to unaware, undereducated, and unsupportive workplaces may result in increased work-family conflicts, intentional or unintentional misuse and abuse of the leave (e.g., taking too much time off), low employee morale [13,15], and even lawsuits against the employer [18]. It is important to educate supervisors, as the key person between employer and employee, to effectively communicate and support the policy with employees [19].

Ryan and Kossek 2008 [20] offer a model to depict the complexity of the connection between work-life policy adoption, employee perception of inclusion, and work/health outcomes. Their model (adapted in Fig. 1) emphasizes the role of policy implementation and recognizes the supervisor as the key agent in that implementation.

Fig. 1

Ryan-Kossek (2008) model on work-life policy adoption and implementation. Note. Adapted from “Work-life policy implementation: Breaking down or creating barriers to inclusiveness?,” by A.M. Ryan and E.E. Kossek, 2008, Human Resource Management, 42, 298 p. Copyright 2013, E.E. Kossek. Adapted with permission.

As suggested by the Ryan-Kossek model (Fig. 1), employee perceptions and understanding of the benefits can influence employee attitudes and satisfaction, which in turn is linked to outcomes such as work-life conflicts, stress, job commitment, and productivity [6,21]. A trained supervisor who supports FMLA can create a supportive atmosphere in which employees do not fear retribution for taking needed leave and are supported in managing their workload. This may lead to a healthy workplace with positive outcomes such as job satisfaction and productivity.

Most organizations and employers rely on their HR department to effectively communicate the benefit regulations and their values to their employees. However, HR management has become increasingly decentralized, and implementation of work-life policies has shifted to the frontline supervisors. Through supervisors' behaviors and attitudes, the employee perceives the work environment as either supportive or nonsupportive of these policies [20,22-25]. Supervisors are the gatekeepers to policy implementation and communication and training supervisors in policy implementation and informal practices is crucial [20,21]. Even though supervisor support is positively linked to reduced work-family conflict [26], there is a gap between espoused and enacted policy practices and one of the most significant reasons for this gap is that supervisors are not sufficiently trained in policy and benefit practices and delivery [27].

Training is an essential tool to maintain a healthy and competitive workplace [28]. It is a multibillion-dollar industry worldwide (in 2010 approximately $52.8 billion was spent on training in the United States) with management and supervisory training predicted to lead in future spending increases. The e-learning market can provide superior effectiveness and reduced costs compared to classroom training and is beneficial because it can fit around day-to-day work activities [29-31]. However, the literature reveals a lack of workplace interventions and trainings in the work-life field [32]. Supervisor training to increase work-life support is one of the most frequently advocated interventions by work-life experts [19,21,33,34].

Purpose of the study

The purpose of this study was to assess and then increase supervisors' knowledge of FMLA in the United States, a work-life benefit that is the supervisors' responsibility to communicate and support in order to effectively implement the policy.

Materials and methods

Participants and recruitment

Oregon county governments (urban and rural areas) were invited to participate in a research study including a computer-based FMLA training for supervisors. Twenty-six of Oregon's 36 counties (72%) accepted. The county populations participating ranged from 1,426 to 540,410 residents [35].

The counties' HR/personnel departments were requested to recruit all supervisors and managers by distributing a training advertisement flyer (approved by Oregon Health and Science University and The Johns Hopkins University's Institutional Review Board) through postings, pay envelope, and/or e-mail. HR employees in a payroll or administrative role but not in a supervisory position also completed the training (11% of participants). In eight counties (31%) the training was mandatory; in most counties it was optional but encouraged by the HR department. A lottery-style incentive for participating supervisors ($25 or $49 visa gift card) was offered and 20 counties (70%) accepted. No training performance differences were found due to mandatory versus optional or incentive versus no incentive counties.

Training design and topics

Broad dissemination to our target audience was essential so we selected interactive computer-based training (CBT) due to its potential delivery as web training as the most time-flexible choice for supervisors. The “Family and Medical Leave” training was presented in cTRAIN (NwETA.com; Lake Oswego, OR, USA [36]), a CBT format incorporating behavioral education principles [37,38] such as self-pacing, real-life scenarios, and interactivity (e.g., pretraining and post-training questions, quiz questions during training with immediate feedback, movies). We used a simplified response unit with nine buttons, placed over the laptop keyboard to provide an easy input option for the keys 1–9 required to interact with the training. The simple navigation together with clear instructions and a consistent screen layout (use of color, function keys, and uncluttered text) was designed to reduce cognitive overload due to readjustment or confusion and disorientation [36,39,40]. If implemented effectively, this will ensure that the participant focuses on training content instead of the training format [31,41,42]. Previous training with cTRAIN increased knowledge and achieved large effect sizes, a measure used to compare trainings across studies [33,43].

The “Family and Medical Leave” training described general benefit requirements such as eligibility and qualifying FMLA situations, the role and responsibilities of the supervisor, and information on the request and approval process. The training also described Oregon's version of family medical leave, the OFLA, and described the differences between FMLA and OFLA (see Table I in Appendix 1 for an overview). The “Family and Medical Leave” training was reviewed and pilot tested for accuracy, clarity and duration by HR experts, benefit specialists, and an attorney specializing in FMLA and policy development.

Study design and measures

The study determined the effectiveness of FMLA training targeting supervisors employed in participating Oregon county governments. Outcomes were measured pretraining and immediate post-training with follow-ups at 3 months, 6 months, and 12 months post-training intervention. The main pretraining and post-training measures included a knowledge test and a survey on FMLA awareness and experience. Additionally, the number of employees using leave for FMLA and other reasons was collected monthly from counties and walkthroughs of three to five county buildings in each county to observe bulletin boards to record evidence of posted information on FMLA were conducted pretraining and post-training (see Table 2 for details).

Table 2

Study measures and time points when they were collected

Measure	Time points
Knowledge test	Immediate pre-trainingImmediate post-training
Survey – Questions 1–6 (FMLA awareness)	Immediate pre-trainingPost-training follow-ups (at 3 mo, 6 mo, and 12 mo)
Survey – Questions 7–12 (FMLA experience)	Post-training (at 6 mo and 12 mo)
Employee leave data	Pre-training (average of 6 mo)Post-training (average of 6 mo)
Observation of FMLA postings	Pre-training (at 3 mo prior training)Post-training follow-ups (at 6 mo and 12 mo)

FMLA, Family and Medical Leave Act.

Procedures

The training was implemented using laptops on site in counties' conference or training rooms. After signing a consent form, each participant completed six sections: demographic questions, survey, pretraining knowledge test, FMLA training, and post-training knowledge test and evaluation (Table 3).

Table 3

Training procedures in detail

Sections	Details	Example
Demographic information	15 questions on age, gender, race, job duration, computer usage, etc.	What is your gender?
Survey	6 questions on FMLA awareness/workplace support	As far as I am aware, information about FMLA/OFLA leave is posted for the employees at the workplace.
Knowledge test	10 questions to measure baseline knowledge	What are the variables used to define eligibility for FMLA/OFLA?
FMLA training	Interactive training with 48 Info screens and 13 quiz questions	For details see Appendix A1
Knowledge test (repeated)	10 questions to measure knowledge gain	What are the variables used to define eligibility for FMLA/OFLA?
Evaluation	3 reaction questions	What is your overall rating of this FMLA and OFLA training?

FMLA, Family and Medical Leave Act; OFLA, Oregon Family Leave Act.

Supervisors received the option to participate in the research study by first completing 15 demographic questions prior to the training. If they declined to participate, they were routed automatically to the beginning of the training session and their test data were excluded from analyses. The training started with a survey including six questions (yes/no) on supervisor's awareness of FMLA postings, usage, and organizational support such as FMLA training. This survey was repeated online (surveymonkey.com) at 3 months, 6 months, and 12 months after the training and was offered to all supervisors in each participating county, including the question whether or not they had taken the training. During the 6 month and 12 month follow-ups we also asked six yes/no questions regarding their personal experience with FMLA. Following the survey, supervisors completed 10 multiple-choice pretest questions measuring knowledge of FMLA. Each question had four answer options, a chance level of 25%. Next, participants started the training with 10 information sections on FMLA, each concluding with quiz questions that had to be answered correctly in order to proceed to the next section. If answered incorrectly, the participant automatically repeated the information section and answered the quiz question again, until correct. At the conclusion of the training, supervisors completed a knowledge post-test (the same 10 questions as in the pretest, but with the answer options in a different order), rated the training and its design, and received a training completion certificate.

Data analysis

Data were analyzed by using SPSS version 15.0 (SPSS Inc., Chicago, IL, USA, 2006). Analyses included Student's t tests for variables with two levels (e.g. gender, survey questions) and analyses of variance (ANOVA) for variables with more than two between-subject levels (e.g., age) or more than two within-subject levels (e.g., survey questions repeated over four different time periods). Additionally, analyses of covariance (ANCOVAs) to evaluate the independent variable by controlling for the effects of other continuous variables not of primary interests were performed (e.g., knowledge gain as independent variable, age as independent variable, pretest scores as covariate). In order to compare pretraining and post-training survey responses, the data were analyzed on a county level as supervisors were not identifiable and predata and postdata could not be matched on an individual level.

Results

Sample description

A total of 793 supervisors completed the FMLA training; 54.2% were women and 94.2% were white (Table 4). The supervisor sample is similar to the distribution of demographics in Oregon (50.5% women, 88.6% white) [35].

Table 4

Participant characteristics (n = 793)

Question	Answer	n (%)
Gender	Female	430 (54.2)
Age	>50 y	401 (50.6)
Ethnicity	Hispanic/Latino	18 (2.3)
Race	White	747 (94.2)
Education	>13 y	675 (85.1)
Supervision of	1–10 employees	529 (66.7)
Current job duration	≤10 y	410 (51.7)
Working h/wk	≥40 h	719 (90.7)
Job position	Manager or supervisor	702 (88.5)
Time spent on a computer	>5 h per wk	714 (90.1)

County level analyses

Survey analyses on supervisor's awareness and opinions

The survey completed prior to the training and 3 months, 6 months, and 12 months post-training was analyzed on a county level with 15 participating counties with respondents of 5 to 121 supervisors per county who also indicated by self-report that they took the previous training (11 counties in which fewer than 5 supervisors responded to the follow-up surveys were excluded). An ANOVA with the percentage of the survey answer “yes” as the independent variable and the repeated within-subject variable “time” with four levels (pretraining, 3-month follow-up, 6-month follow-up, 12-month follow-up) revealed positive increases in “yes” answers to every question. Almost every question revealed significant changes (Table 5). Prior to the training, a mean of 7% of supervisors per county believed that FMLA leave was not “taken” at their workplace; this decreased to a mean of 4% of supervisors per county at the 12-month post-training survey. This was the only question that did not reach significance (p = 0.088). Additionally, on average prior to the training, approximately 10% of participants per county stated that their workplace had no postings on FMLA for employees; this significantly declined to 2% at the survey 12-month post–training (p = 0.002). Pretraining and post-training workplace visits confirmed that all counties followed the federal requirement and had the official FMLA posters clearly visible. Finally, 33% of supervisors per county stated prior to the training that they had not received previous training on FMLA. This significantly declined to 12% at the 12 months follow-up survey (p < 0.000).

Table 5

Pretraining and post-training survey questions analyzed on county level

Survey question (n = 15 counties)	Yes answers (%)				p
	Prior to training	3 mo after	6 mo after	12 mo after
As far as I am aware, FMLA/OFLA leave is taken by employees.	92.8	92.7	95.9	95.3	0.088
As far as I am aware, my workplace has required training on FMLA/OFLA leave for supervisors.	66.5	85.5	89.5	87.6	<0.000
As far as I am aware, supervisors at my workplace are trained to help with FMLA/OFLA related questions and guide through the approval process.	65.1	83.5	84.3	84.7	<0.000
As far as I am aware, information on FMLA/OFLA leave is posted for the employees at the workplace.	90.3	91.9	96.6	97.7	0.002
As far as I am aware, information about FMLA/OFLA leave is posted for the employees at the workplace in multiple languages.	49.8	61.6	58.6	72.5	0.000
As far as I am aware, the posted information about FMLA/OFLA leave is kept updated.	87.9	93.4	94.6	96.5	0.001

FMLA, Family and Medical Leave Act; OFLA, Oregon Family Leave Act.

A county-level comparison of pre-training survey data with 3 month, 6 month, and 12-month follow-up survey responses only from supervisors who did not take the training (responses varied from 6 to 121 responses per county) did not reveal significant changes. However, due to the small number of supervisors responding “not taking the training,” we could only include five counties that had more than four respondents throughout all surveys.

Table 6 shows the additional six post-training survey questions on supervisor's personal experience and opinion on FMLA that were collected 6 months and 12 months after the training. These questions were also analyzed on a county level, including 16 counties with 6–75 respondents per county who self-reported that they took the training (10 counties with fewer than 5 respondents were excluded). During the 12-month follow-up, more than 70% of supervisors per county answered that they had helped an employee with FMLA leave requests in the past year and approximately 15% of the supervisors per county had personally used FMLA in the past. Approximately 35% of supervisors per county revealed that they still did not feel confident guiding an employee through the FMLA process.

Table 6

Additional post-training survey questions analyzed on county level

Survey question (n = 16 counties)	Yes answers (%)		p
	6 mo after training	12 mo after training
As a supervisor I have helped employees with their FMLA/OFLA leave request in the past (e.g., providing information and forms).	72.0	71.8	0.839
I have used FMLA or OFLA leave in the past year.	15.4	14.1	0.513
I feel confident to guide an employee through the FMLA or OFLA process.	64.1	67.7	0.493
The requirements for FMLA/OFLA (e.g., time of employment, necessary documentation/ certification, giving notice requirement) are too restrictive.	8.9	13.6	0.023
It is a problem for the employee that FMLA/OFLA leave is unpaid.	50.8	46.0	0.251
It is a problem for the workplace that FMLA/OFLA provides protective leave because it is difficult to complete the work for the employee who takes leave.	35.0	34.4	0.884

FMLA, Family and Medical Leave Act; OFLA, Oregon Family Leave Act.

County leave information

Pretraining leave information, collected for 6 months prior to the training, revealed that 23 of 26 counties (89%) had at least one employee who took FMLA leave within the 6 months prior to the training. The number of employees per county ranged from 38 to 1,744 in the smallest and largest of the 26 counties, respectively. County level analyses of leave data from 20 counties (6 counties excluded due to an incomplete data set) categorized as either FMLA, sick, vacation, or other leave, revealed that per county and per month, a mean of 7.5% of all leave hours taken were FMLA leave hours and that a mean of 4.1% of all employees requesting leave took FMLA-related leave. FMLA was the leave category with the longest duration of leave time taken, with a county mean of 7.4 leave days per month and employee (Fig. 2). Post-training leave data did not show significant differences from the pretraining data.

Fig. 2

Average monthly leave information analyzed on county level for Family and Medical Leave Act (FMLA), sick, vacation or other leave.

Training pretest analyses: Characteristics of supervisors with the greatest knowledge gap

Analysis of the pretest results revealed significant pretest knowledge score differences for sex, age, job position, and job duration. Table 7 shows that men, supervisors older than 50 years, and supervisors who have been working at their job for more than 30 years achieved significantly lower pretest knowledge scores than their counterparts. Participants who answered the question for their job position with “Supervisor” achieved lower knowledge scores than those answering with “Manager” or “Other.” Supervisors who responded “no” to the survey awareness questions on perceived FMLA usage, FMLA postings, and required FMLA training at their workplace also received significantly lower scores than supervisors who responded “yes” to these questions.

Table 7

Pretest score differences by supervisor characteristics and survey response (n = 793)

Variable∗	Pretest knowledge scores (%)	N	p
Gender	Males = 58.93Females = 62.74	363430	0.001
Age	<51 y = 62.45>50 y = 58.58	392401	0.015
Job position	Supervisor = 59.15Manager = 61.94Other = 65.60	38831491	0.002
Job duration	0–10 y = 61.9011–20 y = 59.5221–30 y = 62.8330+ y = 55.24	41022811342	0.024
As far as I am aware, FMLA/OFLA leave is taken by employees.	Yes = 60.45No = 52.45	62349	0.001
As far as I am aware, my workplace has required training on FMLA/OFLA leave for supervisors.	Yes = 62.59No = 57.18	559234	<0.001
As far as I am aware, information on FMLA/OFLA leave is posted for the employees at the workplace.	Yes = 61.70No = 54.42	71677	<0.001

FMLA, Family and Medical Leave Act; OFLA, Oregon Family Leave Act.

Only variables where pretest scores significantly differed between levels are shown.

Training effectiveness

A two-tailed, paired sample t test [t(1,791) = 54.79, p < 0.001] showed that supervisors’ knowledge of FMLA significantly improved from pretest [mean: 61.0% correct; standard deviation (SD) = 16.7] to post-test (mean: 94.3% correct, SD = 9.5). This reflects an effect size (d) of 2.0 associated with the training; effect sizes >0.8 are considered large [44]. Pretest scores ranged from 10% to 100% correct, with 37% of supervisors receiving pretest scores <60%. Post-test scores ranged from 30% to 100% correct. On the post-test, only five of the 793 supervisors (fewer than 1% of all supervisors) received scores <60%, indicating a strong learning effect for the participants.

At pretest, the items with the greatest knowledge gap included information on insurance coverage during family medical leave (17%), the special military leave regulations (39% correct), employer responsibilities such as required postings (39% correct), and differences between the federal FMLA and state OFLA (eligibility differences: 56% correct; leave time overlap: 63%; selection criteria: 66%). All other knowledge questions were answered correctly by more than 70% of supervisors. At the post-test, each question was answered correctly by more than 85% of participants. The number of supervisors answering the individual post-test questions correctly ranged from 88.1% to 98.7% depending on the question. Participants completed the training in a mean of 62.3 minutes, including the demographic questions (mean of 2.0 min), the pretest questions (mean of 8.0 min), and the post-test questions (mean of 3.1 min). They were significantly faster in completing the post-test questions than the pretest questions [t(1,792) = 34.86; p < 0.001].

Knowledge gain analyses: Supervisors who benefited the most

Knowledge gain analyses revealed that the training was differentially effective across supervisors. ANCOVAs were conducted to test if the effect of the training varied by supervisor characteristics, including pretest knowledge score as a covariate to control for differences in knowledge prior to the training. Female supervisors, supervisors younger than 50 years, supervisors who had <10 years at their job, supervisors with ≥13 years of education, and supervisors who reported spending >5 hours per week on a computer achieved significantly higher knowledge gain scores than their counterparts (Table 8).

Table 8

Knowledge gain among supervisors (n = 793), controlled for pretest scores as a confounding variable

Variable∗	Knowledge gain (%)	N	p
Gender	Males = 32.12Females = 34.26	363430	0.001
Age	<50 y = 34.28>50 y = 31.45	392401	<0.001
Education	≤13 y = 30.77>13 y = 33.72	118675	0.001
Job duration	0–10 y = 34.0211–20 y = 32.8921–30 y = 33.01>30 y = 29.01	41022811342	0.007
Computer time	<5 h per wk = 28.33>5 h per wk = 33.84	79714	0.001

Only variables where knowledge gain significantly differed between levels are shown.

Training evaluation

Participants rated the “Family Medical Leave” training on overall satisfaction, usefulness, and design/format using a four-point scale (Poor, OK/neutral, Good, Excellent). The training received a good or excellent satisfaction rating from 85.8% of supervisors, whereas 74.0% found the training good or excellent in terms of usefulness, and 86.3% rated the design and format as good or excellent (Fig. 3). Supervisors' rating of the training and their knowledge gain were not significantly associated.

Fig. 3

Supervisors evaluated the training (n = 793).

Discussion

This study provided a large-scale workplace intervention that: (1) improved knowledge about FMLA as part of a work-life balance policy in a sample of US county government supervisors in Oregon, and (2) provided information on supervisors' knowledge, awareness, and training effectiveness on this important but complex employee benefit. The intervention evaluations identified a lack of critical information for conveying FMLA regulations required by US national and state law. It also showed the potential for an economical training approach, CBT, to impart the necessary information and to increase awareness in supervisors, necessary to be effective communicators of the FMLA as proposed by the Ryan-Kossek 2008 model on policy adoption and implementation [20]. This study supports the need for effective intervention in the area of supervisor training called for by Hopkins [19] and addresses the identified gap of work-life intervention studies [45].

Supervisors' limited knowledge and the need for training

Leave information directly received from Oregon counties prior to our training revealed that FMLA was used in a substantial percentage of Oregon counties (89% of participating counties) and employees (4% of employees who took leave per month requested it under FMLA). During our 6 month and 12 month follow-up survey, approximately 15% of supervisors per county who took our training answered that they personally had previously used FMLA. Family medical leave also accounted for approximately 8% of all leave hours taken per month in the counties (including vacation, sick, and other employee leave), indicating the importance of knowledge and information dissemination of this well-utilized benefit. Supervisors are generally the first members of the organization to interact with employees on questions of their benefit and leave options such as maternity and other types of FMLA leave. However, according to self-reports from these Oregon supervisors, more than 35% of supervisors had not received training on FMLA and did not feel prepared to guide an employee who had FMLA-related questions prior to our intervention. Indeed, part of some counties' motivation for participating in the study was to receive FMLA training at no cost.

The training's knowledge pretest revealed that supervisors answered only slightly more than half of the questions correctly (mean: 61%; chance = 25%), illustrating a limitation in supervisors' FMLA knowledge. The test identified important FMLA areas and topics on which supervisors especially struggled such as insurance coverage under FMLA (only 17% correct) and employer responsibilities (only 39% correct), details about the military option to take the leave (only 39% correct), and also the differences between the federal (FMLA) and their own county's specific Oregon state (OFLA) requirements of Family Medical Leave (56% to 66% correct).

This lack of knowledge may affect communication and workplace support. Employees might not be aware of their right to take FMLA and balance work and family; they might also feel a lack of support and fear retribution. Ineffective policy implementation may lead to negative outcomes for the employee and the organization [20] as they have the legal responsibility to grant adequate leave. Failing to provide the leave or inform the employee about it in a timely manner can lead to financial costs to the organization [13]. It raises important concerns for organizations in the United States or anywhere that must teach complex requirements of employee benefit topics to their supervisors as the ones increasingly charged to implement HR practices, and thus implement those policies effectively.

Training effectiveness and Kirkpatrick's evaluation levels

The CBT proved to be a successful tool to educate the supervisors on FMLA and, furthermore, it was well received by the supervisors themselves. The training significantly improved supervisors' knowledge from 61% to 94% and all 10 post-test questions were answered correctly by more than 88% of the supervisors. With an effect size of d = 2.0, this training is well above that calculated for other safety trainings in the workplace. An effect size above 0.8 is considered to be a large effect [44]. According to Burke et al's [46] meta-analysis of 95 studies (from 1971 to 2003) on workplace health and safety trainings aiming at modifying “Safety Knowledge”, “Safety Performance”, and “Safety and Health Outcomes”, only five studies achieved an effect size above 2.0 for the category “Safety Knowledge”. In addition, >80% of supervisors' reaction to the FMLA training, its usefulness, and design and format revealed positive (good or excellent) ratings, thus supporting an e-learning training tool based on behavioral education principles as effective in participants' opinions.

The computer-based FMLA training program was well received by more than 86% of supervisors for format and design. A moderately engaging, interactive training method such as CBT is presumably more effective than passive training (e.g., lectures) [39], and in the long term less costly than highly engaging methods (e.g., hands-on demonstrations) [46]. The current study suggests that even complex employee benefit information can be taught effectively using CBT.

Kirkpatrick's [47] four-level model of training evaluation comprises: (1) reaction towards the training, (2) learning and knowledge improvement, (3) behavior changes, and (4) results of the training. Our training demonstrated highly positive outcomes for the first two levels (reaction, knowledge) of his model, although most training is evaluated by only one of these criteria [48]. Furthermore, our survey on supervisors' opinions, administered prior to and after the training, revealed an incomplete degree of training and awareness of FMLA usage and postings prior to the training, which improved significantly 3 months, 6 months, and 12 months after training. The increased awareness in postings (90% prior to vs. 97% 12 months after training) and usage (92% prior to vs. 95% 12 months after training) indicates positive training transfer and verbal behavior change towards attitudes and awareness, the third level of Kirkpatrick's model.

Individual differences in training effectiveness

As the training scores reveal, age, sex, and computer usage had an effect on the computer training performance. Especially the effects of age and computer literacy on CBT are supported by previous studies [49]. In our training women and younger supervisors (50 years or younger) achieved higher pretest scores. They were also the ones with significantly higher knowledge gain than their counterparts (male supervisors and older supervisors). This indicates that especially men and supervisors older than 50 years may require special attention to training on benefit topics and perhaps supplemental training, in this case on the US Family and Medical Leave.

In our training the post-test performance declined gradually with increasing age. Although supervisors younger than 30 years did best with a performance of 98% correct answers, supervisors aged 31–50 years received a performance of 96% and supervisors older than 50 years received the lowest scores (92%). Even though older adults are well aware of the benefits of computer use and are actively seeking training programs, research has shown that older adults exhibit significantly lower performance results than younger adults in computer training [49,50].

Our training revealed that supervisors employed for more than 30 years at their current job received the lowest pretest scores and that supervisors at their current job less than 11 years achieved higher knowledge gain than supervisors with longer job durations. It is reasonable to speculate that supervisors who have been at their workplace for a long time might not receive the most current information about newly implemented benefit topics and leave laws. The training was also more effective for highly educated supervisors (at least 13 years of education), and for supervisors who spend >5 hours per week at a computer.

Individual differences including age, computer literacy, and previous use of CBT do affect training effectiveness and attitude toward e-learning [49,50]. As CBT is preprogrammed, it is not individualized. However, it is possible to create different versions and incorporate intelligent tutoring methods to address individual needs based on a comprehensive person analysis [31]. Wallen and Mulloy [50] suggested the use of pictures and audio narration to adjust for employees older than 45 years, which were both incorporated in our FMLA training.

This study provides a significant methodological contribution to the sparse workplace intervention literature on a benefit topic and the first evidence of the potential for the role of supervisors to be effective communicators of policy to assure effective implementation as required by the Ryan-Kossek 2008 model for work-life policy adoption [20] (Fig. 1).

Conclusions

Effective policy implementation and support is as important as policy adoption because it hugely influences how the employee perceives the policies and benefits. As the Ryan-Kossek 2008 model [19] points out, for effective implementation, the key step is to train the supervisor, who is directly at the interface of the employer and the employee and can help to create a family-supportive workplace that maintains employee satisfaction and protects the employer. However, supervisors are a population often neglected in training on employee benefits [27]. Our training results show that the organization can help to increase supervisors' knowledge of complex benefit topics by offering training and informational material such as postings. Our CBT demonstrated that the key information of a complex employee benefit such as the United States FMLA can be learned in just over 1 hour of training, and that supervisors are increasingly in need of training the longer they have been with the organization. As benefit regulations are complex, training can ensure that managers and supervisors have the information to take action in accordance with guidelines, benefiting and supporting both employees and their companies.

Conflicts of interest

Oregon Health and Science University (OHSU) and Dr. Anger have a significant financial interest in Northwest Education Training and Assessment, LLC, a company that may have a commercial interest in the results of this research and technology. This potential individual and institutional conflict of interest has been reviewed and managed by OHSU.

Table I.

FMLA training content details.

Section	Content	Quiz question
Pretest	10 pretest questions with 4 multiple choice options and feedback screen of received scores (10 screens)	PQ 1–10
Info-Set 1	Statistics on FMLA/OFLA use (4 screens)	Q1
Info-Set 2	Introduction and definition of FMLA/OFLA. (3 screens)	Q2, Q3
Info-Set 3	Differences between the federal family leave FMLA and the OFLA (5 screens)	Q4, Q5
Info-Set 4	Detailed benefits of FMLA and OFLA for the employee as the amount of protected time, but also benefits for the employer as to follow the law (7 screens)	Q6
Info-Set 5	Basic requirements and limits as eligibility of employer and employee for FMLA and OFLA (4 screens)	Q7
Info-Set 6	Detailed qualifying situations for FMLA and OFLA as serious health condition of oneself or family member, pregnancy, and parental leave, etc. (5 screens)	Q8
Info-Set 7	Military leave (8 screens)	Q9
Info-Set 8	Amount of possible leave time and how this differs between FMLA and OFLA with clarifying scenarios (5 screens)	Q10, Q11
Info-Set 9	Leave types and relationship to other leave laws (5 screens)	Q12
Info-Set 10	Employee responsibilities to give notice and for the employer and manager/supervisor role to inform about the leave law with required posters and to grant adequate leave request.Approval process with adequate request forms and medical certifications and tracking of FMLA and OFLA information (7 screens)	Q13
Posttest	10 Posttest questions (same as pretest questions) with 4 multiple choice options (different order of answer options than in pre-test) and feedback screen of received scores (10 screens)	PQ 1–10

FMLA, Family and Medical Leave Act; OFLA, Oregon Family Leave Act.