Implementing a Sleep Health Education and Sleep Disorders Screening Program in Fire Departments: A Comparison of Methodology.

OBJECTIVE: The objective of this study is to compare three methods of administering a sleep health program (SHP) in fire departments.
METHODS: An SHP, comprising sleep health education and screening for common sleep disorders, was implemented in eight fire departments using three approaches: expert-led, train-the-trainer, and online. Participation rates, knowledge assessments, surveys, and focus group interviews were analyzed to assess the reach and effectiveness of the methodologies.
RESULTS: The Expert-led SHP had the highest participation rate, greatest improvement in knowledge scores, and prompted more firefighters to seek clinical sleep disorder evaluations (41%) than the other approaches (20 to 25%). Forty-two percent of focus group participants reported changing their sleep behaviors.
CONCLUSION: All approaches yielded reasonable participation rates, but expert-led programs had the greatest reach and effectiveness in educating and screening firefighters for sleep disorders.

The public expects firefighters to be available around-the-clock and to perform flawlessly when called upon. The majority of career firefighters work 24-hour extended-duration shifts, which cause acute sleep deprivation, chronic sleep deficiency, and circadian misalignment.[1] The negative consequences of sleep loss are of major concern to firefighters[2] and more than half of firefighters report sleep disturbances.[3] In addition, a significant proportion of firefighters suffer from undiagnosed sleep disorders, which further impair sleep and exacerbate fatigue.[4] Sleep deficiency and sleep disorders are associated with an increased risk of crashes, injuries, and adverse mental and physical health outcomes,[3-8] including a greatly increased risk of motor vehicle crashes and heart attacks, the two leading causes of death in firefighters.[9] In addition, the annual direct costs to the United States from sleep disorders and sleepiness are estimated to be in excess of 15 billion dollars, and total costs including lost productivity may exceed 150 billion dollars.[10] The total cost of accidents due to sleepiness has been estimated to be between 43 and 56 billion dollars.[11]

Fatigue education and awareness training programs are required in commercial aviation[12] and similar programs have been instituted successfully for railroad workers,[13] truckers,[14] and nurses.[15,16] Recently, we developed a sleep health program (SHP) for firefighters, which included some of the key components recommended for a comprehensive fatigue risk management program[17]: sleep health education, incorporating caffeine re-education, and advice on dealing with shiftwork, as well as screening for common undiagnosed sleep disorders. We educated and screened almost 7000 firefighters in 66 US fire departments across 32 states, with departments ranging in size from 22 to 1574 firefighters. Sleep disorders were highly prevalent in these active-duty career firefighters, with over one-third (37%) of firefighters screening positive for at least one sleep disorder, most of whom were previously undiagnosed. Twenty-eight percent screened positive for obstructive sleep apnea (OSA), 6% for insomnia, 9% for shift work disorder (SWD), and 3% for restless legs syndrome (RLS). Compared with those who did not screen positive, firefighters who screened positive for a sleep disorder were 3.5 times more likely to report getting injured, more than twice as likely to take a sick day or self-report falling asleep while driving, and had double the risk of reporting a motor vehicle crash.[4]

In the current study, we employed multiple approaches to evaluate the reach and effectiveness of three delivery methods for the SHP: expert-led (EL); train-the-trainer (TT); and online (OL). We characterized participation rates, knowledge gained by firefighters after completing the education, the rate of evaluation, and treatment for firefighters identified as being at risk for a sleep disorder, and the perceived strengths and challenges associated with each implementation methodology.

METHODS

We performed a study to compare the reach and effectiveness of three methodologies to deliver an SHP to firefighters. The SHP consisted of a 30-minute educational training session on healthy sleep and fatigue countermeasures, including video testimonials from firefighters previously treated for sleep disorders, and union and departmental leadership encouraging participation in the program, followed by a screening for common sleep disorders. We conducted this program in eight fire departments and compared the participation rates, knowledge gained, and effectiveness of the programs following three different implementation approaches: EL (two departments); TT (two departments); and OL (four departments).

Recruitment

Recruitment letters were sent to the 3511 city, county, and federal U.S. fire departments, which have three or more station houses, in order to gauge interest in participating in the SHP. A semi-structured phone interview was conducted with interested groups to determine eligibility and identify the most suitable form of program administration.

Sixty-six out of 117 departments interviewed were selected to participate in the survey study and eight of these were further selected to participate in the evaluation of implementation methodology. On the basis of the number of firefighters, level of departmental interest, experience implementing wellness programs, and the ability to meet implementation requirements (eg, Internet access, availability of trainers), two departments were assigned to EL, two departments to TT, and four departments to OL methodology. Departments were not assigned randomly. Two additional departments were included in the OL group in anticipation of lower participation rates.

Implementation

The sleep health education was delivered using three different methodologies. In the departments assigned to EL, a member of the research team presented the education and screening program in-person to groups ranging from 2 to 50 firefighters. Each department assigned to the TT methodology appointed 8 to 12 peer trainers who completed a 2-day training course taught by the research team. This course provided trainers with a background understanding of sleep health, and training in the logistics necessary to conduct the sleep disorders screening. The peer trainers subsequently led the sleep health educational training in their departments to groups of 3 to 16 firefighters. To ensure consistency and quality, a member of the investigative team evaluated approximately 10% of peer trainers’ presentations using an assessment checklist and provided summarized critiques to the trainers and supervisors. For the OL departments, a Web site was provided with a video of the education session and a toll-free helpline.

Sleep Disorders Screening

Following the education training session, firefighters were given the opportunity to participate in a voluntary screening program for common sleep disorders. Validated tools were used for identifying the risk of OSA,[18] moderate to severe insomnia,[19] and RLS.[20] For SWD, we created a screening tool on the basis of The International Classification of Sleep Disorders-2 (ICSD-2) diagnostic criteria.[21] This screening was conducted on paper in the EL and one TT departments and online in the OL and the second TT departments. Firefighters were informed of their sleep disorder risk either online (online questionnaire) or via a letter (paper questionnaire). In the EL departments, we referred at-risk firefighters to a sleep clinic near their location. In the TT and OL departments, at-risk firefighters were referred to a list of board-certified sleep specialists in their area provided by the American Academy of Sleep Medicine web site.

Surveys

To evaluate each firefighter's knowledge of the educational program content, anonymous assessments were administered to each firefighter before and following the educational training session. Assessments included eight multiple-choice or true/false questions that broadly covered the educational content. After approximately 1 year, firefighters were also asked to complete an end-of-program questionnaire, which asked them about their impressions of the SHP, to confirm that they were notified of their sleep disorder risk, and whether or not they sought further evaluation and treatment. Firefighters also rated the program using discrete 7-point Likert scales, with 1 anchored as “Not at all” and 7 anchored with “Very important or helpful.” All end-of-program surveys were completed online.

Confidentiality

In addition to institutional approval from the Partners Healthcare Research Committee, we obtained a Certificate of Confidentiality from the National Institutes of Health to ensure confidentiality of the sensitive data that were collected. Data from individual participants were assigned numerical identification numbers, which were used to link data records for each individual, and the remainder of the analysis record was de-identified. The name–code index was maintained in a secure location, in an encrypted file on a password-protected, IP-restricted computer system. Data were not shared with fire department administrators or union personnel.

Statistical Analysis

A Chi-square was used to compare participation rates, changes in assessment scores, firefighters’ rating of the program, and the rate of evaluation and treatment for firefighters identified as being at risk for a sleep disorder among program implementation methods. Further analysis using logistical regression, treating department as a random effect to account for inter-department variability under the same program implementation method, was also conducted. Alpha was set at 0.05.

Focus Groups

To evaluate the impact of the SHP further and to achieve a deeper understanding of the program's effectiveness from a firefighter's viewpoint, we conducted nine focus groups 4 to 15 months following the education sessions across three departments, each representing one of the three implementation approaches. Within each department, participants were divided into three separate focus groups: (1) firefighters who completed educational training and the sleep disorders screening questionnaire; (2) firefighters who completed educational training and did not complete the sleep disorders screening questionnaire; and (3) high ranking officers and fire department administrators, regardless of their participation. Focus group participants were given a short survey to assess their views of the SHP. The focus group lasted approximately 2 hours and the moderator followed a discussion guide to ensure consistency across groups and sessions. Each focus group was recorded by video and was notated using Verbatim Transcription. Using NVIVO 9.0 software (QSR International, Doncaster, Victoria, Australia), transcripts were analyzed to categorize comments on the basis of the core topics that emerged.[22]

RESULTS

Participation

Firefighters in the three implementation groups (Table 1) were comparable in self-reported demographics, health, and work-related characteristics (Table 2). A total of 6101 firefighters were eligible to participate in the SHP: 1630 in EL departments, 1519 in TT departments, and 2952 in OL departments. A total of 4603 firefighters (75.4%) attended the sleep health education. The participation rate was significantly higher in the EL (83.9%) and TT (88.5%) departments, than the OL departments (64.0%) (P < 0.0001, Table 1). The cooperation rate (ie, the number of firefighters who participated in the sleep disorders screening/the number of firefighters present for the educational training) was also significantly higher in the EL (73.2%) and TT (66.3%) departments than in the OL departments (39.7%) (P < 0.001, Table 1). The cooperation rate for the end-of-program questionnaire was 25.9% overall, and was significantly higher in the OL departments (43.0%) than in the EL (21.4%) and TT (16.7%) departments (P < 0.0001, Table 3).

TABLE 1

Comparison of Departments Grouped by Program Administration Type

	Expert-Led	Train-the-Trainer	Online	Total	P
Number of departments	2	2	4	8
Number of stations	38	87	119	244
Area covered (square miles)	441.1	8780	1136	10,357.1
Number of firefighters	1630	1519	2952	6101
Number of training sessions	110	142	—	—
Training participation rate, n (%)	1368 (83.9)	1345 (88.5)	1890 (64.0)	4603 (75.4)	<0.0001
Baseline survey participation rate*, n (%)	1002 (61.5)	892 (58.7)	751 (25.4)	2645 (43.4)	<0.0001
Baseline survey cooperation rate*, n (%)	1002 (73.2)	892 (66.3)	751 (39.7)	2645 (57.5)	<0.0001
End-of-program questionnaire cooperation rate*, n (%)	214 (21.4)	149 (16.7)	323 (43.0)	686 (25.9)	<0.0001
Training knowledge assessment (n pre/n post-training)	750/749	1252/1249	1381/910	3383/2908
Training, mandatory or with continuing education (CE) credit	Mandatory	Mandatory	Mixed	Mixed

*Participation rate is the response rate that includes all potentially eligible participants. Cooperation rate is the response rate of participants that were contacted for a specific part of the study.

TABLE 2

Self-Reported Participant Characteristics

Characteristic	Expert-Led	“Train-the-Trainer”	Online	Total
N	1002	892	751	2645
Age, yrs
Mean ± SD (range)	42.9 ± 8.4 (22–1)	38.0 ± 9.6 (20–5)	40.8 ± 7.9 (21–2)	40.6 ± 8.9 (20–1)
Sex, n (%)
Women	22 (2.2)	65 (7.3)	28 (3.7)	115 (4.3)
Men	955 (95.3)	801 (89.8)	705 (93.9)	2461 (93.0)
Not known	25 (2.5)	26 (2.9)	18 (2.4)	69 (2.6)
Race, n (%)*
White	855 (85.3)	687 (77.0)	640 (85.2)	2182 (82.5)
Black	68 (6.8)	56 (6.3)	23 (3.1)	147 (5.6)
Asian	3 (0.3)	9 (1.0)	2 (0.3)	14 (0.5)
Native American	19 (1.9)	17 (1.9)	27 (3.6)	63 (2.4)
Pacific	3 (0.3)	3 (0.3)	3 (0.4)	9 (0.3)
Other	37 (3.7)	82 (9.2)	45 (6.0)	164 (6.2)
Not known	39 (3.9)	59 (6.6)	25 (3.3)	123 (4.7)
Ethnicity, n (%)
Hispanic	60 (6.0)	158 (17.7)	137 (18.2)	355 (13.4)
Other	880 (87,8)	633 (71.0)	567 (75.5)	2080 (78.6)
Not known	62 (6.2)	101 (11.3)	47 (6.3)	210 (7.9)
Body mass index (kg/m2), n (%)
<25	144 (14.4)	200 (22.8)	108 (14.4)	452 (17.1)
≥25 and <30	522 (52.1)	424 (48.2)	373 (49.7)	1319 (49.9)
≥30 and <35	248 (24.8)	200 (22.8)	171 (22.8)	619 (23.4)
≥35	82 (8.2)	55 (6.3)	56 (7.5)	193 (7.3)
Not known	6 (0.6)	13 (1.1)	43 (5.7)	62 (2.3)
Body mass index, kg/m2
Mean ± SD (range)	28.8 ± 4.3 (15.8–55.2)	28.1 ± 4.3 (16.7–48.8)	28.7 ± 4.5 (17.6–53.2)	28.5 ± 4.3 (15.8–55.2)
Health (subjective), n (%)
Poor	6 (0.6)	3 (0.3)	1 (0.1)	10 (0.4)
Fair	42 (4.2)	26 (2.9)	50 (6.7)	118 (4.5)
Good	398 (39.7)	306 (34.3)	274 (36.5)	978 (37.0)
Very good	418 (41.7)	388 43.5)	293 (39.0)	1099 (41.6)
Excellent	129 (12.9)	162 (18.2)	111 (14.8)	402 (15.2)
Not known	9 (0.9)	7 (0.8)	22 (2.9)	38 (1.4)
Employed in the fire service, yrs
Mean ± SD (range)	15.6 ± 8.0 (0.3–40.4)	11.3 ± 9.1 (0.1–38.0)	14.4 ± 7.8 (1.0–38.2)	13.9 ± 8.5 (0.1–40.4)
Scheduled shift duration, n (%)
8	122 (12.2)	9 (1.0)	20 (2.7)	151 (5.7)
24	759 (75.7)	456 (51.1)	584 (77.8)	1799 (68.0)
48	106 (10.6)	341 (38.2)	0 (0.0)	447 (16.9)
Other: 12, 10/14, 72	10 (1.0)	20 (2.2)	33 (4.4)	63 (2.4)
Unknown	5 (0.5)	66 (7.4)	114 (15.2)	185 (7.0)
Average scheduled hours per week Mean ± SD (range)	50.0 ± 9.4 (3–144)	58.2 ± 9.9 (40–127)	60.5 ± 12.1 (2–127)	55.4 ± 11.3 (2–144)
Average monthly work hours Mean ± SD (range)	222.5 + 80.5 (24–648)	253.0 + 90.6 (22–656)	263.4 + 87.0 (30–608)	242.7 + 87.4 (22–656)
Second job, n (%)	430 (42.9)	221 (24.8)	286 (38.1)	937 (35.4)

*Respondents could select more than one race (57 reported more than one race).

TABLE 3

End-of-Program Questionnaire Participation, Diagnosis, and Treatment Compliance

	Expert-Led	“Train-the-Trainer”	Online	Total	χ2 P
Total participants in end-of-program questionnaire	214	149	323	686	—
Screened at high risk for a sleep disorder, n (%)	87 (40.7)	60 (40.3)	138 (42.7)	285 (41.5)	0.84
Reported in the end-of-program questionnaire that they screened at high risk for a sleep disorder, n (%)	61 (70.1)	37 (61.7)	83 (60.1)	181 (63.5)	0.30
Sought clinical evaluation as a result of the program, n (%)*	36 (41.4*)	15 (25.0*)	27 (19.6*)	78 (27.4*)	0.0015
Recommended treatment, n (%)†	30 (83.3)	8 (53.3)	17 (63.0*)	55 (70.5*)	0.057
Compliant with treatment, n (%)‡	23 (76.7)	7 (87.5)	14 (82.4*)	44 (80.0*)	0.81

*Number of subjects who screened at a high risk for a sleep disorder used as a denominator.

†Number who sought clinical evaluation as a result of the program used as a denominator.

‡Number recommended for treatment used as a denominator.

Knowledge Assessment

Overall, firefighters showed a 28.6% improvement in knowledge assessment scores after training. Firefighters in the EL departments showed the most improvement (34.0%), followed by TT (28.5%) and OL (24.2%) departments. There was a significant improvement in the knowledge gained by firefighters in the EL departments, when compared separately with the firefighters in the OL departments (posthoc P = 0.03) but when adjusted for multiple comparisons, only a trend was shown (P = 0.10).

Overall, 40.8% (1079/2645) of firefighters screened at risk for at least one sleep disorder. The risk of having a sleep disorder was consistent among methodology groups (Chi-square, P = 0.84; logistic model with fire department as a random effect, P = 0.91). Of those who completed the end-of-program questionnaire, 41.5% (285/686) had screened at risk for a sleep disorder. Of those who had screened positive and were notified that they were at risk for a sleep disorder, only 63.5% (181/285) answered affirmatively when queried as to whether they had been notified that they were at risk. The firefighters who screened positive in the EL departments reported a significantly higher notification rate (70.1%) than those firefighters in the TT (61.7%) and OL (60.1%) departments (P = 0.0015, Table 3).

Of the 285 firefighters who screened positive for a sleep disorder and completed the end-of-program questionnaire, 78 (27.4%) reported that they sought a clinical evaluation as a result of the program. Firefighters in EL departments were significantly more likely to seek a clinical evaluation (36/87; 41.4%) than the firefighters in TT (15/60; 25%) and OL (27/138; 19.6%) departments (P < 0.0015, Table 3). There was no difference in treatment compliance among groups (Table 3).

Program Impact

On the end-of-program questionnaire, 563 of 686 (82.1%) firefighters reported participating in the SHP. More than half (52.0%, 293/563) of firefighters rated the importance of the SHP as 5 or higher out of 7 and there was no statistical significant difference between EL, TT, and OL departments (P = 0.87). Overall, 42.6% (240/563) rated the program 5 or higher for “useful information” and similarly between the programs (P = 0.61), and 50.3% (283/563) rated it 5 or higher for “recommend to other fire departments” with no differences between the groups (P = 0.67).

A total of 62 firefighters participated in the nine focus groups (three to nine participants in each) and completed a short survey to assess their views of the SHP (not all firefighters answered every survey question). Overall, 42% (24/57) of the firefighters reported changing their sleep behavior as a result of the program (31.6, 57.9, and 36.8% for EL, TT, and OL, respectively, P = 0.22) and 29% (17/58) of firefighters reported that the program had a positive impact on their life (Fig. 1) (29.4%, 35.3%, and 35.3%, for EL, TT, and OL, respectively, P = 0.94).

FIGURE 1

Focus group participants reported, in an anonymous survey, the effects of the fatigue risk management program and specific behavioral changes. There was no difference among groups in the percentage of firefighters reporting behavior change.

In the focus group discussion, firefighters reported that the SHP was effective for all methods of implementation. When all implementation methods were described, the majority of firefighters thought that the EL format seemed the best for an occupational fatigue risk management program for firefighters. When asked to assess additional considerations such as program cost, logistics of scheduling, and confidentiality, however, it was difficult to find consensus as to which implementation method was best. Most firefighters agreed that individual departments would need to assess their own needs and assets before determining the best program implementation to employ. A list of the pros and cons of each implementation method was created on the basis of the focus group discussion (Table 4). Four major themes emerged from the content analysis: communication/confidentiality; awareness of fatigue and sleep-related topics; behavioral and lifestyle changes; and firefighter culture and the challenges it creates for an SHP.

TABLE 4

Focus Groups: Pros and Cons of Program Administration Type

	Expert-Led	Train-the-Trainer	Online
Pros	Professionalism	Peers can be trusted	Inexpensive
	Material is solid and accurate	Less expensive than Expert-led	Easy to distribute
	Participant engagement is more likely	Trainers are in place	No scheduling necessary
	Individual questions possible	Individual questions possible	Wider Reach
	No need for technology or software	No need for technology or software	Participant can stay on duty
	Procedures are closely followed	Wider reach than Expert-led	Consistent messaging
	Personal approach	Personal approach	Immediate feedback from screening
Cons	Expensive	More expensive than Online	Easy for participants not to engage
	May be unfamiliar with the population	Participants answers may be less honest	Lower participation rates
	Smaller Reach	Paperwork can be lost	Impersonal
	Scheduling can be difficult	Not experts in material	No ability to ask individualized questions
	Takes participants out of service	Accuracy of training questioned	Technology required
	Outsiders mistrusted	Less respected than experts	Technical problems
	Fear of Intent	Confidentiality a concern	Increased preparation time required
		Topic dependent

Communication/Confidentiality

In focus groups, firefighters expressed a great deal of initial skepticism and even some fear regarding occupational health programs in general. One expressed mixed feelings, saying, “I think there are some people in the department who don’t like the idea that their job and their profession are involved in their personal and medical information and then there are others who think it's a good, proactive way to stay on top of your health.” Firefighters stated that transparent and effective communication about the purpose and content of the program was extremely important. Firefighters recommended that union groups should communicate information about upcoming programs in-person at meetings and ensure adequate dissemination of information about the program. Having a local champion advocating for the program was seen as positive. Firefighters in the focus groups believed that the educational portion of the program was essential and it increased their interest in participating in the sleep disorders screening survey.

Despite the measures we took to ensure confidentiality, some firefighters also stated that they were concerned about it and how any data collected during the program could negatively impact their work or social life. Apprehensions mentioned by the firefighters included the potential for new work hours guidelines, lost work time or punitive effects on the job, the cost of sleep disorder treatment, and their ability to obtain life insurance if found to have a sleep disorder.

Most firefighters stated that confidentiality was adequately addressed and maintained during this SHP and their fears of providing sensitive and personal information were significantly reduced as a result. This opinion was especially true in EL department who completed the paper survey. Although data were not shared with fire departments, firefighters in one of the TT departments mentioned concerns about providing confidential information to their peers or entering such information on departmental computers. Some firefighters who did not participate in the sleep disorders screening said that they might have participated had the screening been anonymous rather than confidential.

Awareness of Fatigue- and Sleep-Related Topics

The majority of firefighters agreed that the most important outcomes of the SHP included an increased awareness of fatigue-related issues among firefighters, the opportunity to receive sleep disorder treatment, and information to help improve habits that impact sleep on and off the job. There was a small minority that disagreed with these observations, although these were typically individuals who opted out of completing the sleep disorders screening survey.

Firefighters stated that, following the program, they witnessed an impact on their own awareness as well as their colleagues regarding sleep and fatigue topics. Individuals raised specific elements discussed in the educational training, such as the dangers of fatigue, the negative health consequences of sleep deficiency, as well as demonstrating an increased understanding of sleep disorders. Others mentioned positive cultural effects on raising awareness around sleep, including facilitating discussions of the problems associated with sleep disorders, such that the stigma associated with having or being treated for a sleep disorder was diminished. One firefighter stated, “I heard of a lot of people having sleep studies. I think if you hadn’t come to town half these people wouldn’t have [had] sleep studies” and another commented “There's an engineer [who] used to be asleep, it seemed like the entire shift, because he was just so out of it. He got the CPAP and he's just awake all day long.”

Almost unanimously, firefighters expressed the opinion that the education and screening program should continue to be offered by the Federal Emergency Management Agency (who provided grant support for this work) for all fire departments and should be implemented in other industries that schedule shift work. Many believed that the results of this project would help to increase participation in future SHPs and some firefighters stated that this program could initiate a cultural change on the topic of sleep in the fire service.

Behavioral and Lifestyle Changes

Many firefighters in the focus groups commented that the SHP had a substantial impact on the behavior of firefighters in their department (43.5% EL, 36.8% TT, and 46.7% OL reported behavioral changes, P = 0.83) and that they believed that some of these changes may have a lasting effect. Some firefighters stated that they had adopted healthier habits and others reported that they were formally diagnosed and treated for sleep disorders. Although most changes noted were on the individual level (ie, “I take naps at the station now after this [program] simply because the department supports this;” “Knowing now that it is much more important, sleep is precious to me.”), some were noted at the departmental level (ie, “Station-wise, it has [changed]. The guys will do half caffeine per coffee. We take blood pressures once a week after fires. People have started to monitor themselves a little bit more.”).

Firefighters reported that sleep, sleep disorders, and fatigue were not common discussion topics before the implementation of the SHP. Many firefighters believed that the training session increased the likelihood that firefighters would seek medical advice about a sleep disorder concern. Some firefighters who received notification that they had a high risk of a sleep disorder did not seek further evaluation. Those firefighters reported that the cost, time, effort, fear of the unknown, spousal pressure, peer pressure, fear of punitive damages, aversion to lifelong treatment with continuous positive airway pressure (CPAP), and the impact on their ability to work were barriers to seeking treatment.

Firefighter Culture and Challenges

There are unique aspects to the firefighter culture that are important to consider in creating an SHP. Firefighters in the focus groups emphasized this uniqueness citing the close proximity of sleeping quarters and sharing of bunk rooms often led to coworkers being able to observe and impact each other's sleep quality and quantity directly. For example, if one firefighter snores loudly, he may disrupt the sleep of many other firefighters in the station (“Can you believe he's keeping us all awake? ... the guys who may not necessarily have the sleep problem are up because of the guys that have the sleep problem.”).

Firefighters also stated that their training schedules were extremely full and many felt overburdened at times by this. Thus, firefighters were sensitive to the addition of new training requirements such as the educational portion of this SHP. Nonetheless, given that optional trainings are often disregarded, many firefighters believed that the sleep health training would need to be made mandatory to be effective.

DISCUSSION

Firefighters must provide 24/7 coverage every day of the year. Although they traditionally work in 24-hour shifts,[1] it is imperative that firefighters remain alert in order to conduct their work safely and effectively. Moreover, being predominantly middle-aged shiftworkers, firefighters have an increased risk of sleep disorders. We therefore developed and evaluated an SHP to educate firefighters on sleep health and screened for common sleep disorders. In order to inform the best practice for implementation of the program, we evaluated three different implementation methods.

Regardless of whether the program was conducted with EL, TT, or OL methodology, firefighters reported that the SHP was important, provided helpful information, and led to positive behavioral changes. A greater percentage of firefighters participated in the EL and TT departments. Firefighters increased their knowledge of sleep health in all departments, with EL departments showing the largest gains. Firefighters in EL departments also sought clinical follow-up and had higher rates of treatment compliance more than the TT and OL departments.

Education on healthy sleep was an important aspect of the SHP, as it engaged firefighters in the topic and promoted participation in the sleep disorders screening. In the focus groups, many firefighters reported that they had little or no interest in undergoing screening until they participated in an educational training session. Although the use of online learning is becoming increasingly popular, in this SHP, an in-person presentation from a sleep health expert or trained firefighter elicited more participation in the sleep disorders screening than did the online education. Firefighters who participated in the EL education also had a greater increase in knowledge, though we recognize that online learning can be more effective than face-to-face delivery in other settings.[23] Our online educational module may have been strengthened by increased interaction with the user, a tactic that has been shown to enhance learning.[23] In volunteer and career firefighters, an interactive, multimedia training tool improved knowledge transfer and retention compared with traditional classroom learning.[24] Heightened interactive learning may convince more firefighters to be screened for common sleep disorders.

An important end-point of the SHP is for firefighters who are at a high risk for a sleep disorder to seek further evaluation, diagnosis, and treatment. Firefighters in the EL departments were twice as likely to seek evaluation as those in the OL departments. This difference might be accounted for by the closer working relationship that was established in the EL departments and more personal referrals to a sleep disorders clinic with which we had established a connection. In one EL department, investigators provided additional phone calls and reminders about clinic referrals and this effort likely led to the increased clinic follow-up in that group. Interestingly, although not significant (P = 0.057), treatment recommendation was higher in the EL departments. Again, this is likely due to the physicians in the clinic being familiar with the SHP in contrast to the less interactive process of having firefighters print out a letter explaining their risk to take to a clinic, as was recommended to firefighters in the OL departments.

Communication regarding the risk of a sleep disorder may need to be improved. Although 41.5% of firefighters who took the end-of-program questionnaire had been notified that they were at a high risk of having a sleep disorder, when asked “Did you receive notification that you were at high risk for any sleep disorder (s)?”, only 26.4% answered affirmatively. Thus, we did not communicate effectively to approximately 36% (104/285) of the at-risk firefighters, despite our extensive efforts. Alternatively, the inconsistency in the notification of high risk and the response from the participant in knowing he or she was at high risk may be due to forgetfulness in the intervening year or the discounting of the information as not something critical to remember. Our message, including the seriousness of the level of risk and the adverse health and safety consequences of untreated sleep disorders, needed to be more clearly conveyed. Messaging should be written in a language that is common to firefighters and it should not only inform but also motivate those at high risk to seek appropriate evaluation, diagnosis, and treatment, if necessary. We convened a firefighter research advisory committee, made up of firefighter administration, union, line firefighters, and a fire department physician to assist in the development of the SHP. Using such a group to craft a sleep disorders risk notification that better resonates with firefighters might improve future outcomes.

Senior leaders’ enthusiasm and support of the union members were key to the success of the SHP. Memoranda were signed by each department before the study to demonstrate their commitment to the funding agency and most made the training (but not survey completion) mandatory. It may be possible for some departments to offer continuing education credits to further encourage participation. Firefighters in the focus groups emphasized the importance of having a champion within the department advocate for a novel program such as the SHP. A champion can informally build trust amongst peers, emphasize the importance of the program, alleviate fear, and guide overall perceptions. The ability to have questions about the SHP answered by investigators is also important. Even though there was a “Frequently Asked Questions” page available in the online program, perhaps the ability to directly ask and have answered specific questions about the program increased the participation in the EL and TT departments.

Interestingly, there was a reversal in participation on the end-of-program questionnaire with participation rates in OL departments twice that of EL and TT departments. This incongruity might be due to the fact that firefighters in OL departments were accustomed to participating in the SHP online, whereas most firefighters in EL and TT departments completed the screening survey on paper and were switched to a Web-based format at the end of the program. Consistency in delivery of the program may be an important consideration to prevent attrition. Alternatively, those firefighters in OL departments who completed the online screening survey may have been more self-motivated than those who were encouraged to complete the screening survey by in-person experts or trainers, and thus were willing to continue to complete online surveys. Further, technical issues may have also played a role in EL and TT departments (eg, correct e-mail addresses, retrieving lost passwords, and Web site time-outs), whereas technology was proven at the start of the study in OL departments. Finally, the more intensive interactions in the initial training, both for EL and TT, may have used up the participants’ capacity for this topic and they may have felt overburdened and consequently less responsive when approached a second time. Perhaps future programs should consider this balance when designing an SHP and plan to collect any required information on first contact with participants.

Challenges and Limitations

Implementing an SHP in operational fire departments presents operational implementation challenges and consequent research limitations. Although we selected “interested” fire departments and required signed memorandums of understanding for inclusion in the study, the resultant level of interest was not always easy to predict. In the course of implementing a program in an operational field setting, there may be changes in administrative personnel, climate, and workload. In addition, assignment of the fire departments to one of the three implementation methods based on resource availability represents an experimental design limitation; however, our approach was aimed at evaluating reach and effectiveness and was reflective of how future SHPs will be implemented.

It is challenging to schedule all firefighters in any department for training, as they may be called away due to operational duties and the different methodologies required different policies. During the educational training in EL and TT departments, firefighters were on-duty but off-service to ensure they could complete the in-person training. Firefighters participating in OL departments were typically on-service, and could thus be interrupted by a fire call while viewing the education or completing the screening. Those firefighters could return to the viewing or screening, but the extra step or potential difficulties logging back into the system likely led to lower cooperation rates.

In addition, despite our discouragement, some departments allowed for group viewing of the online educational video, which made tracking of individual participation more difficult. Group viewing of the educational training prevented firefighters from having the chance to immediately complete the screening survey, as individuals were not in front of an individual computer. This restriction may have reduced the participation in the screening survey in OL departments. Group viewing of the educational video may have affected attention of the firefighter, either positively or negatively.[25] Only eight simple questions on the before and after knowledge assessments were used in the analysis of learning. To achieve higher sensitivity and test the practical integration of the sleep health information, further assessments should use a greater number of questions and more difficult questions including firefighter-based scenario questions.

The benefits and challenges that we experienced in the implementation of a SHP in operational fire departments are similar to those seen in other industries. Nurses, who like firefighters work demanding schedules providing 24/7 coverage, identified greater awareness of sleep and fatigue issues and feeling more alert as benefits of an SHP.[15] Following the implementation of the nursing SHP, sleep (duration and quality), alertness, and error prevention were improved.[16] Confidentiality, changing the culture of the organization to embrace napping on-duty, and support of management for the program were challenges faced by hospital staff nurses.[15]

Screening for sleep disorders as part of an SHP introduces confidentiality concerns and potential anxiety, as untreated sleep disorders can impact employability in some occupations.[26] However, screening for common sleep disorders has been successfully accomplished in a large group of North American police officers,[5] and an occupational specific educational Web site with anonymous online sleep disorders screening was established for railroad workers.[13] The goal of any screening program is to achieve a high rate of those at risk to seek evaluation, clinical diagnosis, and treatment of sleep disorders. That was not achieved in any of the implementation modes of the firefighter SHP. More research is needed to understand how best to remove barriers to evaluation, clinical diagnosis, and treatment such as utilizing home-based diagnostic tools.

The existence and persistence of untreated sleep disorders increases the risk of crashes and injuries, adversely affects mental and physical health, including an increased risk of chronic diseases, including cardiovascular disease, obesity and diabetes, and decreases workplace productivity and quality of life.[3-8,27] It is therefore imperative to implement SHP in safety-sensitive occupations, especially in those with nonstandard work shifts, as chronic sleep disorders may affect 60 to 80% of all shift workers.[11]

CONCLUSION

The overall goal of this research was to compare the effectiveness of three different methods of implementation of a SHP. The program was well-received with three-quarters of firefighters participating. Firefighters in the EL departments gained the most knowledge and were most likely to seek further evaluation, diagnosis, and treatment if screened at a high risk for a sleep disorder. Focus groups identified communication, confidentiality, and firefighter culture as key challenges to implementation of SHPs and highlighted the fact that there is not a “one size fits all” program. Each department needs to decide which implementation method is best suited for conducting an SHP in order to successfully improve the health and safety of their employees.