Factor Analysis of the Insomnia Severity Index and Epworth Sleepiness Scale in Shift Workers.

BACKGROUND: Over one million Korean night shift workers undergo clinical assessment of sleep using the Insomnia Severity Index (ISI) and Epworth Sleepiness Scale (ESS) by occupational medical examination each year. Therefore, this study was conducted to evaluate the reliability and validity of the ISI and ESS using occupational medical examination data.
METHODS: The study subjects included 12,056 shift workers at an electronics company who underwent an occupational health examination about shift work in 2018. The evaluation of the ISI and ESS was performed using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).
RESULTS: According to the results of the EFA, the ISI had a single-factor structure, while the ESS had a two-factor structure, which was inconsistent with the findings of previous studies. The results of the EFA of 15 items from the combined ISI and ESS suggested a three-factor structure, with one factor for ISI items and two factors for ESS items, while the results of the CFA suggested sufficient validity of the combination of the ISI and ESS for sleep evaluation.
CONCLUSION: The results suggest that the ISI and ESS have sufficient reliability and validity to be used for occupational health examinations about shift work.

INTRODUCTION

There has been a remarkable increase in the number of shift workers worldwide, including Korea, due to changes in the industrial structure and growth of the service industry in recent decades.1 The negative health effects of shift work have been extensively investigated. Shift work has been found to be closely associated with the incidence of various chronic diseases, including hypertension, stroke, myocardial infarction, and diabetes23; in particular, sleep difficulty in shift workers has been demonstrated in a systematic review and is considered to be the most important health issue for these workers.4 Shift work is a direct cause of circadian rhythm disruption; therefore, shift work disorder is a subtype of circadian rhythm sleep-wake disorders.5 In other words, shift workers have difficulty sleeping because they have to work at times that do not fit their physiological or social circadian rhythms.6 As a result, job stress and accident risks increase,7 which leads to increased mortality.8 However, because shift work is considered an irreplaceable form of work in industries with a high degree of division of labor, the need for sleep management is growing.9

Korean workers who handle hazardous materials including 162 chemical factors (organic compounds, metals, acids, and alkalis), 7 dust factors (grain dust, mineral dust, cotton dust, wood dust, welding fumes, glass fiber dust, and asbestos) and 9 physical factors (noise, vibration, radiation, high pressure, low pressure, and ultraviolet, infrared, micro and radio waves), are required to undergo occupational health examinations periodically as per the Occupational Safety and Health Act. In particular, a special screening for shift workers, to be conducted by occupational health physicians, was newly added in 2014. For this, the shift workers must meet one of the following two criteria: 1) an 8-hour work shift (that must include the interval from 12 to 5 am) four or more times a month for six consecutive months, and 2) at least 60 average monthly hours spent working from 10 pm to 6 am for six consecutive months. The first round of screening includes assessment of the Insomnia Severity Index (ISI), abdominal circumference, blood pressure, fasting blood sugar, total cholesterol, triglycerides, high-density lipoprotein (HDL) cholesterol, and gastrointestinal-related symptoms, as well as questions regarding breast cancer-related symptoms. In the second round of screening, the examinees are followed up on the areas that showed abnormalities in the first screening using one or multiple tests, such as the Epworth Sleepiness Scale (ESS), the Pittsburgh Sleep Quality Index (PSQI), blood pressure, fasting blood glucose, glycated hemoglobin, total cholesterol, triglyceride, HDL cholesterol, LDL cholesterol, 24-hour ECG, 24-hour blood pressure, gastroscopy, breast imaging, and breast ultrasound. In general, the test examinees are required to complete all three sleep questionnaires during screening: the ISI, ESS, and PSQI.

Insomnia is the most common sleep disorder with a prevalence of 9%–30% in the general population.10 The ISI is a 7-item self-reported instrument that measures the patient's perception of insomnia severity.1112 The ESS was developed to measure “daytime sleepiness” in adults.13 Daytime sleepiness refers to an individual's average sleep tendency during the day.14 The ESS is the most widely used tool for measuring daytime sleepiness for clinical and research purposes.1516

The validity of the ISI has been demonstrated in various populations, including patients with cancer, older adults, and outpatients.171819 The validity of the ESS has also been demonstrated within populations, such as patients with sleep apnea syndrome and elderly people.2021 The ESS and ISI scores of shift workers were found to be different from those of day workers.22 However, none of these studies have evaluated the validity of both the ISI and ESS in shift workers. The PSQI was not included in the present study because the type and content of its questions are different from those of the ISI and ESS. The ISI and ESS continue to be used for the sleep evaluation of a substantial number of Korean shift workers. More than a million shift workers have undergone occupational health examinations for shift work, and the number is increasing.23 Thus, annually over a million shift workers are evaluated using the ISI and ESS, and the social cost is approximately 40 billion won. Therefore, this study about the validity of the ISI and ESS based on large-scale health screening data from shift workers is very important.

METHODS

Subjects

This study included 12,056 shift workers at an electronics company that conducted occupational health examinations for these workers in 2018. Subjects completed the sleep questionnaires offline or online.

Measures

The Korean version of the ISI was used to assess the subjective severity of insomnia.24 The ISI consists of seven items on a five-point Likert scale (0–4) with a total score between 0 and 28.11 The Korean version of the ESS was used to assess daytime sleepiness.16 The ESS comprises eight items on a four-point Likert scale (0–3) with a total score between 0 and 32.13

Statistical analysis

The reliability of the ISI and ESS was evaluated by estimating internal consistency using Cronbach's alpha. The factorial validity of the ISI and ESS was assessed using exploratory factor analysis (EFA; principal axis factoring) with promax rotation. The number of factors was determined based on eigenvalues (> 1), which represents the variance explained by each factor, as well as the coherence and interpretability of the factors. The factors that were identified by the EFA were tested using confirmatory factor analysis (CFA). Model fit was evaluated using criteria based on fit indices, such as RMR < 0.05, GFI > 0.90, RMSEA < 0.08, NFI > 0.9 and AGFI > 0.90.25 All analyses were conducted with PASW Statistics for Windows, Version 18.0 and AMOS 18.0 (IBM Co., New York, NY, USA).

Ethics statement

The study protocol was reviewed and approved by the Institutional Review Board (IRB) of Kangbuk Samsung Hospital (IRB approval No.2019-06-022). Because only anonymous data with no identifiable personal information were used, prior consent was waived for the participants.

RESULTS

Overall, 43.7% of the subjects were women, and 56.3% were men. The overall mean age was 28.75 ± 8.26. The mean ages of the women and men were 28.58 ± 7.98 and 28.89 ± 8.47 years, respectively. The men were significantly older than the women (P = 0.042).

The mean ISI and ESS scores were 11.88 ± 4.42 and 13.76 ± 3.46, respectively. The prevalence of insomnia was 25.0% when the cutoff value was set at 15.26 The share of those with more than 9 points on the ESS was 89.4%.27 The values of Cronbach's alpha for the ISI and ESS were 0.888 and 0.817, respectively, suggesting that both tests have sufficient internal consistency. The item-total correlations of the ISI and ESS ranged from 0.41 to 0.71 (mean, 0.54) and from 0.20 to 0.55 (mean, 0.36), respectively.

Principal component analysis found only one factor in the ISI with eigenvalues > 1. Factor 1 accounted for 60.3% of the variance. The Kaiser-Meyer-Olkin value (0.876) and the results of Barrett's test of sphericity (P < 0.001) indicated that the factor structure was appropriate (Table 1).

Table 1

Factor structure and loadings of 7 items from ISI in shift workers (n = 12,056)

Variables	Items	Communalities	F1
ISI7	Waking up early in the morning	0.686	0.828
ISI2	Insomnia	0.667	0.817
ISI5	Waking up during the night	0.632	0.795
ISI1	Time to fall asleep	0.599	0.774
ISI3	Difficulty falling asleep	0.562	0.750
ISI4	Disturbed night sleep	0.558	0.747
ISI6	Morning tiredness	0.520	0.721
Eigen value			4.224
% of variance			60.336
Cumulative % of variance			60.336

ISI = insomnia severity index.

KMO = 0.876, Bartlett's χ2 = 45,069.988 (P < 0.001).

Principal component analysis with promax rotation found 2 factors in the ESS with eigenvalues > 1. The same number of factors was indicated by the scree plot. Factor 1 accounted for 44.4% of the variance and comprised lying down, car passenger, sitting quietly, and sitting reading. Factor 2 accounted for 12.9% of the variance and comprised sitting talking, watching TV, car in traffic, and sitting in public. The Kaiser-Meyer-Olkin value (0.857) and the results of Barrett's test of sphericity (P < 0.001) indicated that the factor structure was appropriate (Table 2).

Table 2

Factor structure and loadings of 8 items from ESS in shift workers (n = 12,056)

Variables	Items	Communalities	F1	F2
ESS5	Lying down	0.753	0.858	0.339
ESS4	Car passenger	0.612	0.782	0.392
ESS7	Sitting quietly	0.581	0.757	0.470
ESS1	Sitting reading	0.487	0.611	0.607
ESS6	Sitting talking	0.620	0.236	0.764
ESS2	Watching TV	0.541	0.437	0.732
ESS8	Car in traffic	0.481	0.374	0.693
ESS3	Sitting in public		0.542	0.686
Eigen value			3.554	1.038
% of variance			44.424	12.976
Cumulative % of variance			44.424	57.400

Salient loading values are printed in bold. Rotation method: Promax rotation.

ESS = Epworth sleepiness scale.

KMO = 0.857, Bartlett's χ2 = 26,678.092 (P < 0.001).

Principal component analysis with promax rotation found 3 factors in the combined ISI and ESS of 15 items with eigenvalues > 1. The same number of factors was indicated by the scree plot. Factor 1 accounted for 36.5% of the variance and comprised 7 items of the ISI. Factor 2 accounted for 15.7% of the variance and comprised lying down, car passenger, and sitting quietly. Factor 3 accounted for 6.9% of the variance and comprised sitting talking, watching TV, car in traffic, sitting in public, and sitting reading. The Kaiser-Meyer-Olkin value (0.895) and the results of Barrett's test of sphericity (P < 0.001) indicated that the factor structure was appropriate (Table 3).

Table 3

Factor structure and loadings of combined ISI and ESS 15 items in shift workers (n = 12,056)

Variables	Items	Communalities	F1	F2	F3
ISI2	Insomnia	0.686	0.823	0.160	0.304
ISI7	Waking up early in the morning	0.681	0.823	0.306	0.335
ISI5	Waking up during the night	0.637	0.786	0.362	0.379
ISI1	Time to fall asleep	0.616	0.781	0.158	0.251
ISI3	Difficulty falling asleep	0.583	0.756	0.126	0.282
ISI4	Disturbed night sleep	0.580	0.746	0.335	0.261
ISI6	Morning tiredness	0.517	0.715	0.196	0.329
ESS5	Lying down	0.740	0.206	0.852	0.336
ESS4	Car passenger	0.604	0.231	0.777	0.387
ESS7	Sitting quietly	0.576	0.244	0.753	0.468
ESS6	Sitting talking	0.618	0.286	0.236	0.764
ESS2	Watching TV	0.533	0.328	0.434	0.725
ESS8	Car in traffic	0.487	0.259	0.371	0.697
ESS3	Sitting in public	0.516	0.288	0.548	0.680
ESS1	Sitting reading	0.491	0.239	0.607	0.610
Eigen value			5.474	2.360	1.030
% of variance			36.496	15.736	6.866
Cumulative % of variance			36.496	52.232	59.097

Salient loading values are printed in bold. Rotation method: Promax rotation.

ISI = insomnia severity index, ESS = Epworth sleepiness scale.

KMO = 0.895, Bartlett's χ2 = 74,763.616 (P < 0.001).

The correlations between three factors of the ISI and ESS were significant at 0.285, 0.398, and 0.573 (P < 0.01) (Table 4).

Table 4

Correlation among extracted factors of ISI and ESS and PSQI component in shift workers (n = 12,056)

Factors	F1	F2	F3
F1	1
F2	0.285a	1
F3	0.398a	0.573a	1

aCorrelation is significant at the 0.01 level (2-tailed).

The fit indices indicated a good model fit (RMR, 0.027; GFI, 0.902; and RMSEA, 0.079).25 The values of the AGFI and NFI were less than 0.9 (0.864 and 0.888, respectively) yet were similar to each other. All three absolute fit indices, RMR, GFI, and RMSEA, met the appropriate criteria, suggesting that the model fit is acceptable.

The validity of the combination of the ISI and ESS measures was evaluated based on coefficients for construct, convergent, and discriminant validity. The standardized coefficients for observed variables comprising Factors 1, 2, and 3 ranged from 0.535 to 0.803, thereby exceeding the criterion of 0.5 and suggesting that the combined measure has sufficient construct validity (Fig. 1). The average variance extracted (AVE), used to estimate convergent validity, for the three factors was 0.636, 0.638, and 0.653, exceeding the criterion of 0.5. The construct reliability coefficients were 0.924, 0.840, and 0.903, exceeding the criterion of 0.7, and suggesting that the combined measure has sufficient convergent validity (Table 5).28 The squared correlation coefficients between the individual factors were 0.120, 0.239, and 0.554, which were lower than the AVEs for individual factors and suggested that the combined measure had sufficient discriminant validity (Table 6).29

Fig. 1

Path representation of the proposed model of ISI and ESS.

ISI = insomnia severity index, ESS = Epworth sleepiness scale.

Table 5

Validity of confirmatory factor analysis of the ISI and ESS

Factors		Standardized estimates	AVE (variance extracted index)	CR
Factor 1 → ISI			0.636	0.924
	Factor 1 → ISI 2	0.766
	Factor 1 → ISI 7	0.803
	Factor 1 → ISI 5	0.768
	Factor 1 → ISI 1	0.721
	Factor 1 → ISI 3	0.690
	Factor 1 → ISI 4	0.701
	Factor 1 → ISI 6	0.678
Factor 2 → ESS			0.638	0.840
	Factor 2 → ESS 7	0.706
	Factor 2 → ESS 4	0.673
	Factor 2 → ESS 5	0.758
Factor 3 → ESS			0.653	0.903
	Factor 3 → ESS 1	0.649
	Factor 3 → ESS 3	0.668
	Factor 3 → ESS 8	0.550
	Factor 3 → ESS 2	0.661
	Factor 3 → ESS 6	0.535

ISI = insomnia severity index, ESS = Epworth sleepiness scale, AVE = average variance extracted, CR = construct reliability.

Table 6

Discriminant validity of confirmatory factor analysis of the ISI and ESS

Factors	Correlation		AVE	CR
	Factor 1	Factor 2
Factor 1 (ρ2)	1		0.636	0.924
Factor 2 (ρ2)	0.120	1	0.638	0.840
Factor 3 (ρ2)	0.239	0.554	0.653	0.903

ISI = insomnia severity index, ESS = Epworth sleepiness scale, AVE = average variance extracted, CR = construct reliability.

DISCUSSION

Factor analysis is a good method to evaluate the validity of questionnaires.3031 The study findings suggest that the ISI and ESS are reliable and valid instruments for sleep evaluation in shift workers. In particular, the results of the factor analysis of the combination of the ISI and ESS measures, including the identification of three factors, one factor for the ISI items and two factors for the ESS items, and sufficient levels of validity in varying subtypes, justify the use of the two measures for occupational health examination in shift workers.

In our study, the mean ISI and ESS scores were 11.88 and 13.76, respectively, which are far higher than those in healthy Korean adults32 as well as in the general population in other countries.33 As expected, the overall sleep conditions of shift workers were poor.

Previous studies on the factor structure of the ISI have yielded varied and inconsistent findings. A previous study of patients with insomnia identified three factors, night-time sleep difficulties, sleep dissatisfaction, and daytime impact of insomnia, which explained 72% of the total variance. The ISI score in a study composed of insomnia patients was 19.7.11 A study of patients with breast or prostate cancer presented two factors, night-time sleep difficulties and daytime impact of insomnia, which explained approximately 60% of the total variance. The ISI score of that prior study was 7.3.17 A study of older adults aged 56–87 demonstrated that a single factor explained 69% of the total variance. The mean age in that previous study was 71 years.18 A study of adolescents demonstrated that two factors explained 65.6% of the total variance. The ISI score of that study was 7.4, and the study subjects were 11–20 years old.34 A study of the general population demonstrated three factors. The ISI score of that study was 9.1, and the study subjects were recruited from schools, communities and hospitals.35 A study of university students and police officers demonstrated a single factor structure. The ISI scores of the university students and police officers were 6.56 and 6.98.36 In the present study of shift workers, the ISI was found to have a single-factor structure. Previous studies on a wide range of subjects in both the West and East reported variation across different types of subjects. No difference in factor structures was found between the general population of Hong Kong and Canada, which suggests that the ISI has cross-cultural comparability and is a valid construct for the measurement of subjective insomnia.35 In other words, the differences in the ISI's factor structures across studies are likely due to the various characteristics of the subjects, such as age, job and disease history, rather than regional or cultural differences. The current study was conducted with shift workers and young workers. The mean age and ISI score were 29 years and 11.88, respectively. The mean ISI score was higher for shift workers than for day workers, and higher subjective insomnia was associated with various health problems in shift workers.37 The ISI score of shift workers in this study was higher than that in other studies, including university students and young workers,36 except for insomnia patients. The subjects of this study consisted of only shift workers from a large electronics company. It is believed that the homogeneity of the working condition of subjects showed a single factor structure.

The ESS is a questionnaire designed to evaluate patients' habitual sleepiness during the day, and is used worldwide.13 Other EFA studies on the ESS factor structure have shown a single-factor structure.38394041 The ESS scores of patients who visited the daytime sleepiness clinic, patients with a variety of sleep disorders, patients with sleep apnea and medical students were 10.5, 10.2, 12.1 and 7.6.38394041 The ESS score of this study was 13.8, which is higher than that in previous studies. In contrast, the present study found a two-factor structure in which items 1, 4, 5, and 7 and items 2, 3, 6, and 8 formed factor structures. Item 1 had nearly equal loading on Factors 1 and 2. Factor 1 included lying down, car passenger, and sitting quietly, indicating resting without engaging in a particular activity. The variance explained by Factor 1 was 52.232%, accounting for most of the total variance (59.097%). Factor 2 included sitting talking, watching TV, car in traffic, and sitting in public, which can be summarized as engaging in a particular activity, such as watching something, talking, and driving. Shift workers were found to have greater daytime sleepiness and higher ESS scores than patients with sleep disorders in the present study. Generally shift workers experienced excessive daytime sleepiness more often than the general population too.4243 This due to the changes in circadian rhythms and sleep patterns associated with shift work.44 Daytime sleepiness of shift workers is associated with decreased concentration, which increases the risk of accidents.45 Because of the excessive daytime sleepiness and decreased concentration of shift workers, the difference in daytime sleepiness between sitting quietly and doing something is much larger in shift workers than in the general population. Due to this characteristic of shift workers, the ESS showed a two-factor structure, unlike in previous studies.

Over 2 million workers underwent occupational health examinations in Korea in 2017. Approximately 1.05 million of them underwent occupational health examinations for shift work, and the number increases annually.23 Annually, over a million shift workers have been evaluated for sleep problems using the ISI, ESS, and PSQI in Korea. Based on the results, workers assessed as being in poor condition receive counseling and are sometimes referred to a psychiatric outpatient unit. Despite of the importance of these measures and the need to establish their use as measures for sleep evaluation, research aiming to validate the ISS and ESS for shift workers has been scarce; hence, our study particularly focuses on this area.

In the present study, a novel attempt was made to evaluate the potential of the sleep questionnaires used for the occupational health examination of shift workers by performing a factor analysis of the combined ISI and ESS measures. A previous study using factor analysis of the combined two measures represented good results.46 The EFA results for the combined measure were nearly identical to those obtained for the ISI and ESS individually. The 15 items of the combined ISI and ESS measures were classified into three factors: Factor 1, comprising seven ISI items; Factor 2, comprising three ESS items (sitting quietly); and Factor 3, comprising five ESS items (sitting and doing something). The variance explained by the three factors was 59.097%, indicating excellent explanatory power. The CFA also yielded results that met the criteria for construct, convergent, and discriminant validity, along with the standardized estimates between individual factors and items in the range of 0.54–0.80, which indicates sufficient explanatory power. (Tables 5 and 6) In summary, the sleep measures used for the occupational health examination of shift workers have sufficient validity and comprise appropriate items.

There are some limitations in the present study. The subjects are from one large electronics company and there is no data about the type of work. There are various types of shift work schedules including evening shift, night shift, regular day and night shift, 24 hours shift and others. To represent all shift workers, it is necessary to study various types of companies, occupations and shift work schedules. The study is also limited by the inability to compare with day workers because there is no control group of day workers who took the ISI and ESS. However, this subject sample contains more than 10 thousand individuals, representing approximately 1% of all Korean shift workers, and the sample size is meaningful.

The difference between the factors identified in this study and those in previous studies is likely due to the high risk of sleep disorders in shift workers. Future studies may include comparative analyses with additional variables, such as the type and duration of shift work and the gender of workers, to generate sufficient data to develop strategies to manage and treat shift work sleep disorders. This study provides evidence of the validity and reliability of the ISS and ESS instruments used to evaluate sleep disorders in shift workers, thereby providing a foundation for future research in this field.