Psychometric Validation of the Athens Insomnia Scale Among Nurses: A Robust Approach Using Both Classical Theory and Rating Scale Model Parameters.

Purpose: Few studies have investigated the validity of the Athens insomnia scale (AIS) using a robust approach of both classical theory and the rating scale model. Therefore, in this study, we investigated psychometric validation of the AIS using both of these approaches in nurses.
Methods: Nurses (n= 563, age= 33.2±7.1 years) working in health facilities in Saudi Arabia participated in a cross-sectional study. Participants completed the AIS, socio-demographics tool, and sleep health-related questions.
Results: Confirmatory factor analysis (CFA) favored a 2-factor structure with both comparative fit index (CFI), and incremental fit index (IFI) having values above 0.95. The 2-factor model had the lowest values of Akaike information criterion (AIC), root mean square error of approximation (RMSEA), χ 2, and χ 2/df. This 2-factor structure showed configural invariance (CFI more than 0.95, RMSEA less than 0.08, and Χ2/df less than 3), and metric, scalar, and strict invariance (based on Δ CFI ≤-0.01, and Δ RMSEA ≥ 0.015 criteria). No ceiling/floor effects were seen for the AIS total scores. Infit and outfit mean square values for all the items were within the acceptable range (<1.4, >0.6). The threshold estimates for each item were ordered as expected. Cronbach's α for the AIS tool, factor-1 score, factor-2 score was 0.86, 0.82, and 0.72, respectively. AIS factor scores-1/2 were significantly associated with a habitual feeling of tiredness after usual night sleep (p<0.001), Impairment of daytime socio-occupational functioning (p<0.05), and with a feeling of daytime fatigue, irritability, and restlessness (p<0.05).
Conclusion: The findings favor the validity of a 2-factor structure of the AIS with adequate item properties, convergent validity, and reliability in nurses.

Introduction

Problems in sleep initiation and maintenance characterize insomnia, despite sufficient opportunity to do so, and are often associated with marked daytime impairments. The economic burden of insomnia is substantial; the inflation-adjusted annual estimate for the United States in 2016 was nearly one percent of the gross domestic product.1 A recent study based on a large nationally representative sample in the US found that untreated insomnia increased utilization and higher overall costs of healthcare services among older Medicare beneficiaries.2 Healthcare facilities cannot meet the challenges of insomnia treatment because of the problems of under-diagnosis, and consequent under-treatment. Insomnia under-diagnosis is often played up by its trivialization and underestimating its consequences by primary care physicians.3 Therefore, more streamlined and targeted efforts are imperative to address the challenges of mishandled and under-diagnosed insomnia at the primary care level.4

Insomnia prevalence is consistently showing upward trends in all those countries where population-based longitudinal studies were performed.5 These countries are spread across three continents; South Korea (from 3.10% in 2002 to 7.20% in 2013), Taiwan (from 2.47% in 2002 to 4.17% in 2009), Finland (from 9.0% in 2007 to 9.6% in 2012), Norway (from 11.9% in 2000 to 15.5% in 2010), and US (from 17.5% in 2002 to 19.2% in 2012).6–10 Insomnia in Saudi Arabia is one of the most common sleep disorders.5,11–13 Insomnia has also been investigated among Saudi nurses, and primary care patients in the country.5,11–13 Saudi healthcare delivery relies on a referral system starting with initial enrolment and patient presentation at primary health care centers. However, physicians at these primary care facilities are often not trained and well aware of the diagnostic criteria of sleep disorders, including insomnia. This usually results in overlooking and under-treating insomnia.11 Furthermore, the unavailability of a national clinical guideline together with a lack of awareness and training about holistic management of insomnia even in tertiary care physicians, indicate a necessity to develop practical solutions for advancing sleep health care in the country.12 In such circumstances, the availability of an easily interpretable tool for screening insomnia that has been rigorously validated in the Saudi population may help further the cause of sleep health management at the primary care level.

Athens Insomnia Scale (AIS) is a brief and easily interpretable measure of insomnia. It is a commonly used screening tool for insomnia with eight questions.14 The AIS has shown promising psychometric measures in Greek outpatients visiting primary healthcare.15 The psychometric properties of the AIS have not been investigated in detail in any demographics of the Saudi population, nor has the scale been characterized in nurses. Awadalla and Al-Musa reported an internal reliability measure based on the Cronbach’s alpha in a sample of primary care patients from Saudi Arabia.13 A large-scale study conducted on a nationally representative sample involving 1,000,000 individuals in Taiwan, found that the incidences of adjusted and psychophysiological insomnia were higher in female nurses than other health professionals, and non-health professionals.16 Nurses have a high prevalence of sleep disorders, including insomnia and associated conditions that predispose them to poor sleep. Therefore, in this study, a robust investigation of the psychometric properties of the AIS using classical theory and item response theory parameters was performed. We hypothesized that the psychometric validity of the AIS using the rigorous and robust measures of both classical theory and the Rasch rating scale parameters would be adequate.

Materials and Methods

Procedures

This was a cross-sectional study with a convenient sampling method. Two modes were used during data collection; an online link using a google survey and a paper-based study depending on the participant’s comfort and preference. Participants were briefly informed about the purpose of the study. In the online version, a brief introductory note about the study was included. Text in this introductory section mentioned the following: consent to participate and publish, voluntary nature, no potential risk, complete freedom to withdraw at any stage, and strict arrangement to keep personal information confidential. Participants were given contact details of a researcher to communicate with if they had any queries and doubts.

Similarly, the participants were given a hard copy of the introductory note in the offline survey with all those details mentioned above for the online survey. Written consent was provided by the participants in the offline version, while online survey participants’ consent was recorded by a ‘yes’ response for the consent item. Participants were recruited by trained data collectors who were enrolled in the Master of Nursing program at Majmaah University. For online recruitment, the survey link was shared through e-mail, social messaging networks, etc. The research procedures followed the Helsinki declaration, 2002. The study was approved by the Human Ethics Committee (H-05-FT-083), Ministry of Health, Saudi Arabia. Participants completed a survey questionnaire package in English containing the AIS, semi-structured questionnaires with items recording information related to socio-demographics, habits, and subjective sleep accounts.

Participants

The study participants included nurses working in government and private hospitals and health centers in Saudi Arabia. A total of 563 nurses with an average age of 33.2±7.1 years participated in this study from December 2020 to April 2021. Participating nurses were required to have a valid practice license in Saudi Arabia as indicated by possession of a certificate by the Saudi Commission for Health Specialists. There were no other inclusion and exclusion criteria. The participating nurses were of various nationalities such as Saudis, Indians, Filipinos, Pakistanis, Sudanese, and other Arab nationalities.

In this study, the sample size was determined by the requirements of the factor analysis. Though, this issue of an adequate sample size for factor analysis does not have consensus. However, the employed statistical approach has been recently supported by a systematic review and meta-analysis on the factor analysis of a widely used insomnia questionnaire.17 Moreover, this is one of the most robust recommendations based on the considerations of communality, model error (maybe ignored for practical purposes in empirical datasets), and over-determination: this is based on factor loadings, and a p/r ratio (where p stands for the total number of items in the questionnaire, and r stands for the number of factors.18,19 There are a few previous studies on the factor analysis of the AIS that have published this relevant information. Sirajudeen et al reported that the AIS items showed communality values below 0.5, with some as low as 0.37, and a moderate average factor loading score of 0.58.20 Moreover, previous studies, including that by Sirajudeen et al 2020 have found that the AIS usually has a 2/3-factor structure.20 Therefore, this means that factors of the AIS have 2–4 determining/indicator items. In such a situation of items with low communality score, low determination (2–4 indicator items for each factor), and moderate factor loadings, a sample size of 563 was more than the recommended sample size of 300.19

Few missing values were seen in the socio-demographics, such as age (0.2%) and BMI (0.4%).

Measures

Athens Insomnia Scale (AIS)

The AIS is a brief self-reported questionnaire to assess the severity of insomnia according to the International Classification of Diseases, Tenth Revision (ICD-10) diagnostic criteria.14 Eight questions assess sleep characteristics: sleep induction, awakenings at night, final awakening, sleep duration, sleep quality, and daytime symptoms: wellbeing, functioning, and sleepiness.14,15 All these questions are scored on a Likert scale with a score range of 0 (no sleep/daytime problems) to 3 (acute sleep/daytime difficulties). The total score for the scale is generated by linearly adding scores of individual questions; higher scores indicate increasing severity of insomnia symptoms.14,15 The AIS has been validated in Asian populations, such as Indian computer professionals.20

Socio-Demographic Questionnaire

A list of questions was used to record responses about socio-demographics characteristics. The questions required participants to record their age, sex, height, weight, clinical duty schedules, habitual sports activity, habitual tea/coffee consumption, smoking (cigarette, hookah, shisha, etc.).

Sleep Health Questions

Three dichotomous questions were used to record tiredness after sleep, daytime socio-occupational problems related to sleep problems, and daytime fatigue and restlessness due to sleep disturbances. These questions were administered to assess the convergent validity of the AIS.

Statistical Analysis

All statistical analysis was performed by SPSS version 26.0 and R software package version 4.0.2. Participants’ characteristics, classical theory psychometric properties such as factor analysis, convergent validity, reliability, skewness, kurtosis, and percentage were assessed by SPSS. All the AIS items are Likert scale type scores with 0 to 3; therefore, the Polytomous Rating Scale Model was used. Parametric item response theory (IRT) analysis properties such as item difficulty, Outfit, Infit, and Thresholds (τi1, τi2, τi3) were determined.21 Furthermore, Item Characteristic Curves (ICCs) and Wright map: Person-item distribution were used to show the graphical representation of the estimated item parameters.21 Internal consistency was assessed by Cronbach’s alpha and Spearman correlation. Binary logistic regression was used to determine convergent validity.

For factor analysis, the study sample was split into two halves for carrying out exploratory factor analysis (EFA) and Confirmatory factor analysis (CFA) on two distinct sub-samples. Both the sub-samples satisfied conditions of factor analysis as indicated by the values of the diagonal elements of the anti-image matrix, Bartlett’s Test of Sphericity, Determinant, Kaiser-Meyer-Olkin Test of Sampling Adequacy (KMO), Communality and inter-item correlation coefficients ( and ). Absence of multicollinearity in both sub-samples was indicated by determinant scores above 0.00001 ().22 All the individual AIS items scores were sufficiently correlated with each other to be included in the factor analysis as indicated by values of all the diagonal elements of the anti-image correlations, more than 0.5, in both sub-samples (). Meritorious level of shared variance in the item scores of the AIS was indicated by KMO values of 0.86, and 0.87 in the two sub-samples ().22 Bartlett’s test of Sphericity (p<0.001) implied that the items of the AIS are not orthogonal and share sufficient correlations; therefore, factor analysis is feasible for both sub-samples ().22 In both sub-samples, all the communality scores were above 0.2; this suggested that common factor scores explained an adequate proportion of variance.23 Most inter-item correlations coefficients were above 0.3, in both sub-samples, such correlation matrices are expected to yield a viable factor analysis solution.24 Principal Component Analysis (PCA) extraction for the unrotated solution was performed on the EFA sub-sample. CFA was performed on the second sub-sample with Maximum likelihood extraction with bootstrap for standardized loadings. In the CFA, four distinct models were tested: model-A: 1-Factor,25 Model-B: 2-Factor model,26,27 Model-C: 2-Factor,28 and Model-D: 3-Factor model.20 For comparison between different models, six model fit indices belonging to different categories were determined.29–31 Of these, χ2, χ2/df, and Akaike Information Criterion (AIC) belonged to the Absolute fit indices category.

Root mean square error of approximation (RMSEA) and comparative fit index (CFI) are non-centrality-based Indices, and Incremental Fit Index (IFI) is a relative Fit Index.32 A value above 0.95 for CFI and IFI indicated best fit.33 For RMSEA, a value of 0.08, χ2/df less than 3, and lower AIC values (compared to other models) were taken to indicate a better fit.33 These criteria were used for establishing measurement invariance of the AIS factor structure: (i) configural invariance was indicated by CFI more than 0.95, RMSEA less than 0.08, and Χ2/df less than 3,34 (ii) Metric/scalar/strict invariances were indicated by more general cut-off values of Δ CFI≤-0.01, regardless of sample size variation, group size equality condition, and type of invariance.34

Results

Participants’ Characteristics

The majority of the participating nurses (59.9%) reported an age range of 25–34 years (Table 1). More than 2/3rd of the nurses (79.0%) were females. More than 15% of nurses were obese according to the World Health Organization’s body mass index criteria. More than 2/5th of the nurses (41.7%) worked in rotating shifts or night shifts. More than 2/5th of the nurses (41.2%) reported that they had never-rare habitual sports activity or strenuous exercises. Habitual tobacco smoking was recorded in 8.2% of the nurses. The majority of the nurses (50.3%) reported feeling tired after usual sleep, while 41.6% felt fatigued, irritable, and restless during daytime (Table 1).

Table 1

Participant Characteristics

Characteristics	Mean ± SD/Frequency
Age (yr)	33.2±7.1 (20–62)
18–24	17 (3.0)
25–34	337 (59.9)
35–49	189 (33.6)
50–64	19 (3.4)
Did not report	1 (0.2)
Gender
Male	116 (20.6)
Female	445 (79.0)
Prefer not to say	2 (0.4)
BMI
Underweight (less than 18.49 kg/m2)	35 (6.2)
Normal weight (18.5 to 24.9 kg/m2)	277 (49.2)
Overweight (25.0 to 29.9 kg/m2)	162 (28.8)
Obesity class I (30.0 to 34.9 kg/m2)	58 (10.3)
Obesity class II (35.0 to 39.9 kg/m2)	20 (3.6)
Obesity class III (more than 40 kg/m2)	9 (1.6)
Did not report	2 (0.4)
Clinical duty schedules
Morning shift	319 (56.7)
Afternoon shift	9 (1.6)
Night shift	8 (1.4)
Rotating shift	227 (40.3)
Habitual sports activity or strenuous exercise
Never	99 (17.6)
Rarely	133 (23.6)
Sometimes	232 (41.2)
Very often	44 (7.8)
Always	55 (9.8)
Habitual tea/coffee consumption
Yes	532 (94.5)
No	31 (5.5)
Habitual tobacco smoking (cigarette, hookah, shisha, etc.)
Yes	46 (8.2)
No	517 (91.8)
Feeling of tiredness after usual night sleep
Yes	283 (50.3)
No	280 (49.7)
Impairment of daytime socio-occupational functioning
Yes	127 (22.6)
No	436 (77.4)
Feeling of daytime fatigue, irritability, and restlessness
Yes	234 (41.6)
No	329 (58.4)

Abbreviations: SD, standard deviation; BMI, body mass index.

Factor Analysis

PCA of the AIS

All the four measures (ie, Parallel analysis, Eigenvalue more than 1 criterion, cumulative variance, and scree plot) used to determine the number of factors in the PCA indicated that a 1-Factor solution of the AIS is the most suitable in this study population (Figure 1; ).31,32,35 The component loading scores in the PCA sub-sample ranged from 0.59 to 0.81; therefore, all the AIS items scores met the minimal level of correlation with the latent construct as required by the recommendation of Comrey and Lee36 (Table 2).

Figure 1

Parallel analysis sequence plot of the Athens insomnia scale (AIS) scores in nurses. Monte Carlo parallel analysis with principal components and random normal data generation.

Table 2

Descriptive Statistics of the Athens Insomnia Scale (AIS) Scores in Nurses

Items/Factors of the AIS	Cronbach’s α if Item Deleted		Item-Factor Correlation		Corrected Item-Factor Correlation		Component Loadings#	Communality#	Mean ± SD	Skewness		Kurtosis		Percentage Distribution Across Item Scores
										Statistic (SE)	z	Statistic (SE)	z	0	1	2	3
	Factor-1	Factor-2	Factor-1	Factor-2	Factor-1	Factor-2
Sleep Induction	0.81		0.68*		0.52		0.59	0.35	0.94±0.90	0.63 (0.10)	6.13	−0.47 (0.21)	−2.26	37.5	37.5	19.0	6.0
Awakenings during the night	0.79		0.75*		0.60		0.69	0.48	0.90± 0.89	0.66 (0.10)	6.41	−0.46 (0.21)	−2.23	39.6	36.1	18.8	5.5
Final awakening earlier than desired	0.80		0.68*		0.56		0.69	0.48	0.93±0.84	0.63 (0.10)	6.16	−0.18 (0.21)	−0.87	33.6	44.4	17.1	5.0
Total sleep duration	0.76		0.80*		0.69		0.79	0.62	0.86±0.85	0.69 (0.10)	6.73	−0.32 (0.21)	−1.53	40.3	38.0	17.4	4.3
Overall quality of sleep	0.76		0.80*		0.70		0.81	0.65	0.81±0.87	0.80 (0.10)	7.81	−0.18 (0.21)	−0.87	43.9	35.7	15.8	4.6
Sense of well-being during the day		0.58		0.77*		0.57	0.75	0.56	0.49±0.78	1.61 (0.10)	15.66	1.98 (0.21)	9.63	65.2	24.0	7.3	3.6
Functioning (physical and mental) during the day		0.50		0.74*		0.64	0.73	0.53	0.39±0.70	1.99 (0.10)	19.30	3.70 (0.21)	17.98	71.6	21.0	4.8	2.7
Sleepiness during the day		0.78		0.78*		0.41	0.61	0.37	0.83±0.75	0.72 (0.10)	7.01	0.35 (0.21)	1.71	35.3	49.6	12.1	3.0
Factor-1									4.44±3.31	0.75 (0.10)	7.23	0.33(0.21)	1.61
Factor-2									1.71±1.79	1.49(0.10)	14.50	2.64(0.21)	12.80
AIS total score									6.15±4.67	0.94 (0.10)	9.15	0.98 (0.21)	4.78

Notes: *p<0.01; #Based on Principal Component Analysis extraction for unrotated solution of the sub-sample used in the exploratory factor analysis. All other values are for the total sample size.

Abbreviations: SD, standard deviation; SE, standard error.

CFA of the AIS Models

A 2-factor structure (Model-C) was favored by the fit indices in the CFA (Table 3). Model-C had CFI and IFI values of more than 0.95. Model-C had the lowest values of AIC, RMSEA, χ2, and χ2/df. All the factor loadings (range: 0.45–0.89) in the Model-C were above the minimum criteria36 (Figure 2).

Table 3

Fit Statistics of the Athens Insomnia Scale (AIS) Scores in Nurses

Models	CFI	IFI	AIC	RMSEA	χ2	df	p	χ2/df
A	0.893	0.894	167.946	0.134 (0.111–0.157)	119.946	20	0.000	5.997
B	0.904	0.905	158.323	0.130 (0.106–0.154)	108.323	19	0.000	5.701
C	0.966	0.966	100.697	0.077 (0.052–0.103)	50.697	19	0.000	2.668
D	0.967	0.967	101.869	0.081 (0.054–0.108)	47.869	17	0.000	2.816

Notes: A: 1-Factor 25, B: 2-Factor model 26,27, C: 2-Factor28 ; D: 3-Factor model20 .

Abbreviations: CFI, comparative fit index; IFI,  Incremental Fit Index; AIC, Akaike information criterion; RMSEA, root mean square error of approximation.

Figure 2

Confirmatory factor analysis models of the Athens Insomnia Scale (AIS) scores in nurses. (A) 1-Factor,25 (B) 2-Factor model,26,27 (C) 2-Factor28 and (D) 3-Factor model;20 F1–F3: factors of the AIS models A/B/D; factors of Model-C: Night symptoms, and Daytime symptoms, All coefficients are standardized. Ovals latent variables, rectangles measured variables, circles error terms, single-headed arrows between ovals and rectangles factor loadings, single-headed arrows between circles and rectangles error terms.

Measurement Invariance of the 2-Factor Structure (Model-C) Among Gender Groups

Model-C of the AIS showed configural invariance when compared across gender groups among participating nurses with constraints: CFI more than 0.95, RMSEA less than 0.08, and Χ2/df less than 3 (Table 4).34 The results favored metric invariance (model constrained for loadings versus fully unconstrained model) of this 2-factor structure (Model-C): non-significant ΔΧ2, Δ CFI ≤-0.01, and Δ RMSEA ≥ 0.01534,37 (Table 4). The results favored scalar invariance (model constrained for intercepts versus models constrained for loadings) of this 2-factor structure (Model-C): Δ CFI ≤-0.01, and Δ RMSEA ≥ 0.01534,37 (Table 4). The results favored strict invariance (model constrained for variances versus models constrained for intercepts) of this 2-factor structure (Model-C): Δ CFI ≤-0.01, and Δ RMSEA ≥ 0.01534,37 (Table 4).

Table 4

Measurement Invariance of the 2-Factor Model of the Athens Insomnia Scale (AIS) in Nurses Across Gender Groups

	Χ2	df	P value	Χ2/df	CFI	RMSEA	Χ2 Difference Test Statistics			ΔCFI	ΔRMSEA
							ΔΧ2	Δdf	P value
2-Factor model: AIS
Equal form	94.207	38	0.000	2.479	0.968	0.051
Metric invariance-Equal loadings	104.122	44	0.000	2.366	0.965	0.049	9.915	6	0.128	−0.003	−0.002
Scalar invariance-Equal intercepts	124.773	52	0.000	2.399	0.958	0.050	20.651	8	0.008	−0.007	0.001
Strict invariance-Equal factor variances	151.440	63	0.000	2.404	0.949	0.050	26.667	11	0.005	−0.009	0.000

Abbreviations: CFI, comparative fit index, RMSEA, root mean square error of approximation.

Item Analysis Based on Classical Theory

None of the AIS item scores showed substantial deviation from the univariate normality because skewness (absolute values) were less than 2 and kurtosis (absolute values) were less than 7 (Table 2).38 All the AIS items scores showed floor effect in this study population as the percentage of respondents that recorded the least level of scores exceeded 15% (Table 2).39,40 No ceiling/floor effects were seen for the AIS total scores, AIS factors scores except for a floor effect in the second factor of the AIS (28.2% of the participants recorded response of the lowest possible score).39,40

Rasch Rating Scale Model Analysis for Two Factors of the AIS

The Rasch-derived item and person reliability analysis indicates that both domains had an acceptable level of separation and item reliability (factor 1 = 0.795, factor 2 = 0.641). However, the person’s ability for factor 2 is relatively lower (0.34).

Since all items included responses in the four categories, there were three threshold estimates for each item (τi1, τi2, τi3). Hence, we investigated whether τi1 ≤ τi2, and τi2 ≤ τi3 for each item. Table 5 indicates that the thresholds were ordered as expected given the ordinal rating scale categories for all items in the scale. Moreover, infit and outfit mean square values for all items were within the acceptable range (<1.4, >0.6). The range of difficulty in factor 1 ranged from 1.27 to 1.59 in which the item “Sleep Induction” and “Final Awake” was the easiest task while the item “Sleep Quality” was the most difficult. In factor 2, item difficulty ranged from 1.47 to 2.92, in which the item “Sleepiness during the day” was the easiest item and “Function during the day” was the most difficult one (Table 5). A detailed presentation of the person distribution with respect to item difficulty for individual items of the two dimensions of the AIS is shown in Wright maps shown in and . The column on the left-hand shows person distribution along with the dimension. The column on the right-hand shows different AIS items along with their item difficulty. In and , the histogram on the left side indicates the distribution of abilities of persons, with the ablest on the top and the least able on the bottom. On the right side, the items are indicated. As it is shown, the ablest persons most probably answer category 3 (ie, the highest possible option) for all items. And the least abled (ie, unfavorable, depending on the scale) are less likely to provide category 3 responses for all items (ie, in this case, for none of the items). This implies that the scale is excellent in differentiating participants according to their levels of sleep/daytime symptoms. In the , almost all items are on the same level of difficulty with sleep quality items being slightly bit difficult. In , the difficulty levels of items vary markedly from each other. The ablest persons are likely to locate the highest category (category 3) for Function during the day and Wellbeing during the day. On further manual exploration of the , it is noted that a person who answers category 2 on items recording wellbeing and Function during the day has a similar ability with a person that records category 3 for sleepiness during the day. As indicated in Figure 3A and B, the step measures also functioned as expected and were disordered in the corresponding category probability curves.

Table 5

Summary of Parameter Estimates and Item Fit Statistics of the Rating Scale Model: Athens Insomnia Scale

Item	Severity (SE)	Outfit	Infit	Threshold 1 (τi1)	Threshold 2 (τi2)	Threshold 3 (τi3)
Factor 1
Sleep Induction	1.27 (0.07)	1.21	1.19	−0.64	1.34	3.10
Awakenings during the night	1.35 (0.07)	1.02	1.07	−0.56	1.43	3.19
Final awakening earlier than desired	1.27 (0.07)	1.05	1.01	−0.64	1.35	3.11
Total sleep duration	1.47 (0.07)	0.83	0.86	−0.44	1.55	3.31
Overall quality of sleep	1.59 (0.07)	0.85	0.89	−0.32	1.66	3.42
Factor 2
Sense of well-being during the day	2.50 (0.08)	0.97	1.14	0.66	2.75	4.07
Functioning (physical and mental) during the day	2.92 (0.09)	0.79	1.02	1.09	3.18	4.50
Sleepiness during the day	1.47 (0.07)	1.07	1.00	−0.36	1.73	3.04

Abbreviation: SE, standard error.

Continued.

(A) Individual item characteristic curves of Factor-1 (items assessing sleep symptoms) of the Athens insomnia scale. (B) Individual item characteristic curves of Factor-2 (items assessing daytime symptoms) of the Athens insomnia scale.

Internal Consistency

Cronbach’s α for the AIS tool, Factor-1 score, Factor-2 score were 0.86, 0.82, and 0.72, respectively (Table 2). The range of values of the item-factor correlation coefficients were 0.68 to 0.80 for factor-1, and 0.74 to 0.78 for factor-2, respectively (Table 2). The range of values of the corrected item-factor correlation coefficients were 0.52 to 0.70 for factor-1, and 0.41 to 0.64 for factor-2, respectively (Table 2). The range of values of the Cronbach’s α if item deleted were 0.76 to 0.81 for factor-1, and 0.50 to 0.78 for factor-2, respectively (Table 2).

Convergent Validity: Correlation of the AIS Scores with Related Dichotomous Subjective Sleep Measures

AIS factor-1 [adjusted odds ratio (AOR) = 1.23, 95% CI (1.13–1.33)] and factor-2 [AOR = 1.50, 95% CI (1.28–1.77)] scores were significantly associated with habitual feeling of tiredness after usual night sleep (Table 6). AIS factor-1 [AOR = 1.09, 95% CI (1.00–1.18)] and factor-2 [AOR = 1.31, 95% CI (1.40–1.51)] scores were significantly associated with impairment of daytime socio-occupational functioning. AIS factor-1 [AOR = 1.29, 95% CI (1.19–1.40)] and factor-2 [AOR = 1.22, 95% CI (1.05–1.40)] scores were significantly associated with feeling of daytime fatigue, irritability, and restlessness (Table 6).

Table 6

Convergent Validity: Correlation of the Athens Insomnia Scale (AIS) Scores with Related Dichotomous Subjective Sleep Measures

Subjective Sleep Measures	AIS Scores	AOR (95% CI)	P	Model Summary
Feeling of tiredness after usual night sleep	Factor-1	1.228 (1.135–1.329)	<0.001	χ2(2, N = 563) = 140.943, p < 0.001
	Factor-2	1.505 (1.279–1.771)	<0.001
Impairment of daytime socio-occupational functioning	Factor-1	1.087 (1.005–1.175)	0.038	χ2(2, N = 563) = 49.956, p < 0.001
	Factor-2	1.312 (1.140–1.509)	<0.001
Feeling of daytime fatigue, irritability, and restlessness	Factor-1	1.292 (1.194–1.398)	<0.001	χ2(2, N = 563) = 122.174, p < 0.001
	Factor-2	1.216 (1.054–1.403)	0.007

Abbreviation: AOR, adjusted odds ratio.

Discussion

To the best of the authors’ knowledge, this is (i) the first comprehensive psychometric study of the AIS on a sample of the Saudi population, as well as in nurses, (ii) the first psychometric study to provide statistical support for the addition of items to improve validation measures of the AIS, (iii) the first study to employ appropriate Rasch rating scale approach for item response theory-based psychometric measures, (iv) the first study to record statistical evidence for measurement invariance at four levels, ie, configural, metric, scalar, and strict for the original 2-factor structure of the AIS across gender groups, and (v) one of the very few studies on the factor analysis of the AIS that verified assumptions of various sample size adequacy measures such as anti-image matrix, Bartlett’s Test of Sphericity, Determinant, Kaiser-Meyer-Olkin Test of Sampling Adequacy (KMO), Communality, and inter-item correlation matrix. We found indicative psychometric support to further expand the AIS by adding items in the daytime dysfunctions factor score. Such a future expansion of the AIS may better incorporate items that cover additional diagnostic criteria of insomnia as defined by the International Classification of Sleep Disorders – Third Edition. The current study is one of the few studies that determined the psychometric validity of the AIS using the rigorous and robust measures of both classical theory and the Rasch rating scale parameters. Precisely, evidence favoring factorial validity, measurement invariance, reliability, item analysis, and convergent validity for the original English version of the AIS were found in nurses working in Saudi Arabia.

Factor Analysis: PCA and CFA

Both CFA and multi-group CFA findings favored a two-factor solution though, all the four measures of factor retention (including the robust measure of parallel analysis) in the PCA supported a one-factor solution. As per the accepted practices of reporting of factor analytical studies, we conclude that the two-factor solution supported by CFA and multi-group CFA is the favored.32,35 CFA, unlike PCA, only determines the relationship between variables based on a reliable variance in the data after adjusting measurement error.35 Items of the AIS record responses related to insomnia symptom clusters; therefore, as concluded by Kim 2008, CFA findings are more reliable in this situation.41 Previous studies have broadly reported three different types of dimensionality, including 1-factor, 2-factor, and 3-factor models of the AIS in different populations.20 The results of this study are similar to that reported for the Japanese version of the AIS in two separate studies involving chronic insomniacs, general community, and outpatients of pain management centers.28,42 Moreover, the dimensionality results of this study are also favored by the theoretical construct considerations; the AIS has groups of items broadly representing two groups of symptoms constructs, one assessing sleep (night symptoms) and the other measuring daytime dysfunctions (daytime symptoms).14

Factor Analysis: Measurement Invariance

Furthermore, additional support for the two-factor model of the AIS in the study population was provided by the measurement of invariance at four different levels, ie, configural, metric, scalar, and strict. This is the first study to show measurement invariance for the two-factor solution across gender groups at all of these four levels. Sirajudeen et al 2020 reported measurement invariance at three different levels, ie, configural, metric, and scalar, among gender groups of occupational computer users for a three-factor solution.20 However, the authors reported that their three-factor model did not meet the criteria of the strict measurement invariance.20 Iwasa et al 2019 also found measurement invariance for a similar two-factor structure of the AIS in the Japanese across gender and history of mental illness condition.43 Iwasa et al used a Japanese version of the AIS with the same number of items but with slightly modified scoring patterns. It is noteworthy to mention here the three studies (including the present one) that reported measurement invariance of the factor structures of the AIS, two favored a two-factor solution, which are similar to the one assessing sleep, and the other measuring daytime dysfunctions.43 Two other studies reported measurement invariance at the item level using differential item functioning (DIF) across gender groups. However, a direct comparison with the results of this study is not possible because the DIF does not give evidence about the measurement invariance of factor or construct scores.44

The results supported no major deviance from univariate distribution for items, factors, and total scores of the AIS; this suggests that the score distribution followed a trend that is usually expected in a general population.45 This gives credence to the generality of the study findings. The absence of ceiling/floor effect for the AIS total and factors scores imply that these scores can differentiate between groups even at extreme scores.39,40 However, the evidence did not favor a similar optimum ability to differentiate between groups at the lowest level of the score for factor-two score.39,40 A similar concern is raised for the factor two score by a lower value of Person reliability-a Rasch rating scale parameter. As explained in the section on the Rasch rating scale, future studies may explore this further by adding a few items. This is more pragmatic considering the fact that insomnia, as defined by the International Classification of Sleep Disorders – third Edition, has additional diagnostic criteria compared to the ICD-10 criteria, which were implemented to develop the AIS. It may not be out of place to mention here that Lin et al 2020 also concluded that even though the AIS is a robust measure, there are some concerns about the standard error of measurement, the lower effect size, etc.44

All the measures (except one) of the Rasch rating scale provided additional psychometric validity support for the two-factor solution of the AIS in the study sample. Adequate values of the infit and outfit mean square for all the items, orderly nature of the three threshold estimates, and ICCs indicating that step measures functioned as expected for all the items of the AIS in the study sample. For illustration, at a latent dimension of 1.0 (Figure 3A), the probability of recording the second response category is approximately similar for all the AIS items, ie, around 60%. The findings regarding infit/outfit are similar to that reported in Iranian cancer patients.44 Lin et al 2020 showed that the value of infit (0.85–1.17) and outfit (0.79–1.14) were all in the normal range and were very close to the values in this study.44 There was one notable difference between the two studies while Lin et al used the partial credit model; the Rasch rating scale was used in this study. Evidence shows that outfit statistics is more stable in the Rasch rating scale, and infit statistics is more stable in the partial credit model.46 As all the items of the AIS are scored on same Likert rating scale, therefore, our approach of using Rasch rating scale is statistically more suitable. This is because partial credit models require that all items of the questionnaire employ a unique and distinct rating scale structure, but this is not the case with the AIS items scores (Linacre, 2000)47. Person reliability for factor-2 was less than 0.7; this may be partially explained by the low number of items, ie, only three items compared to five items in the factor-1 of the AIS. However, Cronbach’s alpha was adequate for both factors because, unlike in Rasch reliability indices, extreme values were not excluded from estimation. This may at times give an inflated estimate of reliability, but the presence of extreme values is more pragmatic in empirical datasets. Item reliability for both the factors of the AIS was satisfactory, implying that difficulty range and sample size were sufficient in this study.

Internal Consistency and Convergent Validity

Internal consistency as indicated by Cronbach’s alpha, item-factor correlation coefficients, and corrected item-factor correlation coefficients was good for the AIS tool and both factors. Previous studies have reported similar values for Cronbach’s alpha: 0.81 in Chinese adolescents,26 0.87 in Japanese chronic pain patients,42 0.89 in Greeks’ sample of insomniacs, psychiatry patients, and controls14,20. Sirajudeen et al 2020 reported a lower value of 0.66 in Indian occupational computers users20. In the two studies that reported identical factor structure, Okajima et al 2013 reported a slightly higher Cronbach’s alpha value for both factors, one (0.85) and two (0.78) in a Japanese sample, including insomniacs. While Enomoto et al 2018 reported higher values for factor one (0.89) but a lower value for factor two (0.66).28,42

Significant association of factor-1 and factor-2 scores with dichotomous items feeling of tiredness after sleep, daytime socio-occupational dysfunctions, and daytime fatigue indicate convergent validity of the AIS scores in the study population. There are very few studies that investigated the convergent validity of the AIS. Jeong et al 2015 also reported favorable convergent validity measures for the AIS among Korean.48 The study reported a significant correlation between the AIS scores and the related measures of the Pittsburgh sleep quality index, insomnia severity index, and Epworth sleepiness scale.48

Limitations

It is noteworthy to mention some of the important limitations that may help understand the generalization and implications of the findings. Convenient sampling was used to maximize participation; however, theoretically, such a sampling procedure may pose limitations to generality. However, it may not be out of place to say that the study sample size was substantial, with participating nurses from various parts of Saudi Arabia. Moreover, the lack of significant issues of skewness/kurtosis further supports the representativeness of the study sample. The convergent validity of the AIS was assessed by correlating the AIS score with a set of structured questions about sleep health instead of a validated questionnaire. The study did not investigate the diagnostic validity of the AIS using a clinical diagnosis of insomnia. However, the study did employ robust measures of psychometric validity testing using both classical theory and Rasch rating theory parameters. Finally, the results may not be extended to the general population; because nurses do differ from the general population with regard to shiftwork schedules and health profiles.

Conclusion

The AIS was found to have robust psychometric validity properties, as evidenced by classical and item response theory analysis measures. However, there is some evidence that future research efforts may further enhance AIS psychometrics. Such efforts may benefit from incorporating additional items based on the recent diagnostic requirements of insomnia to expand items of daytime dysfunctions.