Reliability, factor analysis and internal consistency calculation of the Insomnia Severity Index (ISI) in French and in English among Lebanese adolescents.

OBJECTIVES: Our goal is to validate and to verify the reliability of the French and English versions of the Insomnia Severity Index (ISI) in Lebanese adolescents. STUDY
DESIGN: A cross-sectional study was implemented.
METHODS: 104 Lebanese students aged between 14 and 19 years participated in the study. The English version of the questionnaire was distributed to English-speaking students and the French version was administered to French-speaking students. A scale (1 to 7 with 1 = very well understood and 7 = not at all) was used to identify the level of the students' understanding of each instruction, question and answer of the ISI. The scale's structural validity was assessed. The factor structure of ISI was evaluated by principal component analysis. The internal consistency of this scale was evaluated by Cronbach's alpha. To assess test-retest reliability the intraclass correlation coefficient (ICC) was used.
RESULTS: The principal component analysis confirmed the presence of a two-component factor structure in the English version and a three-component factor structure in the French version with eigenvalues > 1. The English version of the ISI had an excellent internal consistency (α = 0.90), while the French version had a good internal consistency (α = 0.70). The ICC presented an excellent agreement in the French version (ICC = 0.914, CI = 0.856-0.949) and a good agreement in the English one (ICC = 0.762, CI = 0.481-890). The Bland-Altman plots of the two versions of the ISI showed that the responses over two weeks' were comparable and very few outliers were detected.
CONCLUSION: The results of our analyses reveal that both English and French versions of the ISI scale have good internal consistency and are reproducible and reliable. Therefore, it can be used to assess the prevalence of insomnia in Lebanese adolescents.

Introduction

Many studies have demonstrated the fundamental role of sleep in health, wellness and physical and psychological balance for all ages [1]. It is not only a good health factor, but it also affects the quality and the speed of learning [2]. In recent years, due to their impact on the health and functioning of children and adolescents, sleep disorders have received more attention [3]. The detection of sleep disorders in adolescents is difficult, but essential. Adolescents experience changes in their central neuroendocrine regulation, which alter their physiologic, cognitive, and emotional functioning [4]. They can experience difficulties with falling asleep before 10:30 or 11 pm owing to a physiological shift in the timing of the release of melatonin [5]. Furthermore, social pressures nowadays, represented by an increasing amount of additional demands on many adolescents and the accessibility of highly entertaining computer and video games, as well as late-night socializing via the Internet, could also delay bedtime on school nights to between 11 pm and midnight [6]. These major factors plus early start times for most high schools may explain why adolescents are predisposed to sleep problems and that most adolescents are thus chronically sleep-deprived [7].

Insomnia is a prevalent complaint in clinical practice and is considered to be one of the most common sleep disorders in adolescents [7], [8].

Insomnia can cause impairment of several daytime cognitive functions (e.g. memory, concentration and attention) [9] and also can be present with several comorbid medical or psychiatric disorders [10]. When left untreated, it may exacerbate or increase other conditions or delay recovery [11], [12], [13]. Thus, it is necessary to recognize insomnia early and initiate the appropriate treatment.

The gold standard for making a valid insomnia diagnosis is a clinical evaluation [12], [13]. Nevertheless, such an evaluation can be time-consuming in routine clinical practice and may discourage health practitioners from systematically inquiring about sleep with all of their patients. Therefore, brief and valid self-reporting scales can facilitate the initial screening and formal evaluation of insomnia [14]. However, there are relatively few self-reporting questionnaires for the assessment and clinical diagnosis of insomnia [15], particularly in Lebanon. Our eventual end point is to employ validated scales to assess main sleep disorders, like insomnia, in Lebanese adolescents where very little is known about their prevalence.

The Insomnia Severity Index (ISI) is a self-reporting scale that includes two of the most common complaints about insomnia: difficulty initiating sleep and frequent awakenings from sleep [16], [17]. This scale has been widely used for clinical purposes and research, since its introduction in 1993 [18]. The ISI has been validated for a number of age groups, from adolescence [19] to the elderly [20], and is reported to be a valuable assessment instrument for individuals with cancer [21], psychiatric diseases [12], [15], and other clinical disorders [22], [23], [24]. The ISI has been translated into several languages, including Spanish [25], Hindi [26] and Chinese [27] and demonstrated adequate psychometric properties with these variety of languages. However, there is no standardization for Lebanese adolescents.

The purpose of the present study is to verify the validity and reliability of the ISI in French and in English as a screening measure of insomnia in the Lebanese teenage population.

Methods

Subjects

Our sample consisted of students aged between 14 and 19 years. In order to examine the level of comprehension of all types of students, adolescents in both schools and technical institutes were chosen. Education in Lebanon is organized in three phases: preschool education, basic education and secondary education. The latter concludes with a Baccalaureate or a ‘professional certificate’, both of which allow the pursuit of higher education [28]. After completing ninth grade, adolescents are given the opportunity to choose between pursuing tenth grade in a regular school or continuing their education in a technical institute where they can take various courses to acquire needed competencies and skills for specific professions.

Arabic language, along with English or French, is taught from early years in Lebanese schools. English or French are the mandatory media of instruction for mathematics and sciences in all schools [29]. Children and adolescents can read and write in these foreign languages early on, and they are generally considered easier to understand than Arabic. Thus, we decided to validate the ISI in both French and English in Lebanese adolescents. Naturally, the scale in English was administered in the school and technical institute where English is the primary teaching language, while the French version of the ISI was distributed in educational institutions where French is the primary language.

In this paper we describe a part of the pretest that was carried out in November and December 2015. First, we were provided with a list of all the schools in Lebanon from the Ministry of Education in order to select randomly four schools, taking into account the different types of learning and the primary teaching language: one private French school, one private English school, one French technical institute and one English technical institute. Second, a sample of 115 students was randomly selected in these schools where all participants met the following criteria: Lebanese, older than 14 years and younger than 19 years old and enrolled in the chosen school or technical institute. We requested each principal of the school or institute selected, to pick 30 students randomly with all the age categories required because a default sample size of 30 participants is recommended to detect misunderstandings, ambiguities or other difficulties participants may encounter with instrument items [30]. The students were selected randomly by the principals according to the timing of the survey and their school schedules. For example, if the students had gym, arts and crafts or music class the principals decided that the adolescents could skip these classes in order to participate in the survey. That was not the case if the students had to attend scientific classes like mathematics or physics. The principals of the four schools chosen had the same random selective strategy. We ended up with 115 participants chosen randomly.

We handed out a questionnaire to students in grades 9 to 12 at each school, and in different technical specialties at each technical institute, which consisted of questions about socio-demographic information, lifestyles, sleep habits, ISI and other validated tests in adolescents.

The present study is a pilot study to test the level of comprehension for each instruction, question and response of the entire questionnaire that will eventually be used in a nationwide survey in order to evaluate lifestyles, sleep habits and risk factors of some of the most common sleep disorders in adolescents and to assess their prevalence. Our goal in this paper is to verify the validity and reliability of the ISI in French and in English as a screening measure of insomnia in the Lebanese teenage population in order to use this scale among Lebanese teens in our national survey.

Eleven students decided not to participate because they did not want to hand out information about their sleep disturbances: five adolescents from the Anglophone school and two and four students from the French and English technical institutes, respectively. Our final sample consisted of 104 adolescents with 58 Francophone and 46 Anglophone students.

Finally, the ISI was administered to the same groups of students, by the same investigators, on a second occasion, which took place two weeks after the first meeting in order to assess the test–retest reliability of the ISI.

The Ethics Committee designated by our research team stated that the written consent of the parents was not required since this was an observational study and the content of the scale tested was acceptable for the underage students to read and to answer to. Therefore, the oral consent of the adolescents was enough in order to participate in this project.

Insomnia Severity Index (ISI)

The ISI is a seven-item self-administered questionnaire assessing the nature, severity and impact of insomnia [14], [22] in adults (≥ 18 years) in English and in French during the last month before administration. ‘Last month’ represents the recall period and the dimensions assessed are: the severity of sleep onset and maintenance difficulties (middle- and early-morning awakening), satisfaction/dissatisfaction with current sleep pattern, interference of sleep difficulties with daytime functioning, noticeability of impairment attributed to the sleep problem by others, and the degree of distress caused by the sleeping problems. A 5-point Likert scale is used to rate each item, and it can take up to 5 min to be completed and < 1 min to be scored [30]. Accordingly, the total score ranges from 0 to 28 with 5 sub scores: scores ranging between 0 and 7 mean no clinically significant insomnia was found; scores ranging between 8 and 14 mean the presence of subthreshold insomnia; scores ranging between 15 and 21 mean the presence of clinical insomnia of moderate severity and finally scores ranging between 21 and 28 mean the presence of severe clinical insomnia. Nevertheless, a study determined that a cutoff score of 14 distinguished subjects with insomnia from normal controls with a sensitivity and specificity of 94% [31]. Therefore we used this cutoff score in our study. The ISI was first administered with several basic socio-demographic questions regarding the students, their lifestyles and their family structure. Two weeks later, the same sample of students (n-104) again answered the ISI only. On both questionnaires we carried out a test–retest analysis to assess the reliability.

Level of comprehension by adolescents of the ISI in French and in English

We created a scale similar to the one used by researchers in the Department of Psychiatry, University of Hong Kong, China to validate the ISI in Chinese adolescents [19]. At the first administration of the questionnaire, the students were asked to rate each instruction, question and response category based on their understanding on a 7-point scale (1 = extremely easy to understand; 4 = not sure; 7 = not at all understandable). Any item that had a mean score > 2.5 was considered unsatisfactory and had to be reworded until consensus between the research team and translators was achieved [32], [33]. For further administration, the final version of the ISI was obtained after completion of these standard procedures, but in this study, the students had to answer to the same initial version of the questionnaire.

Data analysis

All statistical analysis were conducted using SPSS version 22.0 for Windows. The scale's structural validity was assessed: the factor structure of the ISI was evaluated by principal component analysis followed by varimax rotation [11]. The factors were selected according to eigenvalues > 1, scree test and with a value of Kaiser–Meyer–Olkin of 0.6 and above. The internal consistency of the ISI was evaluated by Cronbach's alpha [34].

The test–retest reliability was assessed by Intraclass Correlation Coefficient (ICC) [35] and the Bland-Altman plot.

Finally, a p-value < 0.05 was considered indicative of statistical significance.

Results

Descriptive statistics

Table 1 summarizes descriptive statistics of the sample. All of our participants had a Lebanese nationality, and the sample mean age was 15.96 years ± 1.28. The majority of the sample consisted of females (65.4%) and 30.8% were ninth-grade students. In terms of geographical distribution, 21.2% of our sample was from an English-speaking institute located in the Lebanese capital Beirut, whereas the other chosen schools and institute are located in Mount Lebanon. Moreover, 89.4% of the teenagers had married parents and 51.9% of the mothers fathers also had a university diploma.

Table 1

Socio-demographic characteristics (n = 104).

Characteristics
Age (mean, STD)	15.96 (1.28)
Region, n (%)
Beirut	22 (21.2)
Mount Lebanon	82 (78.8)
Sex, n (%)
Male	36 (34.6)
Female	68 (65.4)
Class, n (%)
9th	32 (30.8)
10th	19 (18.3)
11th	9 (8.7)
12th	5 (4.8)
Technical institute classes	39 (37.4)
Schools and institutes, n (%)
French-speaking school	35 (33.7)
French-speaking technical institute	23 (22.1)
English-speaking school	30 (28.8)
English-speaking technical institute	16 (15.4)
Parents' marital status, n (%)
Married	93 (89.4)
Divorced	5 (4.8)
Father remarried	2 (1.9)
Widow (father passed away)	4 (3.8)
Parental educational level
Mother
Never attended school	1 (0.96)
Elementary school	8 (7.7)
9th-grade diploma	12 (11.5)
Technical institute diploma	4 (3.8)
High school diploma	25 (24)
College degree	54 (51.9)
Father
Never attended school	6 (5.8)
Elementary school	3 (2.9)
9th-grade diploma	15 (14.4)
Technical institute diploma	6 (5.8)
High school diploma	20 (19.2)
College degree	54 (51.9)

We also noticed that only 5.8% of the students had a part-time job and 40.4% of the teens slept in their own bedrooms. Nevertheless, there was no relationship between the presence of insomnia and the adolescents having their own bedroom because no statistically significant difference was observed (p = 0.147).

The prevalence of insomnia in relation to sex and grade is shown in Table 2. In the total sample, the prevalence of insomnia was 19.3% : 42% among males and 58% among females. No statistically significant difference was observed between males and females (p = 0.135). Among males, the highest prevalence of insomnia was in technical institute students, whereas among females the highest prevalence of insomnia was in ninth-graders.

Table 2

Prevalence of insomnia among Lebanese adolescents (n = 104).

Grades	With insomnia (%)	Without insomnia (%)	N
Males
9th	15.4	60.9	16
10th	38.5	13	8
11th	0	4.3	1
12th	7.7	0	1
Technical institutes	39.2	21.7	10
Total	100	100	36
Females
9th	31.4	15.2	16
10th	22.9	9.1	11
11th	8.6	15.2	8
12th	11.4	0	4
Technical institutes	25.7	60.5	29
Total	100	100	68

No statistically significant difference was observed between males and females (p = 0.135).

Level of comprehension by the adolescents

After responding to the 7-point Likert scale, results revealed that for the French version of the ISI, three items were not understood perfectly, with averages of > 2.5 for each item: the first instruction and the interference with the daily functioning question. The noticeability to others regarding the participant's sleeping problem question was not well understood by either the Francophone or the Anglophone students. Also, in the English version of the ISI, the question regarding the satisfaction of sleep was not well understood either.

Table 3 presents the mean score of each item.

Table 3

Mean scores of each item of the ISI in the French and English versions.

Items	Means and standard deviations
	French version		English version
	Mean	Std. deviation	Mean	Std. deviation
ISI
For the first three questions, please rate the severity of your sleep difficulties.	2.72	1.887	2.26	1.932
Difficulty falling asleep	1.02	0.127	1.52	1.363
Difficulty staying asleep	1.16	0.518	1.48	1.313
Problem waking up too early in the morning	1.10	0.433	1.42	1.232
How satisfied/dissatisfied are you with your current sleep pattern?	1.79	1.381	2.73	1.961
To what extent do you consider your sleep problem to interfere with your daily functioning (e.g. daytime fatigue, ability to function at work/daily chores, concentration, memory and mood)?	3.21	1.954	2.13	1.893
How noticeable to others do you think your sleeping problem is in terms of impairing the quality of your life?	3.31	1.646	3.13	1.765
How worried/distressed are you about your current sleep problem?	1.79	1.345	1.55	1.567

The data in bold character indicates an item not well understood by the students thus having an average of > 2.5

Factor analysis

Principal component analysis with varimax rotation found two factors in the English version and three in the French version of the ISI with eigenvalues > 1. The same number of factors was indicated by scree plot. In the English version, factor 1 comprised essentially satisfaction with current sleep patterns, interference with daily functioning and noticeability of impairment. It accounted for 53% of the variance. Factor 2 explained 16% of the variance and comprised the rest of the items (Table 4). However, in the French version, factor 1 comprised interference with daily functioning, noticeability of impairment and level of distress; factor 2 included two items, severity of sleep onset and sleep satisfaction; and finally, factor 3 included the two final items (Table 5). Factors 1, 2 and 3 explained, respectively, 36%, 20% and 16% of the variance in the French version of the ISI. A Kaiser measure of factorability for the English version of the ISI was greater than the French version (0.78 and 0.60, respectively).

Table 4

Principal component analysis (varimax rotation) of the English version of the Insomnia Severity Index.

Insomnia Severity Index items (n = 46)		Factor loadings
		Factor 1	Factor 2
1a	Difficulty falling asleep	0.27	0.70
1b	Difficulty staying asleep	0.30	0.90
1c	Problem waking up too early	0.31	0.70
2	Sleep satisfaction	0.80	0.24
3	Interference	0.86	0.20
4	Noticeability	0.82	0.24
5	Distress	0.46	0.69
	Explained variance: 69%	53%	16%
	Eigenvalue	3.7	1.1

The data in bold character indicates to which factor each item of the ISI belongs.

Table 5

Principal component analysis (varimax rotation) of the French version of the Insomnia Severity Index.

Insomnia Severity Index items (n = 58)		Factor loadings
		Factor 1	Factor 2	Factor 3
1a	Difficulty falling asleep	0.19	0.87	− 0.20
1b	Difficulty staying asleep	0.23	− 0.27	0.94
1c	Problem waking up too early	0.25	0.51	0.63
2	Sleep satisfaction	− 0.25	0.84	0.13
3	Interference	0.77	0.23	0.00
4	Noticeability	0.85	− 0.48	− 0.12
5	Distress	0.71	0.08	0.24
	Explained variance: 72%	36%	20%	16%
	Eigenvalue	2.5	1.4	1.1

The data in bold character indicates to which factor each item of the ISI belongs.

Internal consistency and test–retest reliability

The English version of the ISI showed an excellent internal consistency (α = 0.90). Additionally, acceptable internal consistency was linked to the French version (α = 0.70).

The ISI presented a very good test–retest reliability. We have obtained almost a perfect agreement with the French version of the ISI, where ICC = 0.914 (CI = 0.856–0.949); and a moderate agreement with the English version, where ICC = 0.762 (CI = 0.481–0.890). The Bland–Altman plots of the French version (Fig. 1) and the English version (Fig. 2) of the ISI showed few discrepancies. The average of almost all of the differences is close to zero and between the limits of agreement, especially in the English version. The difference between the responses does not tend to get larger or smaller as the average increases. The variability is consistent across the graph. Very few outliers were detected, with three in each version.

Fig. 1

Bland–Altman plot of the French version of ISI.

Fig. 2

Bland–Altman plot of the English version of ISI.

Discussion

This study focused on one major self-reporting scale: the Insomnia Severity Index (ISI).

The objective was to verify its validity and reliability in Lebanese adolescents.

The level of comprehension by the adolescents was assessed and its structural validity was evaluated. The factor structure of the ISI was estimated by principal component analysis followed by a varimax rotation. Moreover, we obtained excellent (α = 0.90) and acceptable (α = 0.70) internal consistencies with the English and the French versions, respectively.

The ISI is written and validated in adults in French and in English, therefore we did not perform a forward and backward translation from the formal original English version. We first administered the English and French validated versions of the ISI in adults along with the comprehension Likert scale to 104 Lebanese adolescents. As shown in the results, the English version was better understood by the students than the French version. John Steele Gordon stated that English is an easier language to understand and to learn than other (originally) European languages. He concluded that first, verb conjugation in English is easier and second, some languages – including French, German and Greek, among others – are very different in their formal written versus their casual spoken forms, however that is not the case in the English language. Moreover, Gordon stated that matching English nouns and adjectival forms to gender or plural form is a less complicated task: unlike most other European languages, nouns are not arbitrarily divided as being ‘masculine’, ‘feminine’, or ‘neuter’ and adjectives need no change to match the noun gender form, nor do they change if a noun is plural [36]. Consequently, the research team along with the teachers that helped to conduct the survey agreed with Gordon's conclusions.

The outcomes regarding the factor analysis of the French version of the ISI among Lebanese teenagers and of the formal original English version among adults [27] were comparable: three factors were detected and in both studies they represented 72% of the variable, plus Kaiser–Meyer–Olkin (KMO) measures of sampling adequacy were very close in both studies (0.60 and 0.62, respectively). Moreover, in the systemic evaluation [14], it was considered that the factorial structure was easiest to interpret with three components. However, a number of studies of the ISI in different languages found only two factors after executing principal component analysis. For example, a study in China [19], which compared the psychometric properties of the Chinese versions of the ISI and other tests in order to assess insomnia in adolescents, stated that the principal component factor analysis of the ISI indicated two distinct factors involving the severity and impact of insomnia and confirming that this factor structure was more coherent than the original English version's three-factor structure. Our two-factor structure obtained in the English version of the ISI was almost similar, whereas the level of distress was added to the severity of insomnia and the ‘impact’ was more important in our study, explaining 53% of the variable versus only 34.5% in the Chinese study. However, the two principal component analyses of both English and French versions of the ISI in Lebanese adolescents must be conducted on a larger sample of students in order to assure the component structure's stability of each version.

It has become common practice in medical education research [37] and in psychological research to calculate coefficient alpha [38] when multiple-item measures of a concept or construct are employed.

Cronbach's alpha is the most commonly used measure of internal consistency [39] because it can be easily calculated each time a multi-item scale is administered. Also, it is widely used because influential texts have suggested that alpha is necessary to assess reliability [40].

Useful first descriptively, as an indicator of the degree to which constituent parts of a whole cohere, researchers should be encouraged to calculate Cronbach's alpha in their own samples and compare them to normative values to help evaluate the quality of the data they have collected [39].

Cronbach's alpha of both the English and the French versions among Lebanese adolescents were respectively 0.90 and 0.70. A large number of existing studies [14], [17], [20], [22], [26] show that the ISI has acceptable internal consistency for both English and non-English versions in adults with Cronbach's alpha generally exceeding the critical value of 0.60. Consequently, this is also true in our study. The internal consistency of the English version was similar to one achieved by the Arabic (0.92) [41], Hindi (0.91) [26] and Spanish (0.91) [42] versions of the ISI.

Since there are no validated self-reporting scales in our population regarding the diagnosis of insomnia, the comparison of the psychometric properties and the correlation between the ISI and different self-reporting scales diagnosing insomnia must be executed in our population. However, we could not do this because the incorporation of another self-reporting scale diagnosing insomnia would have been a burden on the adolescents since our entire questionnaire needed around 45 min to be completed by the students. A study in Spain [42] examining the reliability, factorial structure, convergent and discriminant validity of the Spanish version of the ISI in an older adult sample, obtained a statistically significant positive correlation between the Spanish version of the ISI and the Athens Insomnia Scale (r = 0.93) and a statistically significant negative correlation with the Mini Mental State Examination (MMSE) (r = − 0.15). Also, using a sample of diagnosed insomniac Lebanese teenagers and controls could have been even more helpful in the validation of the ISI in our population. A study in Iran reviewing the validity and reliability of the Iranian Version of the Insomnia Severity Index stated that the correlation between the ISI and the Pittsburg Sleep Quality Index was stronger in insomniac patients than in healthy ones [15].

Retest reliability plays a considerably smaller role in contemporary research. Some authors renewed the call for attention to retest reliability and proposed the use of a general reliability measure that combines internal consistency and retest reliability [43]. Test–retest reliability, or reproducibility, is a method that evaluates a tool's reliability by administering it to the same group of people, in the same way, on two or more different occasions [44].

In our study, the test–retest reliability for the total score of the ISI was completed over 2 weeks using the ICC and Bland–Altman plot. In terms of reliability, if ICC ≥ 0.9 this means that there is almost perfect agreement, and if ICC < 0.8 it means that there is moderate agreement [45]. Both the French and English versions had an ICC > 0.75. The French version of the ISI had an excellent ICC, which was even greater than the Iranian version of the ISI (ICC = 0.84, CI = 0.78–0.89) [15]. The Bland–Altman plots of the two versions of the ISI showed that the responses over 2 weeks are comparable and very few outliers were detected.

In conclusion, the present study establishes the French and English versions of the ISI to be as reliable and valid for the assessment and screening of the severity of insomnia as the original ISI [46] in Lebanese adolescents. The scale chosen is brief, easy-to-use and possesses satisfactory validity, internal consistency and test–retest reliability.

Since the prevalence of insomnia in our sample was quite high (19.3%), it is a must that we should investigate the prevalence of insomnia in our future nationwide survey using the validated French and English versions of the ISI in Lebanese adolescents.

Author statements

Acknowledgements

The authors would like to thank all the teachers and students that participated in this study.

Ethical approval

The project's protocol was read and approved by the Ethics Committee of the Faculty of Medical Science, Lebanese University, Hadath, Lebanon.

Funding

Lebanese University, Hadath, Lebanon.

Competing interests

None declared.

Authors' contributions

M. Chahoud: project development, data collection, data analysis, data interpretation, manuscript writing, manuscript editing, reading and approving the final version.

R. Chahine: project conception, manuscript editing, reading and approving the final version.

P. Salameh: data analysis and interpretation, manuscript editing, reading and approving the final version.

E.A. Sauleau: project development, data analysis and interpretation, manuscript editing, reading and approving the final version.