Psychometric Properties of the Strengths and Difficulties Questionnaire in a Sample of Albanian Elementary School Children.

INTRODUCTION: The Strengths and Difficulties Questionnaire (SDQ) is a 25-item screening measure for emotional and behavioral problems in children and adolescents ages 4 to 17. AIM: To evaluate the reliability and factor structure of SDQ teacher version in a sample of Albanian elementary school children.
MATERIAL AND METHODS: Teachers of children (N = 542) from three elementary schools of Tirana city, Albania, (aged 6 to 12 years, grade 1-4) completed the Albanian version of SDQ. Internal consistency reliability was determined by calculation of the Cronbach- α coefficient and average inter-item correlation. Both Confirmatory factor analysis (CFA) and Exploratory factor analysis (EFA) examined the underlying factor structure of the questionnaire.
RESULTS: Internal consistency reliability of subscales ranged from "acceptable" to "good" with α values that ranged from α =.68 (prosocial) to α = .83 (hyperactivity-inattention) and average inter-item correlation ranged from 0.5 for hyperactivity scale to 0.20 for total difficulties scale. Confirmatory factor analysis did not confirm the five factor structure of the teacher version of SDQ in Albanian, with values of incremental fit indices (X2(265)= 1562.72, p<.0001; Comparative Fit index, (CFI) =.814; Tucker-Lewis Index, (TLI) =.772) and Root Mean-Square Error of Approximation ( RMSEA) =.095 indicating a misfit. Exploratory factor analysis revealed a different pattern of factor loadings for items of hyperactivity and conduct scale suggesting a non-one-dimensional contribution of the items.
CONCLUSION: The SDQ demonstrated satisfying reliability based on internal consistency coefficients but must be used with caution when screening for Attention Deficit/Hyperactivity disorder in Albanian elementary school children.

INTRODUCTION

Epidemiological data indicate that 20-40% of all children worldwide have a clinically diagnosable disorder, which often manifests as internalizing or externalizing behaviors (1-3) and that many more children exhibit specific symptoms or subclinical problems (4-6).

Despite the prevalence of mental health disorders in youth, only 45% of youth with a mental health diagnosis receive treatment, and only 24% of those individuals receive care in the school system (7). Within the school system, universal screening for mental health difficulties has been recommended as a way to identify children who are experiencing difficulties and provide the services necessary to support them (8). The goal of universal screening is to identify childhood problems before the behaviors exceed the threshold for a parent or teacher referral for services (9). School-based screening is advantageous because of the natural aggregation of most children at the transition to primary school and the accessibility of both teachers and parents as informants which may increase the accuracy of identification of children with problems (10). Within the framework of implementing a program of school-based universal screening for child psychopathology, it is of paramount importance the selection of an appropriate universal screening tool (11). When selecting a screening tool, planning teams must consider the domain of interest, psychometric adequacy, feasibility, acceptability and overall utility of the screening tool (12). In this context, considering the Strengths and Difficulties Questionnaire (SDQ) (13) as a broad-band valid screening tool for childhood mental health difficulties is extensively supported by research studies (14). The Strengths and Difficulties Questionnaire is a brief behavioral screening questionnaire that can be completed by parents and teachers of 4-17 years old and youths 11-16 years old. SDQ consists of 25 items rated on a Likert-type scale and equally divided into five subscales: emotional symptoms, conduct problems, hyper-activity-inattention, peer problems,and prosocial behavior. The sum of the four subscales generates a total difficulties score. SDQ has several features that make it a useful screening measure for child mental health problems. It is freely available, takes only some minutes to be completed and it is translated in 85 languages, including Albanian. Additionally, since parent and teacher versions are identical, they enable the collection of the information about specific behaviors from different informants. Finally, the SDQ covers a broad range of common childhood difficulties, including internalizing, externalizing, inattention/hyperactivity, and peer relationship problems so that information on several aspects of children’s functioning can be gathered with a single, brief measure (14). SDQ parent, teacher and/or self-report versions provide a five-factor structure, i.e., emotional symptoms, conduct problems, hyperactivity/inattention, peer problems and prosocial behavior scale, that corresponds with the domains of psychopathology and personal strengths it intends to measure. Psychometric Properties of the Parent and Teacher SDQ has been investigated in several studies across different countries worldwide (14-18, 27) with mixed results. The reliability of the SDQ can be defined as satisfactory, although it should be noted that the internal consistency of the teacher ratings might be higher except for the peer problem scale having a value below 0.70 (14). However, up to date, no study has investigated the psychometric properties of SDQ in a population of Albanian youths. The present study is the first one to investigate the psychometric properties of the teacher-reported SDQ in an Albanian population of elementary school children.

AIM

To evaluate the reliability and factor structure of SDQ teacher version in a sample of Albanian elementary school children.

MATERIAL AND METHODS

Participants and Procedure

Permission to conduct a research in the public primary schools was first applied for, to the Regional Directory of Education which further granted permission to conduct the study on three public primary schools of one district of Tirana city. After some preliminary meetings with the Principals and teachers of the selected schools, the consenting teachers and students were provided with separate envelopes for parents and teachers, containing a cover letter, informed consent and questionnaires to be completed. Inclusion criteria were children grade 1-4, with no hearing, seeing or major physical impairment. Participants were parents and teachers of the children (N =893 ) from grade 1 to 4 of the three schools, aged 6 to 11 (M=8.85; SD=1.2). Parents returned signed informed consent and completed questionnaires for 546 (61%) children, while teachers returned completed questionnaires for 542 (60%) children. 451 (50.5%) children had teachers and parents completed questionnaire. Overall, there were returned completed questionnaire from parents or teachers for 637 (71%) children (girls = 320, boys = 317). Of 32 teachers, only 24 (75%) consented to participate in the study.

Measures

Teacher official version of the Albanian translated SDQ (www.sdq.org) was used as a measure tool and teachers were asked to think about the child behavior during the last six months and to select the alternative answer that best described the child. They were encouraged to answer all the items even if they were not sure about the best alternative. Ratings are scored as follows: not true =0, some-what true =1, and certainly true =2. Five items are reversely scored (certainly true =2, somewhat true =1, not true =1). Scoring for each subscale was calculated by summing ratings for each of their respective items. Total score was calculated summing the scores of the four problem subscales (emotional symptoms, conduct problems, hyperactivity/impulsivity and peer problems).

Statistical analysis

The reliability of the subscales for teacher SDQ ratings was assessed using Cronbach-alpha coefficients and the average inter-item correlation which examines the extent to which scores on one item are related to scores on all other items in a scale. Ideally the Cronbach alpha should be greater than 0.7 (19) with α =0.70 and below considered low, values between α =0.70 – α = 0.80 considered acceptable and values α = 0.80 and above good (20). SDQ subscales contain five items each and since it is not uncommon with short scales with less than ten items to find low Cronbach alpha values, it is reasonable and more appropriate to use an alternative measure of internal consistency such as the averages inter - correlations of items within a scale (19). Briggs and Cheeck (21), recommend that the optimum range of the average inter-item correlation for a set of items should be between .20 and .40. We further explored the underlying structure of the teacher SDQ version using both confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). To evaluate model fit, CFA was conducted, and a range of fit statistics were calculated. For the incremental fit statistics the Tucker-Lewis Index ( TLI) and the Comparative Fit index (CFI) values less than .90 indicate lack of fit, values between .90 and .95 indicate reasonable fit and values between .95 and 1.00 indicate good fit (22). Byrne (23) describes the Root Mean-Square Error of Approximation (RMSEA) as the most informative statistic in determining model fit as it takes into account the number of variables that are estimated in the model. RMSE A values .05 or lower indicate good fit, Values between .05 and .08 indicate reasonable fit. Factor loading values of .32 or more were required for the variable to be interpreted; values > 45 were considered fair; >55 were considered good; > .63 were considered very good and > .71 were considered excellent (24). After the sample was first assessed for its suitability for factor analysis with Bartlett’s Test of Sphericity and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, to further investigate the number of constructs and structure of this measure, an EFA using Principal Component Analysis (PCA) used to extract the factors followed by oblique rotation of factors using Oblimin rotation (delta = 0) was conducted using SPSS version 24. To establish the number of factors to be retained we first applied Kaiser’s criterion (eigenvalues above 1), along with the inspection of the screeplot, and the use of Horn’s parallel analysis (25) using the software Monte Carlo PA, developed by Watkins (26). Only factors with eigenvalues exceeding the values obtained from the corresponding random data set were retained for further investigation.

RESULTS

Reliability

Cronbach’s alpha values (Table 1) for teacher-rated SDQ subscales ranged from .67 (prosocial problems) to .83 (hyperactivity-inattention) while average interitem correlations fell within the recommended optimum range and ranged from 0.20 for total difficulties scale to 0.50 for hyperactivity scale indicating acceptable to good internal consistency.

Table 1.

Internal Consistency of SDQ subscales

	Emotional problems	Conduct problems	Hyperactivity-innatention	Peer problems	Prosocial behaviour	Total difficulties
Cronbach-α	0.79	0.71	0.83	0.76	0.67	0.83
Mean Interitem correlation	0.45	0.37	0.50	0.4	0.30	0.20

Confirmatory factor analysis

We used AMOS 24 software to perform a CFA based on the data we obtained from teachers for a total of 542 children. We choosed maximum likelihood estimation because our data were normally distributed. The theoretical five factor model proposed by authors is presented in Figure 1. The chi-square test was significant (X2 (265)=1552.549; p<0.001) and the other fit indices (CFI= .814, TLI= 0.772; RMSEA= .095) indicated a misfit of the data to the five-factor model.

Figure 1

CFA path diagram of standardized estimates (hypothesized five – factor model)

Unstandardized estimates together with standardized parameter estimates, and the squared multiple correlation (SMC) values are shown in Table 2. The highest factor loading was found for the item on the prosocial scale (helpful = 0.99) and the lowest but still in the interpretable range was on the conduct scale (obedient=0.39). The squared multiple correlation (SMC) values indicate (lower bound) the reliability of the measure; “fights” (.92) and “obedient” (.15) have the highest and lowest, respectively. A range of alternative factor solutions were tried and no support was found for the three factor solution proposed from Dickey and Blumberg (2004), while other alternative modifications resulted in poor model fits.

Table 2.

Unstandardized and standardized loadings for CFA

Item	Scale	Loadings (unstandardized)	Standard error	P	Loadings (standardized)	Squared Multiple Correlations
24. Easily Scared	Emotional	1.00			.50	.25
16. Nervous	Emotional	1.36	.13	***	.66	.44
13. Unhappy	Emotional	1.24	.12	***	.76	.58
8. Worries	Emotional	1.35	.12	***	.81	.65
3. Somatic	Emotional	1.09	.11	***	.63	.39
22. Steals	Conduct	1.000			.44	.19
18. Lies Or Cheats	Conduct	1.54	.21	***	.40	.16
12. Fights	Conduct	4.48	.43	***	.96	.92
7. Obedient	Conduct	2.22	.31	***	.40	.15
5. Tempers	Conduct	3.86	.38	***	.80	.64
25. Good Attention	Hyper	1.000			.90	.80
21. Thinks Before Acting	Hyper	.77	.05	***	.69	.48
15. Distracted	Hyper	.98	.04	***	.82	.68
10. Fidgets	Hyper	.57	.05	***	.52	.27
2. Restless	Hyper	.62	.06	***	.50	.25
23. Better With Adults	Peer	1.00			.40	.17
19. Picked On Or Bullied	Peer	.76	.10	***	.52	.27
14. Others Like	Peer	2.05	.24	***	.82	.67
11. Good Friend	Peer	1.85	.22	***	.77	.59
6. Alone	Peer	1.42	.18	***	.61	.37
20. Volunteers	Posocial	1.00			1.00	.99
17. Kind To Child	Posocial	.46	.04	***	.45	.20
9. Helpful	Posocial	.98	.01	***	.98	.97
4. Shares Readily	Posocial	.55	.05	***	.47	.22
1. Considerate	Posocial	.42	.04	***	.39	.15

Exploratory factor analysis

The samples were first assessed for their suitability for factor analysis. Bartlett’s Test of Sphericity was highly significant (p < .001) and the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy (KM0=0.810) value supported the factorability of the matrix (21). Principal components analysis revealed six eigenvalues exceeding 1, explaining 57.38% of the variance. Despite the initial extraction resulted in six components with eigenvalues greater than 1, only the first five components exceeded the criterion value obtained from Parallel Analysis (26) and this was further supported from the inspection of the screeplot. So the number of factors that we retained for further analysis was five.

The PCA with oblimin rotation of teacher ratings (Table 3) revealed a different five - component pattern matrix.

Table 3.

Pattern And Structure Matrix for PCA with Oblimin Rotation of Five Factor Solution of SDQ teacher ratings

	Pattern Matrix					Structure Matrix					Co.a
Items	1	2	3	4	5	1	2	3	4	5
8. Worries	0.80					0.80					0.666
13. Unhappy	0.76					0.77					0.608
3. Somatic	0.74					0.72					0.527
24. Easily Scared	0.69					0.69			0.42		0.570
16. Nervous	0.60					0.66				0.40	0.529
22. Steals	0.44					0.51					0.288
14. Others Like*		0.83					0.82				0.682
11. Good Friend*		0.79					0.79				0.636
6. Alone		0.73					0.74				0.556
19. Picked On Or Bullied		0.65					0.66				0.461
23. Better With Adults		0.57					0.56				0.328
20. Volunteers			0.80					0.84		-0.47	0.824
9. Helpful			0.79					0.84		-0.47	0.813
17. Kind To Child			0.69					0.68			0.484
4. Shares Readily			0.67					0.65			0.449
1. Considerate			0.61					0.65			0.473
2. Restless				0.80					0.83		0.724
10. Fidgets				0.80					0.82		0.748
5. Tempers				0.67		0.46			0.74		0.619
12. Fights				0.61		0.55			0.74		0.683
7. Obedient*			-0.40	0.40				-0.45	0.47		0.373
15. Distracted					0.83					0.86	0.758
25. Good Attention*					0.76					0.82	0.745
21. Thinks Before Acting*					0.63				0.41	0.70	0.603
18. Lies Or Cheats					0.42	0.46				0.52	0.397

Comunalities

Reversely scored items

Note: Major loadings (>0.40) for each item are presented

The items of the three SDQ subscales of emotional problems, peer problems and prosocial behavior subscales loaded in the predicted subscales and the loadings fell within the “good” to “excellent” range. Items of two other subscales, specifically items of conduct problems and hyperactivity/inattention scales loaded differently from the predicted SDQ model, with some cross-loadings in more than one component. So, the items, 2 (“restless”), 10 (“fidgets”), 5 (“tempers”), of the hyperactivity scale and items 12 (“fights”) and 7 (“obedient”), loaded strongly within the same component, with the exception of “obedient”, which loaded poorly in this component and cross loaded also poorly on the component of prosocial scale. Furthermore, the rest of the items from hyperactivity scale, 15 (“distracted”), 25 (“good attention”), 21 (“thinks before acting”) and conduct scale 18 (“lies or cheats”), loaded in one separate component and item 22 (“steals”) loaded fairly (item 22=.44) in the component of emotional scale. Examination of structure matrix did not reveal a clear five factor solution.

DISCUSSION

This study examined the internal consistency of the Albanian SDQ version for teachers and both confirmatory and exploratory and factor analysis were performed to examine the fit of the obtained data to the proposed five factor model. Results from a number of studies show acceptable internal consistency for the SDQ subscales for teacher ratings (14) with an adequate internal consistency for the hyperactivity scale. In this study, Cronbach alpha values overall fell within the “acceptable to good range” with the exception of prosocial behavior scales that were considered low. The hyperactivity subscale had the highest value (α =0.83) indicating an adequate fit. Average inter-item correlations values fell within the optimum range recommended for adequate internal consistency and hyperactivity scale had the highest value of 0.50. In conducting CFA, the five-factor model misrepresented the data for teacher ratings and despite attempted modifications no alternative structural model that could fit the data, could be detected. Exploratory factor analysis sustained a five-factor structure but revealed a different pattern of factor loadings for the items of conduct and hyperactivity scales, suggesting a shared contribution of the items in both factors and further a poor differentiation between these two factors. Other studies that have examined the factor structure of the SDQ for parent and teacher ratings have also failed to confirm the five-factor structure of this measure (14) suggesting that the instrument may be measuring different constructs across the groups (17). Hill and Hughes (27) found only marginal baseline fit for a five-factor structure for either informant in their sample of US children (mean age six). Another study (16) also found a different pattern of loadings for the items of conduct scale and a tendency of positively worded items (“obedient”) to load on the prosocial scale. Additionally, other authors that have conducted researches in the field of child psychopathology have reported difficulties in the differentiating between hyperactivity and aggression. In an attempt to disentangle the potentially confounded domains of hyperactivity and conduct problems, Milich and Loney (28) began to derive independent dimensions of hyperactivity and aggression, resulting in a new measuring instrument designed to differentiate between the two dimensions of behavior, but surprisingly, the correlation between the two factors was r = .63, suggesting for an inherent association between these two domains (29). In another study, Schachar , Sandberg, Rutter( 30), examined the agreement between teachers’ ratings and observations of hyperactivity, inattentiveness, and defiance, and found that defiance was more reliably rated than hyperactivity or inattentiveness. In particular, hyperactivity ratings may be influenced by observed defiance and inattentiveness, inattentiveness ratings by observed defiance, and behavior disorder by observed hyperactivity and inattentiveness. So, defiance toward a teacher increased the likelihood that a child would be rated as hyperactive or inattentive regardless of his observed level of activity or attentiveness, indicating that a child’s defiance and disobedience are significant causes of misclassification. This is also known as the halo effect for teachers and it refers to the influence of one class of behaviors into the perception and further ratings of other behaviors (31, 32). Our exploratory factor analysis of teacher ratings revealed also that the items indicating the inattentiveness loaded together in a separate component, suggesting for a separate factor consistent with the arguments of some authors that suggest considering ADHD/I subtype as a distinct diagnostic category (33). Taken together, these differences suggest that certain behaviors might be external manifestations of a range of distinct childhood psychiatric disorders but cannot be considered unique to these disorders.

LIMITATIONS

Our results must be interpreted with caution in the light of limitations that this study has. First, the sample was not representative of the population of elementary school children and it was selected based on an administrative distribution of elementary schools (schools of only one district of Tirana city were included in the study), indicating the need for further studies in more representative samples of the Albanian youth population. Second, the study was performed in a low risk population and as Goodman (34) suggests when screening for child mental health problems in low-risk epidemiological samples, considering of broader internalizing or externalizing rating scales might be more appropriate. Additionally, feasibility issues need to be further investigated in order to increase the participation and collaboration of school staff for the early screening of mental health problems in youths. Volpe, et al (35), argue that although the efficiency with which a wide range of behavioral data can be obtained makes behavior rating scales particularly attractive tools for the purposes of screening and evaluation, feasibility concerns arise in the context of formative assessment. Specifically, informant load, or the amount of time informants are asked to contribute to the assessment process, likely has a negative impact on the quality of data over time and the informant’s willingness to participate.

CONCLUSION

This is the first study to examine the psychometric properties of the SDQ in a population of Albanian children. Overall these preliminary results support the five dimensions of SDQ with satisfying subscale reliability based on internal consistency coefficients. However, the content of the factors obtained do not fully support the original factor structure proposed by the authors. Because of the suggested non-one-dimensional contribution of several SDQ items, caution is recommended when using the SDQ in Albanian children when screening for Attention Deficit/ Hyperactivity Disorder in elementary school population. Further investigations in larger and more representative samples are recommended and establishment of SDQ norms for Albanian youths must be a priority.