Psychometric properties of Persian version of the multifactor leadership questionnaire (MLQ).

BACKGROUND: Outstanding leadership is one of the important and vital concepts in management and educational leadership debates especially in educational organizations such as universities. Thus, effective educational leadership and adopting an appropriate tool to assess leadership in education are crucial in these institutions. The present study was conducted to develop an instrument for measuring the leadership style in faculty members.
METHODS: In this descriptive cross-sectional study, content and face validity of Multifactor Leadership Questionnaire (MLQ) was examined using the opinions of 10 experienced faculty members as a panel of experts. For construct validity, 210 questionnaires were administered to faculty members of Tehran University of Medical Sciences. Confirmatory factor analysis was run using principal component analysis and Varimax rotation method. The reliability of the scale was measured through internal consistency using Cronbach's alpha formula. Confirmatory factor analysis was used to examine the construct validity of the questionnaire. Data were analyzed using SPSS v.16 and LISREL software.
RESULTS: Factor analysis and expert opinion resulted in a questionnaire with 18 items across six subscales including idealized influence, inspirational motivation, intellectual stimulation, individualized consideration, contingent reward, and management by exception. The reliability coefficient of the questionnaire was acceptable (0.90). Confirmatory factor analysis showed that the scale had appropriate goodness of fit.
CONCLUSION: The 18-item Educational Leadership Questionnaire was found to have acceptable validity and reliability for measuring leadership style in the faculty. It is recommended that the questionnaire be administered to a larger sample.

Introduction

Leadership concept goes back to 5000 years ago to Egyptian hieroglyphics in which the words (sehemu) “leader” and (seshement) “leadership” are used .This was the assumption behind ‘The Great Man Theory’. The human relations approach to leadership, which was developed after Mayo’s studies, views leadership as an interactional phenomenon depending on the interrelation of the group members pursuing common goals of the organization. Hence, with a humanistic perspective, leadership is defined in terms of status, interaction, perceptions, and behavior of individuals in relation to other members within an organized group (1).

Since the dawn of the twenty-first century, marked attention has been paid to educational leadership so much as it has been considered as the key factor in organizational success or failure. Once educational institutions wish to provide the best-quality education for their leaners, they need to apply effective leadership. Leadership is thought to contribute to organizational effectiveness as much as 45-50 percent (2).There is a great and widespread interest in educational leadership and management, a trend that has been accelerating in the twenty-first century. The widely accepted opinion that effective leadership is essential for successful schooling is increasingly being supported by document of its beneficial effects. Leadership is now recognized to be the second most significant factor influencing school and pupil outcomes, after classroom practice. Schools often succeed because of the skill and commitment of their principals and chief teams. (3).

According to Stogdill and Arnold, leadership is a complicated concept bearing as many definitions as the number of researchers that have addressed it. This may be why precise definition of the concept is particularly important in non-humanities disciplines such as medicine and especially medical education (4). Leadership in educational institutions such as schools, universities and higher education institutions is not bound to a single individual. Rather, leadership in such organizations is an approach to consolidating all skills for promoting learning. As the educational leader, a head teacher headmaster only facilitates the conditions for learning. In this regard, leadership is a collective attempt made inside educational institutions by educational leaders to improve teaching-learning activities (5).

The major challenges of educational leadership in universities of medical sciences could be categorized into three themes (with three subthemes in parentheses): organizational challenges (ineffective public education leadership, wide-ranging responsibilities and missions, and a lack of concern for the selection of managers), managerial challenges (management styles, mismatch between responsibilities and authority, and educational leadership abilities), and cultural challenges (willingness toward public administration, a boss-centered culture, and low motivation) (6). Identification of leadership styles and models requires assessment. Organizations may conduct self-assessments and continually compare their management style with the role-model style in order to improve their trends and reach their business goals. As such, an instrument should be developed for this purpose. However, researchers need to examine the validity and reliability of an instrument before they use it in real world. Validity is a concept used to examine whether or not the instrument can measure what it claims. Without a sound understanding of the instrument validity, one may not trust the data produced by the instrument. Reliability, on the other hand, is concerned with to what extent the instrument can produce similar results under similar conditions. In other words, if we administer the instrument to the same group of individuals over short intervals, the results should be consistent (7).

Multifactor Leadership Questionnaire (MLQ) is one of the standard tools used to measure leadership styles in organizations (8). Ibrahim and Al-Taneiji studied the reliability of MLQ in Dubai in 2013. They reported the internal consistency of the scale to be 0.95 using Cronbach’s alpha formula. However, they did not address the validity of the questionnaire (9). Armstrong measured reliability of the scale to be 0.86 for the English version of the questionnaire and 0.87 for the Thai version using Cronbach’s alpha formula. Construct validity of the questionnaire was estimated using confirmatory factor analysis through maximum likelihood estimation method in AMOS software. The results revealed that none of the obtained models suited the research data so that validity improved once the relationships of select parameters was changed (10). In a study, Hallaji and colleagues administered MLQ x5 comprising 41 items to a number of 135 individuals. The reliability of all subscales was calculated using SPSS, which yielded a reliability index of 0.95. However, they did not report the construct validity of the questionnaire (11).

MLQ comes in several versions. Originally it was developed by Bernard (Bernie) Bass and Bruce Avolio, two professors of Binghamton University. MLQ x5 is one of the most frequently used versions broadly used in many subject areas. It measures transactional, transformational and avoidant leadership styles. MLQ x5 comprises 45 items where the first 36 items measure the type of leadership style and the next 9 items examine the effectiveness of the manager. MLQ-6S is another version of the questionnaire comprising 21 items. It measures 7 factors pertaining to transformational, transactional and laissez-faire leadership (8). Following consultation with experts on the importance of leadership style in education and the necessity to adopt an appropriate instrument for measuring the construct, the present researcher studied 36 leadership questionnaires and adopted MLQ due to its affinity with educational institutions. Accordingly, we examined the reliability and validity of MLQ-S6. To the best of the authors’ knowledge, no studies in Iran have already addressed the issue.

Methods

Test Development

The study adopted a descriptive method and a cross-sectional approach. Following extensive consultation with experts and comprehensive review of the related literature, the original version of the questionnaire was selected to be used in the study. A query was sent to the developers of the questionnaire on the use of the email version of the scale. The developers agreed with the application of the email version. The research instrument was MLQ-S6 comprising 21 items on a 5-point Likert-type scale including not at all, once in a while, sometimes, fairly often, and frequently if not always. The score for every item ranged from 1 (not at all) to 5 (frequently if not always). The questionnaire was translated and culturally adapted into Farsi according to the method proposed by Wild and colleagues in the following steps (12). In the first stage of forward translation, the scale was translated by two expert translators who were faculty members. One of the translators had a PhD in English, and another one had a PhD in healthcare education. They both were experienced in translating specialized texts and were familiar with the concepts addressed in the scale. In the second stage, the two translations were integrated into a single translation for the reconciliation of forward translation. In the third stage, the final translated version was back-translated into English by two faculty members who were fluent in both Farsi and English. One of the translators had studied management, and another one, the English. In the fourth stage, the back translated versions were reviewed. In the fifth stage, the questionnaire was administered to a few participants in a pilot study in order to examine its wording, scoring, and rating of statements and its components were amended. In the sixth stage of finalization and proofreading, modifications were made in the scales, and their final version was prepared for psychometrics. In the seventh stage, the psychometric properties of the scale were determined including content, face and construct validity.

Content Validity Ratio (CVR) was used to ensure that the scale contains the most important and most accurate content. Content validity index (CVI) was used to ensure that the questionnaire items are best developed to measure the content. The opinions of 10 faculty members of Tehran University of Medical Sciences were sought to examine the face and content validity of the scale using CVR and CVI methods both qualitatively and quantitatively. Of the faculty members, 5 people had studied management or passed management courses and 5 had studied educational sciences or other disciplines. They were considered as the panel of experts who were specialized in developing and administering research instruments related to the topic of the study. The content validity of all items was obtained by averaging the CVR values. The content validity index was measured in terms of four criteria: relevance, clarity, importance, and simplicity. The relevance criterion was examined at four levels: (1) irrelevant (with a score of 1), relevant but requiring serious adjustments (with a score of 2), relevant but requiring partial adjustments (with a score of 3), and completely relevant (with a score of 4). Based on the expert opinion, items with a CVI above 79% were considered as proper statements while items with a CVI of 70-79 were regarded as the items requiring modification. The items with a CVI of less than 70% were eliminated from the scale (13).

Difficulty, irrelevancy and ambiguity were studied in the questionnaire to examine and correct its face validity. The faculty expressed their opinions about editing and adjusting the items consistent with Iranian culture. In the next stage, item impact method was used to eliminate inappropriate items and determine item importance. Since factor analysis depends on the sample size that is 10-15 subjects per variable, 10 respondents were selected per variable to measure the construct validity (14). With regard to the minimum sample size in factor analysis, different sources suggest a minimum of 1 to 5 ratio for the proportion of variable to subject. Thus, the present sample size is more than enough considering the number of items used in exploratory factor analysis. Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy suggests that a KMO value of 0.6 and above is indicative of the adequacy of sample (15).

Since there were 21 items on the questionnaire, the sample size was considered to be n= 210. In order to avoid sample attrition, 40 individuals were added to the sample size. Therefore, a number of 250 questionnaires were administered to the faculty. The number of respondents was computed using stratified-random sampling consistent with the number of faculty members in each college. In this regard, Tehran University of Medical Sciences was divided into 12 strata including all colleges and research centers. Of every stratum, a number of five groups were selected randomly. Subsequently, all groups were divided into two strata of male and female. All colleges were listed separately. Eventually, the sample size was computed to be 250 consistent with the number of professors at college strata and gender. Every college was selected as a cluster using the random number table.

Of the administered questionnaires, the faculty members of Tehran University of Medical Sciences returned 210 ones. For ethical purposes, the respondents were informed of the research objectives and confidentiality of the data before the sampling stage. Subsequently, the completed questionnaires were collected, and the data were entered into the software.

Data analysis

Principal component analysis (PCA) and orthogonal rotation method (Varimax) were run to conduct exploratory analysis of the model. Factor loading was considered to be above 0.3 in the factor analysis. Confirmatory factor analysis was run to examine construct validity. Factor analysis is done to reduce large number of variables into a limited number of factors with minimum data attrition. Confirmatory factor analysis was used through maximum likelihood estimation to examine the structural validity of the scale. In other words, confirmatory factor analysis revealed whether or not the questionnaire items were assigned to and fit the relevant factors as theoretically expected. The analysis was done using LISREL software. The models were analyzed using 7 indices including goodness of fit index (GFI), comparative fit index (CFI), incremental fit index (IFI), chi-square goodness of fit index (X2), root mean square error of approximation (RMSEA), root mean square residual (PMR), and adjusted goodness of fit (AGFI).The reliability of the questionnaire was estimated through internal consistently method using Cronbach’s alpha formula. The analysis was performed using LISREL and SPSS software.

Results

As discussed above, MLQ-S6 consists of 21 items that measure 7 factors pertaining to leadership.

Step 1: MLQ was presented to the panel of experts to measure its content validity, and the scores were collected. Overall, one item was added and one was eliminated from the questionnaire. Items with a CVI above 0.70 were retained in the questionnaire. The results of data analysis revealed the following results: A) relevance with a mean of 94%, a standard deviation (SD) of 1.8 and the highest percentage of 98%; B) clarity with a mean of 94%, a SD of 2.03 and the highest percentage of 100%; C) simplicity with a mean of 95%, a SD of 1.04 and the highest percentage of 100%. Based on quantitative results of CVR, items with a content validity of 0.80 and items with a significance level of p≤0.05 were considered as necessary and important, so that they were retained in the questionnaire, and other items were eliminated.

Following reevaluation and consultation with the experts, some recommendations were approved of, and the 21-item questionnaire was adjusted to be consistent with Iranian culture. Of the 250 questionnaires administered to the sample, 210 respondents completed and returned the questionnaires; thus, the return rate was 84%.

Step 2: Exploratory factor analysis through principle component analysis (PCA) method was used to précis the number of items. The items with an eigenvalue above 1 were retained. Orthogonal rotation through Varimax rotation was run to obtain a simple factor structure. Following the adjustment of the obtained fit, the model was assigned as follows:

Factor 1: idealized influence obtained by totaling the scores of items 1-8.

Factor 2: inspirational motivation obtained by totaling the scores of items 2,9,16

Factor 3: intellectual stimulation obtained by totaling the scores of items 3,10,17.

Factor 4: individualized consideration obtained by totaling the scores of items 4,11,18.

Factor 5: contingent reward obtained by totaling the scores of items 5,12,13,19.

Factor 6: management by exception obtained by totaling the scores of items 6,14,20.

Factor 7: Laissez-faire obtained by totaling the scores of items 7,15,21.

Variable commonality was above 0.4 in all cases. The factors explained the total variance by 61.57%. The eigenvalues for the seven subscales were as follows: idealized influence (84.51%), inspirational motivation (58.55%), intellectual stimulation (66.58%), individualized consideration (49.17%), contingent reward (54.33%), management by exception (53.39%), and laissez-faire (40.99%). The exploratory factor loadings ranged from 0.54 to 0.76 for idealized influence, from 0.49 to 0.76 for inspirational motivation, from 0.36 to 0.77 for intellectual stimulation, from 0.49 to 0.66 for individualized consideration, from 0.60 to 0.71 for contingent reward, and from 0.69 to 1 for management by exception. It is notable that the subscale of idealized influence had the highest eigenvalue. There was a good correlation among variables, which is illustrated in Table 1.

Table 1

Correlation coefficient among factors

Factors	Idealized influence	Inspirational motivation	Intellectual stimulation	Individualized consideration	Contingent reward	Management by exception	Laissez-faire
Idealized influence	1	76.0	54.0	61.0	73.0	69.0	64.0
Inspirational motivation		1	76.0	70.0	72.0	50.0	49.0
Intellectual stimulation			1	77.0	60.0	36.0	46.0
Individualized consideration				1	69.0	49.0	49.0
Contingent reward					1	71.0	60.0
Management by exception						1	69.0
Laissez-faire							1

Step 3. The scale showed good internal consistency using Cronbach’s alpha coefficient as illustrated in Table 2. However, while computing alpha coefficients for individual factors, laissez-faire yielded a small, poor coefficient. Therefore, laissez-faire was eliminated from the model. Consequently, the instrument was finalized with 18 items and 6 factors. The scale yielded a total reliability coefficient of 0.90.

Table 2

Reliability coefficients of the total scale and every factor using Cronbach’s alpha formula

No.	Scale	No. of items	Cronbach's Alpha
1	Idealized influence	2	81.0
2	Inspirational motivation	3	62.0
3	Intellectual stimulation	3	74.0
4	Individualized consideration	3	69.0
5	Contingent reward	4	70.0
6	Management by exception	3	64.0
7	Laissez-faire	3	05.0
8	Total (factor 7 eliminated)	18	90.0

Two raters observed and rated the on-field performance of managers. The correlation coefficient of their observations was ICC=0.88. Internal consistency of the scale and its subscales was computed using Cronbach’s alpha formula. All coefficients were acceptable except for the factor 7. The inter-rater reliability of the scale was calculated to be ICC=0.90.

Step 4: Based on the exploratory factor analysis performed in the previous step, the scale was examined using confirmatory factor analysis. Model fit indices were examined in the LISREL software, and the results are illustrated in Table 3.

Table 3

Fit indices of the 6-factor leadership model with an educational approach

Model	χ2	df	GFI	AGFI	CFI	NFI	IFI	RMR	RMSEA
Adjusted 6-factor model	29.391	89	94.0	91.0	88.0	85.0	87.0	04.0	06.0

GFI: Goodness of Fit Index, AGFI: Adjust Goodness of Fit Index, CFI: Comparative Fit Index, NFI: Normed Fit Index, IFI: Incremental Fit Index, PMR: Root Mean Square Residual, χ2: Chi-square goodness of fit Index, RMSEA: Root Mean Square Error of Approximation

Based on the results of model fit illustrated in Table 3 and the following diagram along with factor loading and PMR in Figure 1, the results supported the good fit of the model. The final results confirmed the validity and reliability of the 6-factor educational leadership tool.

Fig. 1

Estimated 6-factor model of educational leadership

Discussion

The results of validating Bass and Avolio’s MLQ through a 7-factor model with confirmatory and exploratory factor analysis techniques, percentage of the explained variance and model fit index showed acceptable validity and reliability. As discussed above, the study aimed to develop a valid and reliable model of an instrument to be used for measuring leadership styles in universities. To develop such a model, an appropriate instrument is required which could precisely measure the range and nature of leadership styles in educational institutions. Considering the importance of leadership style in teaching, learning and academic achievement, it is necessary to develop a valid and reliable instrument for such measurement. Various instruments have been developed to measure leadership styles based on the research objectives. MLQ has been used in studying leadership styles in various institutions and has yielded better results comparing with other instruments (10).

MLQ was used in the present study based on extensive review of the literature and consistent with the research objectives to develop a valid and reliable instrument in Iranian society and examine educational leadership style in Iranian universities as the first step in the leadership and management process in classrooms and universities.

MLQ was used in the present study for it is a questionnaire based on the leadership definition addressing three main components of leadership behavior including transformational, transactional and laissez-faire leadership across 7 factors. Moreover, questionnaire items provide good information on the range and nature of leadership behavior. In addition, it has been used in many studies in different countries.

Following, translation and back translation of the questionnaire as the standard method of cultural adaptation, content validity of the scale was examined in the first step using CVR and CVI. This was because of the likely difference between cultural norms in Iran as well as cultural context and university regulations with the questionnaire original content.

As it is typical in research, content validity is judged by a panel of experts. For example, in a study to develop and validate ten scales of a tool for prevention of AIDS, a panel of 15 experts was used to examine the content validity (16). In some studies, a quantitative index is used to determine the content validity so that experts are asked to express their opinion on the validity of every item by allocating a score. Finally, following calculations, a number is announced as the content validity index (17).

Subsequently, structural validity of the questionnaire was measured. It, however, should be noted that most of the studies done to validate this scale have used either Rasch method (18), concurrent validity (19), convergent and discriminant validity, or in some cases exploratory factor analysis (10,20).

A review of the literature revealed that no study have used confirmatory factor analysis on this questionnaire in Iran. Since confirmatory factor analysis is to examine to what extent the hypothesized or predicted relations among variables correspond to the actual observed data, it is considered as the best approach to measure the structural validity of a theoretical model (21).

In a study by Hemsworth and colleagues, they reported the internal consistency of the scale to be 0.47 while the present findings showed that the scale had an internal consistency of 0.88. A study finalized MLQ with 5 factors using confirmatory factor analysis while the psychometrics of the scale was confirmed with 6 factors in the present study (22).

Overall, the reliability and validity of the leadership style subscales have been confirmed in the studies conducted by Bass, Avolio, Sosik, Gellis and colleagues as well as Felfe and Lee and colleagues. Factor loadings to examine the construct validity of the scale have been reported to vary between 0.62 and 0.91 using confirmatory factor analysis. Despite Felfe and Lschyns in 2004 who reported a poor reliability of the scale (23), the present findings revealed that the scale had good reliability.

Reliability of the scale was measured using internal consistency method. The results showed that all MLQ subscales (factors) had good reliability except for laissez-faire. Since most sources consider an alpha coefficient of 0.7 or above to be acceptable, it is necessary to calculate alpha coefficient of the whole as well as every subscale (24). In this regard, the Farsi version of the questionnaire showed acceptable reliability both as a whole and as individual subscales. The reliability coefficient of the questionnaire was 0.90.

Factor 7 (laissez-faire) was eliminated from the scale as it showed poor reliability. The results of confirmatory factor analysis revealed that the data had good fit with the 6 constructs. The correlation among factors in each subscale was above average and statistically significant.

The present researchers have provided comprehensive and accurate information as to how they measured the reliability and validity of their developed scale. However, many of the previous studies cited in this paper have not provided information as to the method they adopted to examine the validity and reliability of their scale even when they reported acceptable reliability and validity indices. Considering the direct association between the credibility of research findings and reliability of the instrument, it is crucial for the researchers to pay greater attention to the reliability of research instrument. In the present study, the researchers tried to provide enough information as to the validation process of the research instrument to examine educational leadership styles so that the readers could assure the reliability and validity of the scale. The strengths of the developed scale lie in the psychometric process adopted to develop it, the use of target group ideas, expert opinion, simplicity, clarity, brevity and logical sequencing of the questionnaire items. The limitations of the study include lack of similar studies at the national level and lack of previous examination of the scale at educational institutions to ensure its generalizability.

Conclusion

The results showed that MLQ had acceptable reliability and validity in faculty members of Tehran University of Medical Sciences. Thus, researchers can use the scale as a valid instrument in measuring leadership styles. It, however, should be noted that the present sample were faculty members of Tehran University of Medical Sciences. Thus, care should be taken in generalizing the results to other universities or higher education institutions. It is recommended that the scale be validated in other provinces and universities as well.

Acknowledgements

Our special thanks go to professor Bernard (Bernie) Bass from Binghamton University who provided us with the questionnaire. We should like to thank the professors of Tehran University of Medical Sciences for completing the questionnaires.