Psychometric properties of the mentor behaviour scale in a sample of Malaysian medical students.

OBJECTIVES: To determine the psychometric properties of the Mentor Behaviour Scale (MBS), a 15-item inventory that evaluates four supportive mentor behaviours in terms of construct validity and internal consistency.
METHOD: A cross-sectional study was carried out on a sample of medical students in their final year at Universiti Sains Malaysia. Confirmatory factor analysis (CFA) was performed using AMOS 22 to assess construct validity. Reliability analysis was performed using SPSS 22 to assess internal consistency.
RESULTS: A total of 159 final year medical students participated. CFA showed that the original four-factor model with 15 items achieved acceptable values for the goodness of fit indices, suggesting a good model fit (X2 = 198.295, ChiSq/df = 2.418, RMSEA = 0.095, GFI = 0.867, CFI = 0.953, NFI = 0.923, TLI = 0.940). The Cronbach's alpha values of the mentoring relationship structure, engagement, and competency support domains were 0.96, 0.90 and 0.88, respectively. For autonomy support, the Cronbach's alpha value was 0.62.
CONCLUSION: MBS demonstrates a satisfactory level of construct validity and a high level of internal consistency in measuring supportive mentor behaviours in a medical school setting. This result suggests that MBS can be used as a mentorship evaluation tool for feedback in the context of a Malaysian medical school.

Introduction

Mentoring is a two-way dynamic, a symbiotic and complex relationship between mentor and mentee resulting in career advancement and satisfaction.1, 2 It should be an intentional, nurturing, protective and supportive process that involves acting as a role model.3, 4 Generally, it aims to help, support and enhance performance of mentees during learning and training, and thus prevent problematic behaviours during the training.5, 6 Since mentoring is a symbiotic mutual relationship between two parties, one of the important factors is the mentor's behaviours towards the mentees during the relationship. In a medical setting, mentorship is one of the keys to a successful career pathway7, 8, 9 and having a mentor helps to increase academic commitment and focus, professionalism and wellbeing during medical training.10, 11, 12

From the literature, there is a potential tool to measure mentor behaviour which is the Mentor Behaviour Scale (MBS). It was developed to measure supportive mentor behaviours including mentoring relationship structure, engagement, autonomy support, and competency support. It is a simple, practical and concise tool for measuring attributes of the supportive mentor behaviour and was reported to have good validity and reliability for measuring the mentoring behaviours. Validity is described as the ability of a tool to measure the attributes that are intended to be measured, and reliability is the consistency or reproducibility of measurements over time or on different occasions.14, 15 The MBS developer found that it has high reliability, as three domains (mentoring relationship structure, engagement and competency support) attained Cronbach's alpha coefficient values of more than 0.7, while the autonomy support domain attained a satisfactory coefficient value. With respect to validity, post-exploratory factor analysis and confirmatory factor analysis showed that the 15 items of MBS achieved acceptable results for goodness of fit indices, thus supporting its construct validity. Despite its potential for research, training and evaluation of mentoring behaviours, there is only one study that provided data on its validity. Therefore, further validation studies are required to establish its psychometric properties in other settings.

Validity can be assessed by content (i.e., content validity), response process (i.e., face validity), internal structure (i.e., construct validity and internal consistency), relations to other variables (i.e., predictive validity) and consequences (i.e., impact of the measurement, e.g., pass and fail).16, 17 This study aimed to evaluate the construct validity and internal consistency of MBS among medical students in a Malaysian medical school.

Materials and Methods

Study design and ethical clearance

A cross-sectional study was carried out on 159 medical students at a public Malaysian medical school. They responded to the inventory voluntarily. Ethical clearance was sought from the Human Ethics Committee Universiti Sains Malaysia (USM/JEPeM/15070250).

Sample size and sampling method

The sample size was estimated based on a recommended ratio of 5–10 subjects per item; suggesting that 75 to 150 subjects would be sufficient for testing the internal structure of a 15-item scale (i.e., reliability and construct validity). Purposive sampling was performed to select participants and consent was sought from them before the study. All fifth year medical students were called to a face-to-face session in a hall and they were given a short briefing (less than 5 min) on MBS. The English version of MBS was used in this study. Those who agreed to be part of the study were required to fill out the questionnaire and asked to return it immediately upon completion.

The mentor behaviour scale (MBS)

The MBS is a 15-item scale that was developed and validated by Brodeur et al. Written permission was sought from the developer through e-mail before the study. It is a self-reporting questionnaire that measures the mentee's perceptions on four domains of mentor behaviours, including mentoring relationship structure, engagement, autonomy support and competency support. The tool uses a 5-Likert scale ranging from 1 (does not apply at all to my situation) to 5 (applies very well to my situation). Items 11 and 12 were negative items, requiring reverse scoring prior to data analysis. Mentoring relationship structure is measured in items 1 to 8, which ask about supportive mentor behaviours related to giving feedback, organizing meetings and discussing goals with the mentee. Engagement is measured in items 9 and 10, which ask about the ways the mentor established rapport and bonding, spent quality time and listened attentively to their mentees during mentoring sessions. Autonomy support is measured in items 11 and 12, which ask about the types of assistance provided by mentors to mentees in the decision making process. Competency support is measured in items 13 to 15, which ask about the positive supports mentors provided to mentees in any other situations.

Data analysis

The internal structure of MBS was evaluated by confirmatory factor analysis (CFA) using Analysis of Moment Structure software version 22 (AMOS 22). The goodness of fit indices were assessed to signify the latent constructs of MBS. Relevant indices for goodness of fit were summarized in Table 1 [adopted from Yusoff & Arifin] and the model fit is achieved if it attains the acceptance level.

Table 1

Goodness of fit indices that were used to signify the model fit.

Name of category	Name of index	Level of acceptance
Absolute fita	Root mean square of error approximation (RMSEA)	Less than 0.0827
	Goodness of fit index (GFI)	More than 0.928
Incremental fitb	Comparative fit index (CFI)	More than 0.929
	Tucker–Lewis index (TLI)	More than 0.930
	Normed fit index (NFI)	More than 0.931
Parsimonious fitc	Chi square/degree of freedom (Chisq/df)	Less than 532

Absolute fit: Measures overall goodness-of-fit for both the structural and measurement models collectively. This type of measure does not make any comparison to a specified null model (incremental fit measure) or adjust for the number of parameters in the estimated model (parsimonious fit measure).

Incremental fit: Measures goodness-of-fit that compares the current model to a specified “null” (independence) model to determine the degree of improvement over the null model.

Parsimonious fit: Measures goodness-of-fit representing the degree of model fit per estimated coefficient. This measure attempts to correct for any “overfitting” of the model and evaluates the parsimony of the model compared to the goodness-of-fit.

The contributions of observed variables (i.e., the items of MBS) to respective latent variables (i.e., the four domains) were approximated by the standardized factor loadings (SFL) – high loadings indicate high contribution of items to the domain. Modification indices (MI) estimate correlations between variables and reduction of chi-square values are expected if these correlations contribute to the model fitness. Therefore, items were retained in the model if they met the acceptable values of SFL (more than 0.5) and MI (more than 15). Even so, removal of any items should be based on literature review or have a theoretical basis.21, 22, 23

The internal consistency of MBS was estimated by Cronbach's alpha coefficient calculated by using reliability analysis through Statistical Package for Social Sciences software version 22 (SPSS 22). Any values greater than 0.7 are considered as indicating high internal consistency.

Construct validity of MBS was assessed through convergent validity and discriminant validity, indicating the internal structure of respective domains. Convergent validity was confirmed by size of factor loading, average variance extracted (AVE) values and composite reliability (CR) values. Item factor loading values should be reasonably high for the respective domains to signify convergent validity. The authors calculated AVE and CR manually based on the recommendations of previous studies.24, 25 Convergent validity was achieved if AVE values were more than 0.5 and CR values were more than 0.6.23, 24 Discriminant validity was tested by comparing the domains' shared variance (SV) and AVE values. SV is given as the square of correlation between two constructs. AVE values higher than the SV values signified an acceptable level of discriminant validity. A correlation between constructs of less than 0.95 was considered to signify acceptable discriminant validity.

Results

Table 2 summarizes the CFA results. The analysis revealed a one-factor model of MBS failed to achieve a model fit, indicating that MBS has multiple constructs. The results showed the original four-factor model with 15 items achieved acceptable values of the goodness of fit indices, suggesting good model fit. In addition, the correlation values between the constructs were less than 0.95, indicating acceptable discriminant validity as illustrated in Figure 1.

Table 2

The results of confirmatory factor analysis of MBS.

Variable	X2– statistic (df)	p-value	Goodness of fit indices
			ChiSq/df	RMSEA	GFI	CFI	NFI	TLI
One-factor model*	270.069 (86)	<0.001	3.140	0.116	0.822	0.926	0.896	0.909
4-factor model*	198.295 (82)	<0.001	2.418	0.095	0.867	0.953	0.923	0.940

*The original construct of MBS was supported for a model fit.

Figure 1

Standardized Factor Loading of MBS domains based on the final model.

The reliability analysis confirmed that the final model demonstrated a high level of internal consistency as Cronbach's alpha value was more than 0.7 (Table 3). The composite reliability values of MBS constructs ranged from 0.62 to 0.96 (Table 3), signifying convergent validity. All standardized factor loadings were more than 0.5, suggesting adequate level of convergent validity. In addition, most AVE values for each construct were more than the SV values (except for MRS less than E, and E less than CS), indicating a good level of discriminant validity (Table 4).

Table 3

Reliability analysis of the 15 item MBS based on the final model.

No	Item	SFL	bDomain	aCronbach's Alpha	cAVE	dCR
1	My mentor gives me useful advice when I tell him my needs, my worries, and my difficulties.	0.85	Mentoring relationship structure	0.966	0.773	0.965
2	My mentor likes to give me constructive advice on what I initiate.	0.88
3	My mentor gives me information to help me in my actions and in solving my problems.	0.91
4	My mentor and I are working towards mutually agreed upon goals.	0.89
5	We have established a good understanding of the type of changes that would be good for me.	0.88
6	My mentor and I agree about the things I will need to do to help improve my situation.	0.88
7	We agree on what is important for me to work on.	0.83
8	I believe that the way we are working on my situation is correct.	0.91
9	My mentor understands my needs, my worries, and my problems.	0.92	Engagement	0.906	0.828	0.906
10	My mentor listens attentively to the needs, worries, and achievements that I share with him.	0.90
11	When meeting, my mentor talks more than I do.	0.61	Autonomy support	0.620	0.453	0.621
12	Often, my mentor makes decisions for me.	0.73
13	My mentor values me even after I experience failures.	0.84	Competency support	0.884	0.717	0.884
14	My mentor often tells me what I do well.	0.86
15	My mentor congratulates me when I do something right.	0.84

Reliability analysis; Overall Cronbach's Alpha Coefficient = 0.932.

Domains were predetermined based on a previous study.

AVE (Average Variance Extracted) was calculated manually.

CR (Composite Reliability) was calculated manually.

Table 4

AVE and SV of MBS based on the final model.

Factors	AVE	SV by factor
		MRS	E	AS	CS
Mentoring relationship structure (MRS)	0.773	1	0.828	0.303	0.757
Engagement (E)	0.828	0.828	1	0.168	0.865
Autonomy support (AS)	0.453	0.303	0.168	1	0.230
Competency support (CS)	0.717	0.029	0.096	0.230	1

In summary, the results showed the four-factor MBS achieved an acceptable level of construct (i.e., convergent and discriminant) validity with a high level of internal consistency. The psychometric credentials and usability of this scale as well as future implication of MBS are discussed in the next section.

Discussion

Since its establishment as a tool to measure mentor behaviours, MBS's psychometric properties were investigated on samples of college students. These studies showed that MBS demonstrated good construct validity and high internal consistency for the mentoring relationship structure, engagement and competency support domains, and autonomy support achieved a satisfactory level. The present study, though performed in a different setting, revealed similar findings. This strengthens the psychometric credentials of MBS.

The CFA analysis showed that the goodness of fit indices were not achieved when using the one-factor model of MBS to measure mentor behaviours, suggesting MBS assessed multiple domains. The four-factor model of MBS has been shown to have an appropriate latent construct to measure the mentor behaviours as its goodness of fit indices were achieved – indicating acceptable construct validity. This study confirmed the findings of a previous study that reported high internal consistency for MBS in a college student setting. The reliability analysis showed MBS domains have high internal consistency as indicated by Cronbach's alpha values greater than 0.7, except for the autonomy support domain that recorded a value of 0.62. A similar pattern of internal consistency was reported by a previous study that showed autonomy support demonstrated the lowest level of consistency among the four domains. Moreover, in line with the findings of Brodeur, Larose, we found the correlation between MBS domains were independent as most of the correlation coefficient values were less than 0.95, suggesting good discriminant validity. Discriminant validity indicates there was less redundancy of items measuring similar attributes in a scale. This finding demonstrates MBS measured different attributes of mentor behaviour, thus supporting the multi-dimensionality of MBS. Apart from that, it is worth highlighting that participants used less than 5 min to complete the questionnaires, indicating a good response process. This fact supports the high feasibility of administering MBS within a short span of time. This will facilitate research and evaluation of mentorship. Hence, this study warrants more research to explore the validity of MBS in other educational settings to verify the present findings.

Despite that, we acknowledge several limitations of our study. First, this study was conducted at one medical school, so the findings might not generalize to other medical schools or institutions. For that reason, a multi-centre study is recommended to verify the present findings. Second, this study was conducted at the end of medical training and might not completely reflect the respondents' immediate judgements about mentor behaviours. The mentor behaviours could refer to their experiences with their mentors in a remote context. Therefore, future research should measure mentor behaviours immediately and longitudinally across mentoring sessions to verify the psychometric credentials of MBS domains. Third, these data were merely evaluating the construct validity in terms of its internal structure covering convergent and discriminant validity of MBS domains. It is therefore an incomplete representation of validity as a whole. Further study would benefit from additional validation studies to look for other sources of validity evidence such as relations to other variables and consequences of MBS scores on certain important educational outcomes such as academic performance and psychological health. Finally, the sample size was relatively small for CFA so some fit/error measures may be somewhat poor and it cannot be determined whether that is because of lack of fit or because of sample size issues. Hence future research should involve a bigger sample to verify the present results. Despite the aforementioned limitations, to the authors' knowledge, this study is the first study reporting the validity of MBS in Malaysia, and thus provides additional evidence to support the psychometric credentials of MBS in a different educational setting.

Conclusion

MBS demonstrates a satisfactory level of construct validity and a high level of internal consistency, meaning it can be used to measure supportive mentor behaviours in a medical school setting. This result suggests that MBS could be used as a mentorship evaluation tool for feedback in the context of Malaysian medical school.

Authors' contribution

JAMM and MSBY conceived and designed the study, conducted research, provided research materials, collected and organized the data, analysed and interpreted data, wrote initial draft and finalised article, critically reviewed the final draft and are responsible for the content and similarity of the manuscript.

Conflict of interest

The authors have no conflict of interest to declare.