The effect of personality traits on undergraduate dental students' performance in multiple mini interviews.

BACKGROUND: Multiple mini interviews (MMI) is a structured student selection method where applicants rotate around a series of stations that do not require previously learned knowledge. The Big Five Inventory (BFI) is a tool often used to measure personality traits commonly used in applicant selection.
OBJECTIVE: The aim of the current study was to assess the effect of personality traits using the BFI on undergraduate dental students' performance in MMI.
METHODS: This research was conducted at The Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia. All undergraduates applying to the dental program at King Abdulaziz University, Faculty of Dentistry (KAUFD) in two academic years [(2014-2015) and (2015-2016)] undertook the MMI held at KAUFD and completed the BFI inventory with a total number of 350 students. The MMI consisted of five stations while the BFI contained forty-four items. Data were analysed by SPSS version 20, using independent-samples t-test, Mann-Whitney U test, Spearman's rho correlation coefficient and Chi-square test.
RESULTS: female students scored higher than their male counterparts but scores were only significant in the empathy station (p<0.001) in the first year, and in both empathetic skills and team work abilities (p<0.05) in the second. BFI results indicate that male students scored higher on agreeableness (p<0.003) and lower on neuroticism (p<0.001) in the first year while female students scored higher on agreeableness and conscientiousness (p<0.001) in the second year. Students of private schools had higher total MMI scores compared to those of public schools (p<0.05).
CONCLUSION: MMI combined with the Big Five Inventory can be a useful tool in the admission process.

1. Introduction

Traditionally and as a worldwide established routine, the acceptance of students into medical courses is based on their academic achievement and grade point average (GPA), yet a recently published report on the challenges involved in the selection of medical students suggested that the focus on the personal non-academic qualities of applicants might be a better approach in the selection of students who will make the best doctors. Furthermore, the report revealed that since 1944, a UK government – appointed committee stated that some medical students who are able to pass required examinations lack the necessary aptitude and character for a medical career, adding that the then existing methods for selecting medical students do not meet the problems and changing condition (1). However, little change or progress in the process of admitting medical students occurred over the past 70 years until the introduction of the multiple mini interview system (MMI) which was first developed by the School of Medicine in McMaster University Canada. This was a replacement to the traditional board or panel interview which did not accurately predict future performance in medical school. Additionally, it was found that complaints by patients treated in the medical and dental programs were mostly related to non-cognitive skills including professionalism and ethics (2, 3). Since then, many schools have adopted this form of structured interviews to assess applicants’ non-cognitive skills including ethical and moral values, communication skills, empathy, as well as the awareness of the demands of training as a dental surgeon (4). During the MMI, the applicants are assessed for numerous non-cognitive skills while allowing the applicant to recover if there were problems with previous stations (5). Previous coaching had no effect on MMI scores making it fair, and it was also reported that MMI rated higher than the traditional interview on fairness and effectiveness as an assessment tool by both applicants and evaluators (6, 7). Additionally, MMI are free from gender and other forms of cultural bias including socio-economic disadvantage (8, 9). The systematic scoring of MMI is a significant advantage compared to the traditional panel interviews where contrast effects were found to influence assessors’ scoring, for example, in situations when the assessed candidate is preceded by a relatively weak applicant (10). Other advantages include feasibility, since the MMI does not require more examiners than the panel interview nor does it cost more, additionally the interviews can be completed over a short period of time (8, 11). It was reported that the MMI performance at admission was the best predictor for future OSCE as well as clerkship performance in medical and dental programs (3, 12). MMI performance at admission was reported to be statistically significantly predictive of performance at future examinations (13), though others reported non-significant correlations with future medical examination scores (14). Some reported that better MMI performance at entry to medical school was predictive of higher scores (15) unlike panel interviews, which are unable to predict future academic and/or clinical performance in health care disciplines (16) and may even dissuade some potential applicants (17). The use of personality measures, separately or combined with MMI, is a similarly recognized admission procedure that was reported to be a promising predictor of academic outcome and a useful admission tool. BFI is a forty-four-item inventory that measures a person’s major dimensions of personality namely; extraversion, agreeableness, conscientiousness, neuroticism, and openness for experiences. Higher score on extraversion, indicates a sociable, energetic, enthusiastic and outgoing personality. Higher scores on agreeableness, indicate a sympathetic, forgiving, warm, compliant, and modest character. A higher score on conscientiousness, is indicative of a competent, organized, self-disciplined, and dutiful person. A higher score on neuroticism signals a tense or irritable, shy subject who is neither self-confident nor contented. While, a higher score on openness to experience points to an imaginative, artistic, curious, excitable and unconventional student with wide interests (18). The BFI models are used to investigate the relationship between personality traits and performance in the workplace as well as in predicting academic performance (19, 20). It has been suggested that the use of structured interview including the BFI, enables the selection of medical students who possess more desirable personality characteristics (21). Furthermore, several of the personality traits measured using BFI were reported to be related to college students’ GPA (22) and more specifically, to learning styles applied by college students (23). The aim of this study was to assess the effect of personality traits on undergraduate dental students’ performance in multiple mini interviews as well as investigate any correlations between the total MMI score and various non-cognitive skills while considering the effects of other demographic variables including gender and schooling.

2. Material and Methods

Acceptance to the undergraduate dental program at KAUFD is a two-step process that includes first graduating high school and passing a national standardized exam known as the general aptitude and capacity exam. All students enrolled for health-related sciences schools at King Abdulaziz University (KAU) must then complete the foundation year of physics, chemistry, organic chemistry, biology, mathematics and English. Performance of this year is assessed by GPA, calculated using grades and credit point value of all courses. Upon successfully completing the foundation year, students are allocated to the dental undergraduate program at KAUFD based on weighted percentage which is the average of the high school grades, GPA of the foundation year at KAU and the general aptitude and capacity exam. In the academic years of (2014–2015) and (2015–2016) all students allocated to KAUFD undertook the multiple mini interviews held at KAUFD. The total number of students in the year (2014–2015) was 190 (91 male students and 99 females), while the total number of students in the year (2015–2016) was 160 (75 male students and 85 females). The MMI comprised of 5 stations to evaluate the following aspects: 1) Personal characters, 2) Ethical decision-making, 3) Critical thinking ability, 4) Teamwork ability, and 5) Empathetic skills. These stations were selected based on reviewing the MMI literature as well as the previous MMI interviews at KAUFD (4, 12). The structure of each station included one or more scenarios evaluating the MMI station theme. Stations set up including scenarios, rubrics and timing were constructed and reviewed by an interview committee. Mock interviews were carried out to evaluate the readability of questions, comprehensibility of scenarios, and feasibility of grading rubrics. All faculty raters were trained and calibrated prior to the interviews. The allocated total time for each station was 7 minutes; 2 minutes for reading the scenarios and 5 for discussion. Two faculty members were assigned to each station and due to the high number of students, interviews were delivered over two days where 4 MMI panels were conducted simultaneously with a total of 40 faculty members participating in interviewing the students. BFI measured extraversion, agreeableness, conscientiousness, neuroticism, and openness for experiences. Each dimension was measured using a number of items rated by students on a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly agree) with some items being reversed. The statistical analysis was performed using the IBM© SPSS© Statistics version 20 (IBM© Corp., Armonk, NY, USA). The qualitative data was presented in the form of numbers and percentages, while the mean and standard deviation, or the median (with its 95% confidence interval) were used to describe quantitative data. Inter-rater reliability was measured using the intra-class correlation, and Cronbach’s alpha was used to test the internal consistency reliability of the BFI scale. Comparison of groups was performed using independent-samples t-test if data were normally distributed, or the Mann-Whitney U test if data were not normally distributed. Spearman’s rho correlation coefficient was used to explore relations among numeric variables. Chi-square test was used for testing association among categorical variables and the P values were considered significant at P<0.05.

3. Results

This study included a total of 350 incoming dental students (190 in the year 2014–2015 and 160 in the year 2015–2016). Table 1 displays the demographics as well as the achievements of these students and shows that the majority of students were from the city of Jeddah, with only a few from other cities and geographic locations in the Kingdom of Saudi Arabia. A higher percentage of students were graduates of governmental schools compared to private schools. Table 1 also shows that female students had significantly higher foundation year GPA and weighted percentage compared to their male classmates in both years (p<0.001). The intra-class correlations (ICC) between raters was found to be statistically significant in the year (2014–2015) with the following values: 0.860 for Personal, 0.814 for Ethical decision making, 0.585 for Critical thinking, 0.850 for Team work abilities and 0.761 for Empathetic skills. The corresponding values for the year (2015–2016) were 0.832, 0.867, 0.729, 0.851, and 0.917 respectively (all with p<0.001). Internal consistency reliability of BFI ranges from 0.579 to 0.765 in (2014–2015) and from 0.463 to 0.723 in (2015–2016). Table 2 summarizes the average MMI scores of all stations along with their average BFI scores for both academic years. The table shows that in (2014–2015), the average MMI scores at the different stations were relatively comparable with the highest score being in the ethical decision making station and the lowest in the personal and team work stations. A comparison of male and female students showed a general tendency for higher scores for females but the difference was only significant in their empathetic skills (p<0.001). BFI results indicate that male students scored higher on agreeableness (p<0.003) and lower on neuroticism (p<0.001). By comparison, students of the year (2015–2016) also had relatively comparable average MMI scores at the different stations, and upon comparison of male and female students, a general tendency for higher scores was noted for females but the difference was only significant in their empathetic skills and team work abilities (p<0.05). BFI results indicate that female students scored higher on agreeableness and conscientiousness with a (p<0.001) for both. A correlation matrix of the BFI and MMI scores is presented in Table 3. The matrix shows that students’ total MMI score is generally moderately and positively correlated with their individual scores in each personality trait. The total MMI scores for the year (2014–2015) showed weak positive correlations between students’ scores on extraversion and team work as well as extraversion and personal stations (p<0.05). No other correlations were found with any of the personality traits even when viewed by gender. In contrast, students in the year (2015–2016) displayed more correlations of the BFI and MMI recorded as seen in Table 3. The matrix shows that the total MMI scores were positively correlated with students’ scores on extraversion and all stations except ethical decision making (p<0.05). Furthermore, agreeableness was correlated with ethical decision making and team work abilities as well as empathetic skills, while conscientiousness was related to the personal station, teamwork abilities and empathetic skills whereas openness was found to have positive correlation to team work abilities and empathetic skills. Negative significant correlations were seen between neuroticism and the personal station and the critical thinking. Interpreting these results based on gender, reveals that male students in the year (2014–2015) had the highest correlation between extraversion and 3 of the MMI stations (personal, teamwork abilities and empathetic skills), but none of the personality traits had such influence in female students, while the female students of (2015–2016) had the highest correlation between extraversion and all MMI stations. Our findings show that in general, students of private schools had higher total MMI scores compared to those of governmental schools (p<0.05). Male students had higher scores in the personal and teamwork stations (p<0.05). However, these associations were not found for personality traits as students of governmental and private schools appeared to be comparable. It can be seen that for male students, school type had a significant influence on their MMI performance as male students coming from private schools did better in the personal and empathetic skills stations. On the other hand, female students coming from private schools had higher total MMI scores and higher scores in teamwork abilities. In regard to the associations with personality traits, the results show that female students coming from governmental schools were more likely to score higher in agreeableness than those from private schools. Extraversion in male students had a strong influence on their MMI scores where higher scores on the extraversion scale were coupled with higher total MMI scores, while none of the personality traits had such influence on the female students’ scores. Similarly, other demographic factors including geographic area or students’ residence did not show any association with their performance in MMI or their personality traits.

Table 1

Demographic characteristics and achievement of students

Characteristics		2014–2015			p-value	2015–2016			p-value
		Males (n=91)	Females (n=99)	Total (n=190)		Males (n=75)	Females (n=85)	Total (n=160)
All sample		47.9	52.1	100.0		46.9	53.1	100.0
City	Jeddah	90.1	99.0	94.7	0.006	96.0	89.4	92.5	0.114
	Others	9.9	1.0	5.3		4.0	10.6	7.5
Secondary school	Governmental	59.3	69.7	64.7	0.136	70.7	65.9	68.1	0.517
	Private	40.7	30.3	35.3		29.3	34.1	31.9
Secondary school grades		Mean (SD)	Mean (SD)	Mean (SD)	p-value	Mean (SD)	Mean (SD)	Mean (SD)	p-value
		96.71 (2.88)	97.02 (2.94)	96.87 (2.91)	0.458	95.36 (3.94)	98.63 (1.49)	97.10 (3.33)	<0.001
Foundation year	GPA	4.62 (0.25)	4.81 (0.23)	4.72 (0.26)	<0.001	4.79 (0.09)	4.94 (0.05)	4.87 (0.10)	<0.001
	Weighted %	90.86 (4.68)	94.62 (4.68)	92.82 (5.03)	<0.001	93.15 (1.90)	97.15 (1.74)	95.28 (2.70)	<0.001

Table 2

Descriptive statistics of MMI rating at different stations and personality traits

Year	Scale	Subscales	p-value	Cronbach’s Alpha
2014–2015	MMI	Personal	0.205
		Ethical decision making	0.109
		Critical thinking	0.573
		Team work abilities	0.803
		Empathetic skills	<0.001
		Total score	0.339
	BFI	Extraversion	0.523	0.579
		Agreeableness	0.003	0.611
		Conscientiousness	0.350	0.730
		Neuroticism	0.001	0.765
		Openness	0.978	0.622
2015–2016	MMI	Personal	0.183
		Ethical decision making	0.577
		Critical thinking	0.671
		Team work abilities	0.003
		Empathetic skills	0.010
		Total score	0.171
	BFI	Extraversion	0.173	0.463
		Agreeableness	<0.001	0.634
		Conscientiousness	<0.001	0.651
		Neuroticism	0.626	0.723
		Openness	0.216	0.415

Table 3

Correlation matrix of personality traits and MMI rating at different stations by academic year

Year	BFI	MMI
		Personal	Ethical decision making	Critical thinking	Team work abilities	Empathetic skills	Total MMI score
2014/2015	Extraversion	0.162 (0.026)	0.077 (0.293)	0.022 (0.764)	0.218 (0.002	0.115 (0.114)	0.200 (0.006)
	Agreeableness	−.0020 (0.787)	0.086 (0.240)	0.054 (0.458)	0.055 (0.453)	−0.059 (0.415)	0.012 (0.864)
	Conscientiousness	0.027 (0.712)	−0.056 (0.440)	0.045 (0.540)	0.020 (0.781)	−0.097 (0.185)	−0.030 (0.676)
	Neuroticism	−0.020 (0.782)	0.055 (0.452)	−0.021 (0.770)	−0.082 (0.263)	0.099 (0.176)	0.016 (0.829)
	Openness	0.112 (0.123)	0.036 (0.621)	0.112 (0.125)	0.015 (0.841)	0.137 (0.060)	0.108 (0.139)
2015/2016	Extraversion	0.258 (0.001)	0.133 (0.093)	0.174 (0.027)	0.218 (0.006)	0.210 (0.008)	0.295 (<0.001)
	Agreeableness	0.114 (0.151)	0.209 (0.008)	0.121 (0.129)	0.218 (0.006)	0.212 (0.007)	0.266 (0.001)
	Conscientiousness	0.203 (0.010)	0.131 (0.100)	0.070 (0.382)	0.197 (0.013)	0.342 (<0.001)	0.269 (0.001)
	Neuroticism	−0.164 (0.038)	−0.011 (0.890)	−0.160 (0.043)	−0.099 (0.211)	−0.131 (0.099)	−0.143 (0.071)
	Openness	0.134 (0.091)	0.014 (0.857)	0.072 (0.368)	0.159 (0.045)	0.178 (0.024)	0.155 (0.050)

4. Discussion

Since many medical and health science institutes consider MMI as a highly valid assessment tool as the MMI, scores are not solely correlated with traditional admission tools scores such as the personal interview or undergraduate grades (24, 25). The employment of MMI setup, as part of the acceptance process in medical and other health sciences programs, is now acceptable and widely practiced due to the numerous reports correlating MMI scores and applicants’ cognitive traits where MMI score were found to be positively associated with ethical decision making (26), reasoning skills and written communication (27, 28), but not with emotional intelligence (29). MMIs was reported to be the most consistent predictor of medical school assessments (30). The current work is a report of the MMI at KAUFD for two consecutive academic years (2014–2015 and 2015–2016), designed with 5 stations each lasting a total of 7 minutes employing 2 evaluators in each station, which is consistent with other studies as reported in a systematic review of 30 MMI studies, 20 of which were conducted on medical school applicants alone; ten of these were in Canada, five in Australia, three in the UK and two in the USA. The remaining 10 studies were on other health related sciences including dentistry, veterinary and pharmacy. The number of stations used in these studies ranged from 4 to 12. Fourteen of the studies relied on a single assessor in each station while 4 studies used 2 assessors. The remaining studies did not report the number of assessors per station. The range of time at each station was 5 to 15 minutes with a mode of 8 minutes (31). While assessors found 6 minutes per station to be satisfactory and eight minutes longer than necessary, the applicants preferred the longer time (32). This report displayed significant consistency with intra-class correlations (ICC) between evaluators from different stations and within the same station which is one of the foremost advantages of MMI setups compared to traditional panel interviews. These findings are consistent with the first year MMI findings (4) and are most likely a result of the training workshop which aimed to calibrate the high number of faculty involved in MMI and increase the reliability of the MMI findings. It also insures that no bias was practiced during the MMI stations. Similar high intra-station reliability was reported in the past as well as internal consistency of scores assigned within any one station by multiple interviewers. The reported reliability ranges from moderate (27) to acceptable (26) to high (33). The current report utilized the faculty as interviewers (assistant, associate and full professors within the faculty of dentistry at KAU only) since community assessors were reportedly less consistent than the faculty’s (2). Some MMI studies reported the use of actors while others used a combination of faculty and community practitioners (34), whereas others include students (8). The current report found that the average MMI scores at different stations were relatively comparable with the highest score in the ethical decision making and empathetic skills stations for the year (2014–2015) and similarly for the year (2015–2016), with added high scores in team work abilities and personal stations compared to the lowest score at the personal and team work stations in female students of the year (2014–2015). Although no significant difference was found in the total MMI scores a general tendency for higher scores for females was noted and was significant in the empathetic skills station for the first year and in both the empathetic skills and team work abilities on the following year. This is in agreement with the results of the first MMI held at KAUFD in the academic year (2013–2014) where female students were found to score higher than their male counterparts in all MMI stations (4). Unlike the previous year however, this study combined MMI station with the big five inventory (BFI) in order to obtain a more detailed personality profile of the applicants. The benefits of combining the use of the BFI with MMI has been reported due to the ability of BFI to identify applicants with high levels of conscientiousness and agreeableness as well as those with neuroticism, extraversion, and openness. This study recorded significantly higher scores for empathetic skills in the female applicants, compared to higher scores of agreeableness and lower on neuroticism in their male counterparts. A correlation matrix of the BFI and MMI scores shown in Table 3 indicates that students’ average total MMI score is moderately and positively correlated with their individual scores in each station. The total MMI scores showed mild positive correlation with students’ scores on the extraversion, team work and personal stations. Correlation was also observed between various other traits and upon breaking down these correlations by gender, it was found that extraversion in male students had a strong influence on their MMI scores where higher scores on the extraversion scale were coupled with higher total MMI scores (and higher scores in the personal, teamwork, and empathetic skills stations). This is in total contrast to the female students’ data where none of the personality traits had such influence. The current study also focused on the association between some of the demographic factors including schooling type with MMI and BFI scores, and found that in general, private school graduates had higher total MMI scores compared to those of public schools, as well as higher scores in the personal and empathetic skills in males and most stations in females. This may be a result of the extracurricular assignments and projects in private schools which encourage teamwork and help build character. Such associations, however, were not found for personality traits as students of public and private schools appeared to be comparable. Other demographic factors including geographic area or students’ residence did not show any association with their performance in MMI or their personality traits. When examining the correlation between students’ personality traits and MMI performance with their academic performance in high school, foundation year and the weighted percentage, it was found that teamwork abilities were mildly negatively associated with foundation year grades and weighted percentage in female students, implying that students with better teamwork abilities showed lower foundation year grades and weighted percentage. Similarly, when male students scored higher in agreeableness, a negative influence was noted on their weighted percent. The relation is unclear, but one could speculate that it is common practice to enlist the help of volunteers in many social, charitable and academic events taking place at the university, thus students with higher sense of teamwork might be involved with such activities which may, in turn, result in a lower academic performance, however, this cannot be verified without further investigation. Students’ academic performance was positively associated with certain personality traits in female students including extraversion, conscientiousness and openness. In a report on the relationship between applicants’ personality, MMI scores and medical school acceptance offers, a study on 444 medical school applicants during the year (2010–2011) reported that extraversion was related to MMI performance, though both extraversion and agreeableness were associated with acceptance offers (35). The authors did highlight that relying on MMI in such a strict manner may result in the admission of relatively homogeneous medical school classes thus reducing the diversity of students and limiting the students’ chances of experiencing personal and professional growth. Other reports also stated a significant correlation between MMI performance and the applicant’s personality assessed using the BFI. The MMI was found to be related to extraversion and conscientiousness as well as agreeableness (20). To validate the use of MMI and BFI as part of the acceptance into KAUFD it would be wise to follow up the interviewees’ performance including their graduation GPA, enrolment into post-graduate programs and their offers for employment. Compared to the first MMI held in the academic year (2013–2014), scenario based stations were designed with less direct questions to eliminate the influence of training for interviews.

5. Conclusions

The current report of the MMI scores for the admission classes in (2014 –2015) and (2015–2016) at KAUFD shows that MMI combined with the Big Five Inventory can be a vital aid in the admission process and in predicting suitability for a dentistry as well as discerning students with leadership and teamwork qualities, compared to those in need of additional resources. In the current report, male applicants scored higher on agreeableness and lower on neuroticism in the first year compared to the second year where female applicants scored higher on agreeableness and conscientiousness. Students of private schools had higher total MMI scores compared to those of governmental schools. The BFI and MMI correlation indicated that the total MMI score is moderately and positively correlated with the individual scores in each station and the total MMI scores showed mild positive correlation with students’ scores on the extraversion, team work and personal stations.