Performance of Black and Indigenous applicants in a medical school admissions process.

BACKGROUND: Diversity in medical schools has lagged behind Canada's growing multicultural population. Dalhousie medical school allows Black and Indigenous applicants to self-identify. We examined how these applicants performed and progressed through the admissions process compared to Other group (applicants who did not self-identify).
METHODS: Retrospective analysis of four application cycles (2015-2019) was conducted, comparing demographic data, scores for application components (Computer-Based Assessment for Sampling Personal Characteristics (CASPer), MCAT, GPA, supplemental, discretionary, Multiple Mini Interview (MMI)), and final application status between the three groups.
RESULTS: Of 1322 applicants, 104 identified as Black, 64 Indigenous, and 1154 Other. GPA was higher in the Other compared to the Indigenous group (p < 0.001). CASPer score was higher in the Other compared to the Black group (p = 0.047). There was no difference between groups for all other application components. A large proportion of Black and Indigenous applicants had incomplete applications. Acceptance rates were similar between all groups. Black applicants declined an admission offer substantially more than expected (31%; p < 0.001).
CONCLUSIONS: Black and Indigenous applicants who completed their application progressed well through the admissions process. The pool of diverse applicants needs to be increased and support provided for completion of applications. Further study is warranted to understand why qualified applicants decline acceptance.

Introduction

With increasing diversity and multiculturalism in the Canadian population, it is important to have a workforce of healthcare providers that reflects this diversity. Patients often prefer a healthcare provider from their personal ethnic or cultural background[1] and feel more involved in healthcare decisions.[2] Diversity is also beneficial in working towards broader access to healthcare, in that healthcare providers are more likely to practice in the region of their community of origin.[3] Furthermore, they are likely more apt to understand their patients’ health within the cultural and social context.

Underrepresented populations in medicine are defined as the “racial and ethnic populations that are underrepresented in the medical profession relative to their number in the general population.”[4] The Association of American Medical Colleges (AAMC) has recommended taking a local perspective on underrepresentation. Medical schools are encouraged to collect data on the ethnic and racial self-descriptions in their applicants and students. The Canadian Medical Association (CMA) and Canadian Federation of Medical Students (CFMS) have also formally recognized the need to improve diversity in Canada’s medical profession and provided recommendations.[5] Additionally, one of the “Calls to Action” from the Truth and Reconciliation Commission[6] is to increase the number of Indigenous healthcare providers and improve retention of those working in indigenous communities.

Admissions is a high-stakes and gate-keeping process for Canadian medical schools, given the very low attrition and high graduation rate, i.e. >98%.[7] It is known that Canadian medical schools have a high proportion of students from high income families and non-minority ethnic groups who identify as heterosexual, cis-gender, and non-disabled.[8] The cohorts are also under-representative of the Indigenous, Black, and Filipino ethnicities in Canada.[8]

Multiple investigators have studied the impact of different recruitment and admissions strategies on the diversity of matriculating medical students. These have included adding an “adjustment” to pre-interview scores for URIM applicants,[9] increasing the weighting on non-academic criteria[10] and the use of the MMI. The MMI interview style has gained support as studies have demonstrated that performance does not seem to be impacted by gender, family income level, size of community of origin,[11] or ethnicity.[12],[13]

Diversity is a key area of interest to Dalhousie medical school. According to the Canadian census data from 2016, 5.4% of the population in Nova Scotia identifies as First Nations and 2.9% as Metis, both being indigenous people.[14] Nova Scotia has the fifth largest Black population in Canada, with over 20,000 individuals (2.4% of the population). Dalhousie medical school has recognized these populations as being underrepresented in the medical profession in the Maritime Provinces of Canada. This prompted affirmative action to try to increase recruitment, enrolment, and retention. Prospective students can self-identify as Black or Indigenous, and their application is considered based on qualifications, rather than comparison and ranking relative to other applicants. However, the number of matriculants from these underrepresented in medicine (URIM) groups remains small.

A program review was conducted to study the characteristics of applicants from these URIM groups, performance in the application components, and overall success in progressing to interviews and acceptance offers. We hoped to identify any components of the process that may act as a gatekeeper or disadvantage URIM applicants. Much of the research on medical school admissions and diversity has been performed in the United States, where ethnicities differ somewhat from the Canadian population. Furthermore, each medical school sets its own application process and admission selection, limiting the generalizability of results. Therefore, we felt it would be important and interesting to examine local trends of two URIM populations in Atlantic Canada (i.e. Black and Indigenous).

Methods

Study design and setting

A retrospective analysis was conducted of the admissions application data routinely collected by the Dalhousie University registrar and medical school (Halifax, Nova Scotia, Canada) from 2015-2019, encompassing four application cycles. Applicants can choose to self-identify as Black or Indigenous. The application components and process are the same as for all other applicants. Interviews are offered to all self-identified Black or Indigenous applicants, as well as to all Maritime applicants, who meet minimum thresholds for CASPer (for 2017-2019 application cycles, threshold was 1.5 standard deviations below the applicant pool mean), GPA (3.3 Maritime, 3.7 Non-Maritime), and MCAT (approximately 50th percentile for each category). Due to the volume of applications for limited positions, thresholds are higher for non-Maritime applicants. For non-Maritime non-self-identified applicants, scoring above thresholds does not guarantee an interview invitation due to limited interview capacity. Non-academic qualifications are assessed by CASPer, MMI, personal statement, and supplemental information form. Black or Indigenous applicants may be offered admission to the medical class if they meet the academic and non-academic qualifications.

The data were analyzed in an encrypted electronic spreadsheet with all personal identifying information removed. The study proposal was submitted to Dalhousie University Research Ethics Board and was deemed a Program Evaluation not requiring formal assessment and ethics approval.

Sampling methods and sample size

Individuals who submitted an application during the 2015-2019 admission cycles (4 academic years, 2015-16, 2016-17, 2017-18, 2018-19) were included. The Black and Indigenous groups were comprised of applicants who initiated, submitted, or cancelled an application. The Other group (individuals who did not self-identify as Black or Indigenous) was limited to applicants who submitted an application and completed the interview. Incomplete and cancelled applications from this group are not routinely collected by the Admissions Office. In total, 1322 applicants were included in the analysis; 104 had identified as Black, 64 as Indigenous, and 1154 were Other. Of note, the CASPer examination was incorporated into the Dalhousie admissions process from 2017 onwards (n = 21 Black, 18 Indigenous, and 576 Other), and thus only two admissions cycles (2017-18 and 2018-19) are included in CASPer data.

Outcome measures

Data included demographic characteristics, scores from each component of the admissions process (CASPer exam, MCAT, GPA, personal statement, supplemental section, discretionary, and MMI), and final application status. Information on sex and age was only available for applicants who completed an interview. Geographic information was available for some but not all of the Black and Indigenous applicants, as this was optional to list in the application.

CASPer is an online situational judgement test used to assess non-academic attributes. Each question is scored by an independent rater, and a total score is compiled.

The MCAT is a standardized, computer-based, multiple-choice examination. A new version of the examination was launched in 2015. In order to compare across examinations with different reporting systems for raw scores, each applicant’s MCAT score was converted to a z-score.

The Personal Statement is a narrative from the applicants reflecting their past experiences, adversities, personal skills, and motivation to become a physician. Applicants receive a score based on the file reviews by Admissions Committee members.

The Supplemental Score reflects information about extracurricular, volunteer, and employment activities, as well as awards, research, and other achievements. Applicants receive a score based on the file reviews by Admissions Committee members. The Personal Statement and supplemental information are anonymized for file reviewers, and applicants are advised not to identify themselves in these sections. Photographs are not included in this part of the application.

The Discretionary Score is based on an overall assessment of the personal statement and supplemental information. Points are applied at the discretion of file reviewers, with attention to outstanding achievements and/or suitability for medical school.

The interview consists of a 10-station MMI. Interviewers are trained to use a common scoring rubric to assess communication skills, content, and a global impression of performance. Stations are assessed separately by independent interviewers who are blinded to the candidate’s application (academic scores, personal statement, supplemental). The final MMI score is the sum of scores on individual stations.

Data analysis

Quantitative analyses were conducted using SPSS Statistics software for Mac (version 25.0). Kruskal-Wallis tests were performed for continuous outcomes and Chi-square tests for categorical outcomes. If the initial test statistic showed a significant difference between groups, post-hoc analyses were then performed with a Bonferroni correction to adjust for multiple comparisons. As such, a p value < 0.005 was used to indicate statistical significance for post-hoc analyses. To protect confidentiality, groups of smaller that five applicants are reported as “<5.”

Results

The groups were similar in terms of age (p = 0.28) and sex (p = 0.69). Mean age was 23.8 years for Black applicants, 23 for Indigenous, and 24 for Other. The proportion of female applicants in each group was 50% (Black), 61% (Indigenous), and 52% (Other). The majority of Black (92%, 43/47) and Indigenous (77%, 11/47) applicants listed a community of origin with a population size greater than 10,000 people. Nova Scotia was listed as the home province for 64% (35/55) of Indigenous and 48% (552/1154) of Other applicants. Amongst Black applicants, Nova Scotia (33%, 34/103) and Ontario (36%, 37/103) were the most common home provinces (See Table 1).

Table 1

Demographic characteristics of the applicants.

	Black	Indigenous	Other
Age in years, mean (range)	23.8 (21-33)	23.0 (21-27)	24.0 (20-52)
Sex, Females, N (%)	16 (50%)	14 (60.9%)	602 (52.2%)
Community of origin, N (%)
<10 000	4 (8.5%)	11 (23.4%)	-
>10 000	43 (91.5%)	36 (76.6%)	-
Home province , n (%)
Nova Scotia	34 (33%)	35 (63.6%)	552 (47.8%)
New Brunswick	<5	7 (12.7%)	217 (18.8%)
Prince Edward Island	<5	<5	62 (5.4%)
Newfoundland	<5	<5	9 (0.8%)
Quebec	5 (4.9%)	<5	10 (0.9%)
Ontario	37 (35.9%)	<5	243 (21.1%)
Manitoba	<5	<5	<5
Saskatchewan	<5	<5	<5
Alberta	6 (5.8%)	<5	29 (2.5%)
British Columbia	<5	<5	25 (2.2%)
Non-Canadian	12 (11.7%)	<5	<5
Non-Maritime Provinces	65 (63.1%)	12 (21.8%)	323 (28%)

For age and sex, n = 32 (Black), 23 (Indigenous), 1152 (Other). For community of origin, n = 47 (Black) and 47 (Indigenous). For home province, n = 103 (Black), 55 (Indigenous), 1154 (Other).

There was a significant difference between the groups in terms of GPA (p <0.001); see Table 2. Post-hoc analyses identified higher GPA in the Other group (mean=3.85 ±0.17) compared to the Indigenous group (mean=3.76 ±0.18) (p <0.001). There was also a significant difference between the groups in terms of CASPer score (p < 0.01). CASPer scores were higher in the Other group (mean z-score = 0.35 ±0.83) compared to the Black group (mean z-score= -0.22 ±1.01) (p < 0.05). There was no statistically significant difference between the groups in terms of MCAT, supplemental, essay, discretionary, MMI, or total application score.

Table 2

Mean scores of applicants in each component of the application process.

	Black (N=104) †	Indigenous (N = 64) †	Other (N = 1154) †	p value
MCAT, z-score	-4.47 (-7.31 to -1.63)	-1.78 (-5.63 to 2.07)	0.20 (-0.38 to 0.78)	0.05
GPA	3.8 (3.76-3.84)	3.76 (3.71-3.81)*	3.85 (3.84-3.86)*	<0.001
CASPer, z-score	-0.22 (-0.68 to 0.24)*	-0.14 (-0.57 to 0.30)	0.35 (0.28- 0.41)*	0.047
Supplemental	12.16 (11.49-12.82)	11.28 (9.94-12.62)	11.58 (11.43-11.73)	0.47
Essay	6.43 (5.99-6.87)	6.26 (5.77-6.75)	6.33 (6.26-6.40)	0.68
Discretionary	2.58 (2.34-2.82)	2.44 (1.99-2.89)	2.42 (2.37-2.46)	0.44
MMI, Mean	28.35 (26.90-29.80)	26.45 (24.88-28.02)	27.60 (27.37-27.83)	0.21
Total score	70.52 (68.31-72.72)	67.00 (64.01-69.98)	68.95 (68.56-69.34)	0.15

Statistically significant results are denoted with an asterisk (*). Data is presented as mean (95% confidence interval). MCAT – Medical College Admission Test. GPA – grade point average. CASPer – Computer-Based Assessment for Sampling Personal Characteristics. MMI – multiple mini interviews. † For CASPer, n = 21 (Black), 18 (Indigenous), 576 (Other).

A large proportion of applicants in the URIM groups started but did not complete their application (Table 3). MCAT was the most common component to render an applicant ineligible for interview invite. Overall, 25% (26/104) of Black and 23% (15/64) of Indigenous applicants who started an application received an acceptance offer.

Table 3

Application status for all Black and Indigenous applicants to Dalhousie medical school from 2015-2019

	Black (n = 104)*	Indigenous (n = 64)*
Incomplete application	53 (51%)	27 (42.2%)
Ineligible components	19 (18.3%)	14 (21.9%)
Ineligible CASPer	<5	<5
Ineligible MCAT	16 (15.4%)	13 (20.3%)
Ineligible GPA	<5	<5
Ineligible course load	<5	<5
Accepted	16 (15.4%)	13 (20.3%)
Declined	10 (9.6%)	2 (3.1%)
Rejected	6 (5.8%)	8 (12.5%)

MCAT – Medical College Admission Test. GPA – grade point average. CASPer – Computer-Based Assessment for Sampling Personal Characteristics. *For CASPer, n= 21 (Black), n= 18 (Indigenous).

Among those who were interviewed, significantly more Black applicants declined an offer of admission (31.1%, 10/32, p <0.001), and significantly fewer Black applicants were rejected (18.8%, 6/32, p <0.001) than expected by chance; see Figure 1. Significantly more applicants in the Other group were rejected (55.7%, 642/1153, p <0.001), and fewer Other applicants declined (9.3%, 107/1153, p=0.003) than expected by chance. The proportion of Indigenous applicants accepted (56.5%, 13/23), declined (8.7%, 2/23), and rejected (34.8%, 8/23) did not differ significantly from what was expected by chance (all p > 0.05).

Figure 1

Application status for Black, Indigenous and Other applicants who completed an interview.

N = 32 (Black), 23 (Indigenous), 1153 (Other).

Discussion

In this study of 1322 medical school applicants, mean GPA was statistically higher in the group of non-self-identified individuals compared to the Indigenous group, similar to findings from other researchers who have examined GPA and URIM applicants.[11],[12],[15] It seems that increasing the weighting of non-academic components over academic components (i.e. GPA, MCAT) of the application stage can increase the proportion of URIM students accepted.[10] The converse has also been demonstrated, in that URIM acceptance rates decreased as the weighting on GPA and MCAT increased.[12]However, these conclusions have not been universal. At a medical school in Denmark, diversity was unaffected by a selection process that was attribute-based (versus grade-based).[16] Reiter et al hypothesized that higher weighting on MMI scores would balance the academic scores that may otherwise limit the diversity in their selected applicants; however, they found that even extreme weightings of MMI versus GPA did not increase acceptance of diverse applicants.[11] Thus, efforts to increase URIM enrolment may need to be further “upstream” in the medical school admissions process, such as recruiting a greater number of diverse applicants.[16]

Although we found that mean GPA was lower in the Indigenous group, less than five Indigenous applicants were disqualified due to GPA scores below the minimum requirement. This suggests that the current GPA threshold at Dalhousie medical school is adequate in enabling applicants to progress through the admissions process. There was no significant difference between groups based on MCAT score, yet a substantial proportion of our self-identified applicants were disqualified because of their MCAT score (Table 3). It is plausible that some URIM applicants are unintentionally disadvantaged when too much emphasis is placed on purely academic criterion. In fact, Lucey & Saguil[17] and Goode & Landefeld[18] make convincing arguments for why these students may score lower. They discuss how structural racism, which starts early in life, leads to differences in education (i.e. under resourced schools, resulting in lower foundational skills) and opportunity (i.e. financial barriers precluding volunteer positions, preparatory courses). They reason that scores ultimately reflect these barriers, instead of differences in ability or capacity for a career in medicine. If medical schools notice a trend of URIM applicants being disqualified because of academic scores, perhaps more attention should be directed towards identifying and addressing inequalities at a community level.

The CASPer/situational judgement tests (SJT) and MMI have received a fair amount of attention as processes that could help promote diversity in health professional programs. We found higher CASPer scores in the Other group compared to the Black group, which aligns with a previous study.[19] It is unclear why ethnicity appears to impact SJT performance, but one can hypothesize how it could be linked to life-long differences in education and opportunity, as discussed above. Since the CASPer was only adopted into the application process in recent years, one likely needs more time to follow these trends and the impact on diversity in medical schools. With this early data, we note that less than five applicants in either of the URIM groups had an ineligible CASPer score, suggesting that Dalhousie medical school’s threshold at this time is not acting as a significant barrier for the URIM applicants.

In terms of MMI, we did not find a statistically significant difference between Black, Indigenous, and Other applicants. The results of previous studies have been mixed. Similar to our results, some groups have found that MMI scores were comparable between URIM and other applicants.[13],[20] A hypothetical scenario showed that the largest proportion of URIM applicants were admitted when the ranking for admissions offers was based 100% on MMI.[12] However, in medical schools across the United Kingdom, a shift from traditional interviews to increasing use of MMI did not seem to impact the proportion of URIM in the medical school class.[21] Perhaps this was related to the relative weighting of the MMI versus other components of the application, for which there may be larger differences between URIM and non-minority applicants in the other components.

The total number of applicants who self-identified as Black or Indigenous was relatively low in our population, ranging from 18-33 and 13-24 in each of the two years. This should be an area needing attention, in that expanding the pool of individuals from diverse backgrounds may ultimately help increase the enrolment of URIM students. Many strategies that have been suggested, such as altered weighting of academic and non-academic components, use of the MMI, or adjusting scores/thresholds for URIM applicant. This may increase the competitiveness of those who have applied but does not change the size of the URIM applicant pool. Splenser et al[22] studied recruitment strategies in physiotherapy schools and found that the number of recruitment strategies was not correlated with the number of minority applicants or the number of accepted minority students. However, they did note that schools with special efforts to recruit and retain minority students had more minority applicants and graduates. This suggests that recruitment efforts should be targeted and be unique to the population of interest.

Another important factor to consider is support through the application process. In our population, a large proportion of URIM applicants started but did not complete their application. Unfortunately, we do not have any information about what happened to these individuals or why their applications were left incomplete. Other groups have examined perceived challenges in the admissions process as per URIM students. A qualitative study with Black/African American and Hispanic/Latino medical students in the United States identified several perceived barriers in the admissions process, including lack of information, guidance, and finances.[23] Similarly, commonly reported barriers from a survey of American Indian and Alaskan Inuit applicants were the MCAT, insufficient finances, and lack of information.[24] Based on these findings, dedicated support and mentorship of URIM applicants through the multiple steps and challenges of applying to medical school may be beneficial in encouraging interested individuals to both start and complete an application.

Our study demonstrated that Black and Indigenous individuals who completed an application progressed well through the interview stage to acceptance offers. Eighty-one percent of Black and 65% of Indigenous applicants who completed the interview were ultimately offered a position in the first-year medical class. This is in comparison to an acceptance rate of 44% of Other applicants who completed an interview. However, we found that 31% of Black applicants declined their acceptance offer, a proportion more than three times greater than those who declined in the Indigenous and Other groups. We do not have any descriptive data about why these students declined their acceptance offer. It is possible that that they were accepted at other medical schools or had other life circumstances or financial constraints precluding them from starting the program. Considering that over one-third of Black applicants listed Ontario as their home province, it is probable that an applicant’s local/regional context is an important factor in their choice of medical school. This issue should be studied further, as a better understanding of why applicants decline on offer of admission may provide insight into novel ways that medical schools can support and retain diverse applicants.

This study was limited in that we were only able to study two URIM groups and relied on applicants to self-identify. If applicants chose not to self-identify, they would have been mis-classified into the Other group. There are many additional groups that are underrepresented in medicine; however, the Admissions Office at Dalhousie medical school does not currently collect information that would enable study of additional URIM groups. We analyzed data from a single institution, with the aim of better understanding the performance of URIM applicants in our local population. However, the generalizability of our findings may be limited by differences in the admissions processes at other Canadian medical schools, or by the unique characteristics and circumstances of Black and Indigenous individuals applying to Dalhousie medical school. Comparing our findings to those from other medical schools in Canada will be of interest, particularly those with a similar application process. We compiled data over four years of admissions cycles in order to increase the size of our groups of interest. As applicant data was de-identified, it is possible that repeat applicants were analyzed as unique individuals. However, we feel that this is unlikely to have a significant impact on the results.

Our next steps are focused on knowledge translation. Specifically, engaging with the Admissions Committee, Black, and Indigenous representatives at Dalhousie medical school to present the results and discuss the interpretation and any implications.

Conclusions

Black and Indigenous individuals who completed an application progressed well through the admissions process of a Canadian medical school, with no difference in acceptance rates compared to Other (non-self-identified) applicants. A substantial proportion of Black applicants declined an offer of admission. The pool of diverse applicants needs to be increased and support provided to ensure completion of their applications. Further study is warranted to understand why a significant number of qualified applicants choose to decline an acceptance.