Association between the Medical College Admission Test scores and Alpha Omega Alpha Medical Honors Society membership.

INTRODUCTION: Medical schools worldwide are faced with the challenge of selecting from among many qualified applicants. One factor that might help admissions committees identify future exceptional medical students is scores on standardized entrance exams. The purpose of this study was to determine the association between scores on the most commonly used standardized medical school entrance exam in the USA, the Medical College Admission Test (MCAT), and election to the US medical honors society, Alpha Omega Alpha (AOA).
METHOD: MCAT scores and AOA membership data were analyzed for all the students pursuing Doctor of Medicine degrees at the University of Minnesota Medical School and who graduated between 2012-2016 (n=1,309).
RESULTS: An independent-samples t-test found a significant difference (t=6.132, p<0.001) in MCAT scores between those who were elected to AOA (n=179) and those who were not (n=1,130). On average, students who were elected to AOA had composite MCAT scores of 1.65 points higher than those who were not. Percentages of students elected to AOA gradually but inconsistently increased with MCAT score. No student who scored <27 on the MCAT was elected to AOA. Among students with MCAT scores at the 99th percentile or above (scores of ≥38), 13 of 48 (27.1%) were elected to AOA. DISCUSSION: Election to AOA during medical school was significantly associated with higher MCAT scores. Admissions committees should carefully consider the role of standardized entrance exam scores, in the context of a holistic review, when selecting for exceptional medical students.

Plain language summary

Medical school admissions committees are faced with the challenge of identifying potential top-performing medical students from among many highly qualified applicants. One indicator of exceptional performance in medical school in the USA is membership in the national medical honors society, Alpha Omega Alpha (AOA). The purpose of this study was to explore whether membership in AOA was related to scores on the Medical College Admission Test (MCAT), the most commonly used standardized entrance exam for medical school in the USA.

The researchers analyzed data from 1,309 students at the University of Minnesota Medical School who graduated in 2012–2016. They found that those students who were members of AOA did have statistically significantly higher MCAT scores than those who were not. As MCAT scores increased, so did the proportion of students who were AOA members. No student who scored <27 on the MCAT was elected to AOA in the five graduating classes in this study.

These results find that admissions committees should consider MCAT scores, in the context of a holistic review process, when attempting to identify future exceptional performers at their medical school.

Introduction

Amid calls for transparent and holistic admissions practices, medical school admissions committees worldwide debate how to weight the various possible criteria they may use to discern future exceptional performers from among many highly qualified applicants. One indicator of exceptional academic performance in medical school in the USA is election to the national medical honors society, Alpha Omega Alpha (AOA). This study attempts to examine the relationship between AOA membership and scores on the Medical College Admission Test (MCAT), the most frequently used standardized entrance exam for medical school in the USA.

AOA is the national medical honors society in the USA. Founded in 1902, it now has 128 chapters in the USA and Lebanon, and more than 150,000 members have been elected since its founding.1 Each medical school with a chapter elects up to one sixth of their expected graduating class from the top quartile of academic performers.2 The specific process for election is determined by the medical school, and processes vary widely. Since election to AOA provides students with a certain stature that tends to positively influence their future careers,3 calls have been made for fairness, transparency, and periodic evaluations of election processes.4

The literature shows that election to AOA has positive repercussions for medical students applying for residency. In a national survey of Residency Program Directors in the USA, the National Resident Matching Program (NRMP) found that 61% of residency programs consider AOA membership when selecting students to interview, and 45% of them consider AOA when ranking students.5 Decroff et al3 found that AOA membership significantly improved a student’s chances of matching into the more competitive specialties. They found that AOA membership was also beneficial for matching into less competitive specialties, but less so than for more competitive specialties. This pattern is carried out in the national-level match data from the NRMP, with up to 51% of successfully matched US allopathic seniors in competitive specialties such as dermatology belonging to AOA.6 Research in orthopedic surgery residency programs has found that AOA membership correlates significantly with global evaluation scores7 and trends toward significance with in-training examination performance.8

As medical school admissions committees consider which applicants to select for their institutions, it is helpful for them to be able to identify potential high-performing students. Standardized exams are an attractive option for evaluating applicants, as they allow for the comparison of applicants across a heterogeneous pool of undergraduate institutions.9 In the USA, the most commonly used standardized entrance exam for medical school is the MCAT. Research has found the MCAT to be valuable for predicting medical school success10,11 and licensing exam scores.12,13 However, a literature search found no research to date on the association between MCAT scores and AOA membership. The purpose of this study, therefore, was to explore the association between MCAT scores and AOA membership in a large sample including students from five graduating classes at the University of Minnesota (UMN) Medical School. We hypothesized that AOA membership would be significantly associated with higher MCAT composite scores.

At UMN, there are two steps in the AOA election process. First, a de-identified class rank list is compiled based on course grades and US Medical Licensing Exam (USMLE) Step 1 scores, and students in the top quartile are invited to submit essays, the topic of which is usually related to personal qualities such as professionalism and empathy. These essays are then scored on a scale of 1 (worst)–10 (best) by members of the AOA Committee, including students, faculty, and alumni. The exact number of graders varies by year, but typically each essay is scored by at least 10 graders. The highest and lowest scores are dropped, and the remaining scores are averaged. The essay scores are then added to the class rank scores such that the essay score comprises 20% of the overall score, and the list is reranked. The top members of the reranked list are then invited to join AOA.14

Method

Institutional approval

Ethical approval for this research was granted by the Institutional Review Board (IRB) at UMN (on May 4, 2017; IRB ID: STUDY00000329). The IRB waived the requirement of informed consent for this medical student record review.

Participants

The participants for this study included all students pursuing doctor of medicine degrees at UMN who graduated in 2012–2016 and had MCAT scores on record (n=1,309). At UMN, medical students matriculate at either the Twin Cities campus or the Duluth campus. They complete the first 2 years of the degree (foundational science courses) at their campus of matriculation, and then, all students complete the second 2 years of the degree (clinical clerkships) through the Twin Cities campus. Of the students in this study, 974 (74.4%) students matriculated at the Twin Cities campus and 335 (25.6%) matriculated at the Duluth campus. Table 1 provides means and SDs for exam scores, as well as demographic information.

Table 1

Demographic information, MCAT scores, and USMLE Step 1 and Step 2 CK scores for graduates of the University of Minnesota Medical School between 2012 to 2016, divided by membership in the national medical honors society, AOA

	Number of male/female	Mean age at matriculation (SD), yearsa	Mean MCAT composite score (SD)a	Mean USMLE Step 1 score (SD)a	Mean USMLE Step 2 CK score (SD)a
AOA members (n=179)	79 (44.1%) M; 100 (55.9%) F	23.30 (2.12)	32.84 (3.00)	245.86 (12.30)	253.85 (12.02)
Non-AOA members (n=1,130)	582 (51.5%) M; 548 (48.5%) F	23.86 (2.93)	31.19 (3.41)	224.16 (18.13)	235.97 (15.73)
Total (n=1,309)	661 (50.5%) M; 648 (49.5%) F	23.78 (2.84)	31.42 (3.40)	227.13 (18.97)	238.44 (16.47)

Note:

Difference between AOA and non-AOA groups is significant at the p<0.05 level.

Abbreviations: AOA, Alpha Omega Alpha; CK, clinical knowledge; F, female; M, male; MCAT, Medical College Admission Test; USMLE, US Medical Licensing Exam.

A new version of the MCAT was released in 2015. All students in this study took the version of the MCAT that was in use prior to the 2015 revision, since all the included students matriculated prior to 2015. Possible composite MCAT scores for that version ranged from 3 to 45. For students in the current study, composite MCAT scores ranged from 19 to 44 (mean =31.4, SD =3.4).

Sources of data

Academic, demographic, and AOA membership data were collected from student records held by the Office of Medical Education in the Academic Health Center at UMN. Data regarding participants’ MCAT composite scores were retrieved from UMN’s access to primary application data through the American Medical College Application Service. For students who had multiple MCAT scores, the highest composite score was utilized, as this is the score that is given most weight by the admissions committee at UMN. The data used in this study were de-identified.

Analyses

Using SPSS version 22 (IBM, Armonk, NY, USA), we performed an independent samples t-test to examine the association between AOA membership and MCAT scores. We used SPSS and Excel 2016 (Microsoft, Redmond, WA, USA) to calculate numbers and percentages of students with each MCAT score who were elected to AOA.

Results

Table 2 presents the numbers and percentages of students who scored each MCAT composite score and were elected to AOA. Figure 1 presents the cumulative percentages of students elected and not elected to AOA for each MCAT composite score. Overall, 179 of the 1,309 students in the data set (13.67%) were elected to AOA. Percentages of students elected to AOA gradually but inconsistently increased with MCAT score. No student who scored <27 on the MCAT was elected to AOA in the five graduating classes included in this analysis. Among students with MCAT scores at the 99th percentile or above (scores of ≥38), 13 of 48 (27.1%) were elected to AOA.

Table 2

MCAT composite scores for students at the University of Minnesota Medical School who graduated between 2012–2016 (n=1,309), the number of students achieving each score, and the number and percentage of students with each score who were elected to the AOA Medical Honors Society

MCAT score (Pre-2015 composite)	Total number of students with score	Number of AOA members with score	Percentage of AOA membership within score (number of AOA members with score/total number of students with score)
19	1	0	0.0
20	0	0	–
21	0	0	–
22	4	0	0.0
23	13	0	0.0
24	18	0	0.0
25	37	0	0.0
26	35	0	0.0
27	45	2	4.4
28	92	8	8.7
29	90	12	13.3
30	153	24	15.7
31	174	21	12.1
32	173	20	11.6
33	126	15	11.9
34	124	27	21.8
35	80	17	21.3
36	62	14	22.6
37	34	6	17.7
38	28	7	25.0
39	10	3	30.0
40	4	0	0.0
41	3	1	33.3
42	2	2	100.0
43	0	0	–
44	1	0	0.0
Total	1,309	179	13.7

Abbreviations: AOA, Alpha Omega Alpha; MCAT, Medical College Admission Test.

Figure 1

Cumulative percentages of students who were (n=179) and were not (n=1,130) elected to AOA for each MCAT composite score, for graduates of the University of Minnesota Medical School from 2012–2016 (total n=1,309).

Abbreviations: AOA, Alpha Omega Alpha; MCAT, Medical College Admission Test.

An independent samples t-test found a significant difference (t=6.132, p<0.001) in MCAT scores between those who were elected to AOA (n=179) and those who were not (n=1,130). On average, students who were elected to AOA had composite MCAT scores 1.65 points higher than those who were not.

Discussion

The results of our analyses confirm our hypothesis that there is a relationship between standardized entrance exam (MCAT) scores and election to the national medical honors society (AOA). Students who were elected to AOA had significantly higher MCAT scores than students who were not. In fact, no student with a composite MCAT score <27 was elected to AOA at UMN in the five graduating classes under review. This pattern of higher scores for AOA members is further illustrated by the cumulative percentages of each group scoring each MCAT composite score in Figure 1. It should be noted, however, that AOA membership was far from perfectly predicted by MCAT score. MCAT score should be viewed as a predictive, but not deterministic, factor for eventual AOA membership. Our results support the continued use of the MCAT in medical school admissions, in the context of the holistic process advocated by the Association of American Medical Colleges (AAMC).15,16

Medical school admissions committees must balance the desire to select the best possible applicants for their institutions with an understanding that not all future physicians need to be exceptional academically. Many very successful physicians are not AOA members, and aspects of successful medical practice such as empathy and professionalism, while included to an extent in the AOA election process, are de-emphasized in the election process relative to academic performance. Even when it comes to applying for residency, AOA membership is not necessary for a successful match process, even in the most competitive residency programs.6

A recent article by Boatright et al. investigated racial and ethnic disparities in AOA membership among medical students from 123 allopathic US medical schools. The authors found that, even after controlling for possible confounding factors, black and Asian medical students were less likely to be AOA members.17 Furthermore, a study of national MCAT data in 2013 found that black and Latino examinees had lower mean MCAT scores than white examinees, although the same article also concluded that the difference was not due to racial bias in the exam.18 These articles suggest that medical schools must be proactive in attempts to avoid bias, both when admitting students and when electing students to AOA.

Limitations

In April 2015, the AAMC released a new version of the MCAT, featuring revised content and a new scoring system.19 Therefore, future medical school applicants will take a version of the MCAT that is not the same as the version under analysis in the current study. It is a fair assumption, however, that the versions are not so different that the clear pattern found in our analyses will not continue to hold for the new version of the exam. Furthermore, the current analyses provide foundational information on the old version of the exam that will help facilitate the analysis of the new version.

It could be argued that AOA membership is simply a reflection of test-taking ability, given that Step 1 scores and grades (which are themselves largely composed of test scores) comprise a large part of the election criteria. In that case, the association between high MCAT scores and AOA membership could simply be due to both measuring the same criterion of test-taking ability. However, AOA election criteria at UMN also include a personal essay and clinical clerkship performance. Therefore, AOA membership, at least at UMN, does appear to capture more than simple test-taking ability. On the other hand, the predictive value of essays in accurately reflecting the personal characteristics of applicants is debatable. Future research should explore the extent to which AOA membership reflects desirable personal characteristics outside of academic and test-taking ability and whether essays are the most effective method to select for those characteristics.

Conclusion

In our study of 1,309 medical students over five graduating classes, we found a significant association between MCAT scores and membership in AOA. As AOA membership is a reflection of exceptional medical school performance, as well as beneficial for matching into competitive residency programs, medical school admissions committees should consider the role of standardized entrance exam scores in the context of a holistic review when selecting for successful medical school applicants.