Criteria for selection to anesthesia residency programs: a survey of Canadian anesthesia program directors.

INTRODUCTION: Applicants to specialty programs lack guidance on knowing what exactly is desired by selection committees and program directors. Anesthesia is especially opaque, given its failure to provide transparency reports nationally. This study was developed to survey Canadian anesthesia program directors about the aspects of the application package desired in an anesthesia applicant. The primary objective is to identify the preferred attributes of anesthesia applications by those mandating the selection committees.
METHODS: Survey was developed via Google Surveys, and sent online over a period of two months in June and July 2020. All program directors were sent requests for filling in the survey. STATA was used for all statistical analyses. Two analyses, Mann-Whitney and ANOVA tests, were performed for comparison groups. A p < 0.05 was considered significant.
RESULTS: Fourteen of seventeen (83%) Canadian anesthesia program directors completed the survey. Having done an anesthesia elective, good performance in it, and excellence of preclinical academic performance were considered among the most important aspects of the application package with the highest ranking important and smallest standard deviation. Any form of red flag was also considered an important criterion, again with little variation among program directors. The reference letters selected by the applicants were also important, with a personal relationship and well written reference being identified as most important (p < 0.05).
CONCLUSIONS: An applicant who has good academic performance, having anesthesia elective experience, personal, well-written reference letters, and general activity and interests that are not necessarily anesthesia-focused would be favoured by Canadian anesthesia programs.

Introduction

Each year, Canadian medical students interested in anesthesia use the Canadian Residency Matching Service (CaRMS) to apply to postgraduate residency programs. CaRMS facilitates the process, by streamlining the application package, which includes reference letters, personal letters of intent, the medical student personal record (MSPR, previously known as the Dean’s Letter), the curriculum vitae, transcripts, and other accessibility concerns, to each individual program as selected by the applicant.[1] Once received, entire application packages are studied, interviews are offered, and after a consideration that includes both the CaRMS package and the interview with the applicant, positions are offered to the anesthesia residency by the programs.

In 2019, the Association of Faculties of Medicine Canada’s (AFMC) Undergraduate Medical Education Committee activated the AFMC Student Elective Diversification Policy (SEDP), which enforced a national cap of eight weeks for each student and recommended that programs provide more information regarding the preferred specific content of their application given the cap in electives.[2] This selection transparency is part of an extended effort since 2013, where the AFMC created the Best Practices and Selection report that identified the attributes of some of the applicants matched to their programs.[3]

Unfortunately, not all anesthesia programs have provided this information despite continuous pressures from the AFMC,[4] nor have they regularly updated the Best Practices and Selection report. Questions therefore remain how best an applicant is able to demonstrate their interest and acceptability to any given anesthesia program. Do programs emphasize research extracurricular activities and which ones; are advocacy and community health projects or entrepreneurial projects preferred? Currently, anesthesia applicants do not have tangible guideposts for their applications in Canada. Previous match rates provide few clues, where students have a varied match history based on elective choices alone (Figure 1).

Figure 1

Number of applicants who matched in anesthesia. Orange is applicants who took 3 or more electives, blue is 1 to 2 electives, and red is 0 electives in anesthesia.

Given this gap in information and actionable efforts for anesthesia applicants, we surveyed Canadian anesthesia program directors to try to determine aspects of the application package most considered in the selection of future residents. We hypothesize that programs will prefer candidates whose technical skills have been directly witnessed, as seen in other surgical subspecialties (5), rather than reliance on a curriculum vitae. The primary objective of this study was to identify important criteria for selection to anesthesia residency by those guiding the selection committees, and to provide information to anesthesia applicants to better tailor their future applications.

Methods

Survey development

The survey was constructed in a similar manner as a previous survey of urological selection committees.[5] Much of the urological discussion focussed on the CaRMS application specifically. we expanded the questions to consider the entirety of the selection process including focusing on anesthesia-specific questions such as technical skill and also added coronavirus-specific questions. This further included segmenting the survey into questions regarding general electives, references, general comments on the MSPR, academic performance, extracurriculars, career aspirations, and coronavirus-related questions. We furthermore asked general questions such as preferred language proficiency, quality of research publications, and program directors’ satisfaction with the national CaRMS process.

The survey was created on Google Forms (Google Form, Mountain View, California) with 78 questions (Appendix A). 5-point Likert scales, with 5 being “very important” and 1 being “not at all important” were used. The survey was only available in English.

Data collection

Initial contact was made with anesthesia program directors nationally at their annual meeting in June 2020which was held virtually. Four email reminders were sent in a span of three weeks. The survey was closed on July 15th, 2020. All results collected were anonymous.

Statistical analysis

All data were catalogued on Excel (Microsoft, Richmond, Virginia). The mean score for each question was calculated. No statistical analysis with comparison groups was performed given the small sample size. The qualitative comments were coded for inductive analysis, similar to a method by Joanna Briggs.[6]

Results

General characteristics

Fourteen of seventeen (83%) program directors filled out the survey, results of which can be found in Appendix A. The average number of years as program director was 4 years (1.5 to 7, SD 1.57), with the mean score of selection satisfaction of 3.85 (2 to 5, SD 0.86). The program directors saw the national changes as slightly beneficial to anesthesia applicants (average 2.93, SD 1.21) and similarly beneficial to anesthesia programs (average 3.14, SD 1.17).

Electives

Program directors noted that taking an anesthesia elective at either home schools (average 4.36, SD 0.84) or others (average 4.12, SD 0.89) was particularly important. At least one elective in anesthesia was important (average 4.32, SD 0.9), with no importance given to clinical electives in anesthesia subspecialties. Only exposure to a general anesthesia rotation was considered important (average 3.71, SD 1.14). The only non-anesthesia elective rotation considered important was intensive care (average 3.07, SD 1.0), while rotations in cardiology, respirology, general surgery, and emergency medicine were not deemed important.

Academic performance

There was a high degree of agreement that red flags such as academic probation were important (average 4.71, SD 0.61). Preclinical academic performance was important (average 4.00, SD 1.11), with English proficiency being similarly important (average 3.29, SD 1.27). Of all degrees and since Quebec applicants can be admitted directly from CEGEP (grade 13), having an undergraduate degree was found to be somewhat important (average 2.93, SD 1.38).

Letters

Program directors generally agreed that it was important that reference letters be from anesthesiologists (average 3.86, SD 0.53). The content of the letter considered important included the referee knowing the student personally through work or otherwise (average 4.07, SD 0.47), the letter being well written (average 3.93, SD 0.62), and a statement indicating that the student is above average in knowledge (average 3.36, SD 0.63).

The contents of the MSPR deemed important were comments on patient empathy (average 3.93, SD 1.02), technical skill (average 3.64, SD 1.01), the ability to deal with crisis (average 3.57, SD 1.02), and, with equal importance, comments on effectiveness, time management, and ability to work within a multidisciplinary team (average 3.43, SD 1.08). Core rotation comments were largely considered equally important, from internal medicine, emergency medicine, and surgery (average 3.57, SD 1.02).

Extracurricular activities

Of all the extracurriculars, the most important was considered community involvement (average 3.86, SD 0.66), followed by advocacy work (average 3.64, SD 1.01). Program directors did not agree on the importance of having research experience. Although it was identified that research experience could be general (average 3.00 SD 1.04) and not necessarily specific to anesthesia (2.71 average, SD..), it is the quality of the research that was perceived to be important (average 3.43, SD 1.22).

Qualitative comments

Table 1 notes qualitative comments replying to the open-ended question “Anything else to add?” 9/14 program directors included comments for the survey, with 6 of the 9 adding comments specific to the selection process. The majority (4/6) highlighted concerns surrounding the MSPR, noting it provided little value given overwhelming positiveness and lack of national standardization. There was a comment regarding the difficulty in assessing honesty of applicant, such as if they mentioned “specific interest in attending my program.”

Table 1

Codified qualitative comments by program directors.

Number	Full Comment	Coded
1	A useful MSPR that actually give you a sense of an applicant's performance in medical school (that was consistent across universities) would be the most useful addition to an application give that it is impossible to have all students complete electives at your institution so you can get to know them better.	MRSP
2	Some of my answers reflect the lack of trust I put in self report. For example, if one tells me on their personal letter that they are SPECIFICALLY interested in attending my program it is rarely honest.	Honest applicant
3	Dean's letters provide no value for CaRMS.	MRSP
4	The CanMEDS criteria are components of certification that programs must sign off at the completion of a residency, so I am not sure that this section adds to medical students.	CANMeds
5	We're mulling about how best to improve our program. MRSP removing may be one.	MRSP
6	We find that the Dean's letter and reference letters are generally not useful and they do not allow us to distinguish between candidates. All are overwhelmingly positive.	MRSP

Discussion

In this national survey, program directors in anesthesia reported that having performed well in an anesthesia elective and excellence in preclinical academic performance were, from their perspective, the most important aspects of the application package. Any form of red flag was also considered an important criterion, again with high ratings and little variation among program directors. The reference letters selected by the applicants were also important, with a personal relationship and well written reference being identified as most important. The comments on the MSPR were less important, with the exception of comments on patient empathy and during anesthesia electives. It was identified that there was no advantage to taking subspecialty anesthesia electives over a general anesthesia elective.

Taking electives in specific fields is highly valued by many selection committees.[5] As shown in Figure 1, the empirical evidence is that most successful anesthesia applicants take three or more electives in anesthesia, with an average of 4.2 electives in anesthesia and 7.7 electives outside of anesthesia.[1] This survey found, however, that only an ICU elective was considered important.

The survey suggests there is explicit encouragement for applicants to take anesthesia electives at their home universities which similarly drives universities to choose applicants who have taken electives at their university. Applicants have noted that they prefer taking an elective at a home school since it takes time to fully understand a culture of a place, the electronic systems, OR environment which all affect their ability to perform and meaningfully contribute.[7]

This ability to take a broad range of courses is important, as the SEDP mandates only eight weeks in anesthesia, which itself was noted as not important. Beyond the practical aspect that ensuring more well-rounded physicians who do not forget their fundamentals of medicine,[7] this survey suggests that hyperspecialization for an applicant is not necessary when applying to anesthesia. This differs from what has been seen in other professions,[8,11] likely because in anesthesia, such widespread knowledge is necessary in the practice of anesthesia.

This generalist approach was seen for extracurriculars such as research and personal activities. Although students with research experience were successful in their match,[9] research or publications in anesthesia in particular was not overwhelmingly important. Similarly, participation in anesthesia interest group, another previous indicator of success to matching in other disciplines such as family medicine[10] was also not important. Even the reference letter is better valued for general comments about the applicant, such as their empathy towards patients than on their technical skills during an anesthesia elective.

The AFMC changes with SEDP are meant to highlight these more general characteristics of an applicant,[4] but this survey notes that there are still difficulties in its implementation and general process. With most program directors complaining of the unhelpfulness of the Dean’s Letter due to its undiscriminating positivity, the elements of the anesthesia CaRMS application process remains the same. Future steps will look at how anesthesia program directors’ factor other personal demographics such as gender and race, how they recommend changes on a national stage, and how they expect the policy to change regarding visiting electives.

There are several limitations to this study, however. Only program directors were surveyed, instead of members of entire selection committees who also review applications. Furthermore, questions regarding personal demographics of applicants beyond geographic preference were omitted. Both were outside the scope of this study.

Conclusion

We found little disagreement among Canadian program directors regarding important criteria in applicants in their selection process. This survey reports they value candidates with good academic performance, anesthesia elective experience, personal, well-written reference letters, and general activity and interests that are not necessarily anesthesia-focused

	Rank	Average	SD	Rank
Red flags such as academic probation	1	4.71	0.61	3 - 5
Anesthesia elective performance at home school	2	4.36	0.84	1 - 5
Student taking at least one anesthesia elective	3	4.32	0.8	3 - 5
Anesthesia elective performance at any school	4	4.12	0.89	2 - 5
Referee personally knows the student by clerkship or elective	5	4.07	0.47	3 - 5
Preclinical academic performance	6	4.00	1.11	2 - 5
Reference letter is well written	7	3.93	0.62	3 - 5
Comments on patient empathy	8	3.93	1.02	1 - 5
Comments during anesthesia electives	9	3.93	1.07	1 - 5
Reference letters are from anesthesiologists	10	3.86	0.53	3 - 5
Community involvement	11	3.86	0.66	3 - 5
Reference letter states student is competitive in any residency	12	3.71	0.61	2 - 4
Exposure to general anesthesia	13	3.71	1.14	1 - 5
Comments on technical skill	14	3.64	1.01	1 - 4
Advocacy work	15	3.64	1.01	1 - 5
Career direction in personal letter clear with applicant’s history	16	3.64	1.08	2 - 5
Comments on ability to deal with crisis	17	3.57	1.02	1 - 5
Comments on performance during core medicine rotation	18	3.57	1.02	1 - 5
Comments on performance during core ER rotation	19	3.57	1.02	1 - 5
Comments on performance during core surgical rotation	20	3.57	1.01	1 - 5
Athletic accomplishments and involvement	21	3.57	0.94	1 - 5
Taking an elective at home school	22	3.43	1.02	1 - 5
Comments on effectiveness	23	3.43	0.94	1 - 4
Comments on time management	24	3.43	0.94	1 - 4
Political leadership involvement	25	3.43	1.02	1 - 5
Comments on working with multidisciplinary team	26	3.43	1.08	1 - 5
Quality of research in any other field	27	3.43	1.22	1 - 5
Reference letter states student is above others in knowledge	28	3.36	0.63	2 - 4
Artistic and musical activities	29	3.36	0.84	1 - 4
English proficiency	30	3.29	1.27	1 - 5
Student taking an ICU elective	31	3.07	1	1 - 4
Reference states student is able to make unique contributions	32	3.07	0.92	1 - 4
Entrepreneurial activities	33	3.07	0.92	1 - 4
Comments that the student will be competitive to anesthesia	34	3.00	1.24	1 - 5
Exposure to general research in any topic	35	3.00	1.04	2 - 5
Quality of research in anesthesia	36	3.00	1.18	1 - 5
Having an undergraduate degree	37	2.93	1.38	1 - 5
Length of time doing research in any topic	38	2.93	1.14	1 - 4
Number of publications in any topic	39	2.79	1.05	1 - 4
Reference is a highly recognized in the respective field	40	2.71	0.73	1 - 4
Exposure to general research in anesthesia	41	2.71	0.83	1 - 4
Being a student at the home institution	42	2.71	0.61	2 - 4
Want to work in the local community	43	2.71	0.73	1 - 4
Comments on patient conduct	44	2.64	0.92	1 - 5
Number of research articles published in anesthesia	45	2.64	0.75	1 - 4
Applicant coming from the city they are applying to	46	2.57	1.02	1 - 4
Having an PhD	47	2.57	1.09	1 - 4
Number of research projects in any topic	48	2.57	1.09	1 - 4
Reference letters from internal medicine	49	2.50	1.02	1 - 4
Taking maximum number of anesthesia electives	50	2.43	1.02	1 - 4
Number of general conferences	51	2.43	0.94	1 - 4
Number of conferences presented in anesthesia	52	2.43	0.85	1 - 4
Number of research projects in anesthesia	53	2.43	0.76	1- 4
Want to do research in the future	54	2.43	0.76	1 - 4
Want to work at your home institution	55	2.36	1.22	1 - 5
Want to work rurally	56	2.29	1.07	1 - 4
Reference letters from general surgeons	57	2.29	0.91	1 - 4
Student taking an ER elective	58	2.21	1.12	1 - 4
Having a master’s degree	59	2.21	0.8	1 - 3
Exposure to chronic or acute pain	60	2.14	1.03	1 - 4
Student taking a cardiology elective	61	2.14	0.95	1 - 4
French proficiency	62	2.14	1.17	1 - 5
Having an MBA	63	2.14	0.77	1 - 3
Want to work at University hospital	64	2.14	0.77	1 - 3
Student taking a respirology elective	65	2.07	0.92	1 - 4
Having multiple language proficiency	66	2.07	0.92	1 - 4
Activity in an anesthesia interest group	67	2.07	1	1 - 4
Student taking an ENT elective	68	1.93	0.83	1 - 3
Exposure to obstetrical anesthesia	69	1.93	0.83	1 - 3
Want to work in private practice	70	1.93	0.73	1 - 3
Exposure to pediatric anesthesia	71	1.86	0.86	1 - 3
Student taking a general surgery elective	72	1.79	0.97	1 - 4
Coming from an Ivy League school	73	1.29	0.47	1 -2