Evaluation of a Resident-led Residency Preparation Series for Fourth Year Pharmacy Students.

DESCRIPTION OF PROBLEM: The increased interest in residency programs nationwide has made the application process more competitive and complex. STATEMENT OF INNOVATION: In 2015, Wake Forest Baptist Health (WFBH) created a resident-led residency preparatory series (RPS) to assist advanced pharmacy practice experience (APPE) students in preparing for residency applications. This study sought to evaluate the perceived value of the resident-led RPS by fourth year APPE students. INNOVATION: This single-center, survey-based, descriptive study aimed to evaluate the perceived impact of a resident-led RPS. All APPE students during the 2016-2017 and 2017-2018 academic years who completed rotations in the Triad region of North Carolina were invited to participate in the RPS. Surveys were sent to eligible students and data was captured for respondents who indicated an interest in pursuing residency and attendance to at least one RPS. Survey questions evaluated satisfaction with the RPS. Survey data was collected and analyzed using Qualtrics©. CRITICAL ANALYSIS: A total of 84 students were invited to attend the RPS. Thirty-three respondents participated in the RPS and indicated an interest in applying for a residency. The resident-led RPS was consistently well received by the majority of students over the course of two years, with over two-thirds of all respondents finding every session extremely or very useful. The majority of students agreed or strongly agreed with all confidence statements.
CONCLUSION: A resident-led RPS resulted in positive student perceptions and increased confidence in the residency application process. © Individual authors.

Description of Problem

With the continuing evolution of pharmacy as a clinical profession, more students are pursuing postgraduate training.[1-3] In 2019, the National Matching Services reported 5,937 applicants participating in the match for a post graduate year 1 (PGY1) residency program, up from 5,560 in 2018. To meet the rising number of residency candidates, the number of new residency programs has also increased steadily. From 2013 to 2019, the number of PGY1 residency positions offered has grown 20%, from 2,694 to 3,359.[2,3] Despite the increase in residency positions, the number of residency candidates has increased at a faster rate, which has resulted in over one third of residency applicants unable to match with a PGY1 position in 2019.[3]

Due to the nationwide increase in pharmacy residency supply and demand, the application process has become more diverse and complex. To centralize and standardize the residency recruitment process, most residency programs require applicants to apply through an online application portal, Pharmacy Online Residency Centralized Application Service (PhORCAS). Despite utilization of a standardized application portal, each residency program has slightly different requirements, including variations in letters of recommendation format, number and types of preferred references, and interview styles. For many students, navigating this increasingly complex process can be confusing, overwhelming, and time intensive. In addition, many students do not fully comprehend the Match process including Phase I, Phase II, and the Post-Match.

Given the competitive and complex nature of the residency application process, institutions have developed coursework and structured activities to increase students’ knowledge, confidence in, and preparedness for the residency application process. A survey of pharmacy colleges identified structured activities used to prepare students for residency training.[4] Of the 71 colleges who responded, only 16 offered structured, formalized programming with many more offering informal programs and sessions.[4]

Statement of Innovation

Recent articles have highlighted the positive impact of residency panels and faculty-directed residency interest groups.[5-6] Although it has been reported that insight from residents is helpful to pharmacy students in the preparation for residency applications, there are no reports of a residency preparatory program developed, coordinated, and led by residents.[6] Wake Forest Baptist Health (WFBH) provided approximately 190 advanced pharmacy practice experience (APPE) student rotation blocks each year from two universities, Wingate University School of Pharmacy (WUSOP) and Campbell University College of Pharmacy and Health Sciences (CUCPHS). In response to verbal student and faculty feedback expressing concern over the limited resources and time spent preparing students for obtaining post-graduate training, WFBH PGY1 residents created a residency preparation series (RPS) to assist APPE students in preparing for the residency application and interview process. The RPS is coordinated and executed by the PGY1 residents, with encouraged participation from the entire residency class. During the 2016-2017 academic year, there were eight PGY1 residents and eight PGY2 specialty residents. By the 2017-2018 academic year, this number increased to 11 PGY1 residents and eight PGY2 specialty residents. The aim of this work was to evaluate the perceived value of the resident-led RPS by the consumers, fourth year APPE pharmacy students.

The Innovation

Design

All APPE students from WUSOP and CUCPHS who completed rotations within the Piedmont Triad (Greensboro-Winston-Salem-High Point) regional zone of North Carolina during the 2016-2017 or 2017-2018 academic years were invited via email to attend the RPS sessions.

The RPS consisted of five educational sessions delivered over 6 months (Table 1). Sessions were held on weekday evenings from 1700 to 1830 to allow student attendance after rotations. Students were sent a schedule at the beginning of the academic year outlining dates, times, and subject matter. The dates and topics of the RPS were designed to coincide with the residency application process. Attendance at the RPS was optional and had no impact on grades or academic standing. All material was developed and delivered by the active PGY1 residents, without contribution or review from local faculty. Students were not required to complete any preparatory work prior to attending the RPS. Each year, two PGY1 pharmacy residents volunteered or were selected by the chief resident to lead the RPS. The two lead residents were responsible for organizing and advertising the RPS, reviewing and updating subject matter from the previous year, setting up each session, and overseeing the series. Additionally, they served as the point people for questions and communication between students, residents, and faculty regarding the series. The chief resident supported the two lead PGY1s on the project, and also offered insight into the structure and process from the previous year. With the requirement that the WFBH chief resident must complete their PGY1 residency at WFBH to be eligible for chief, it ensured that they were involved in the series the previous year and could help maintain consistency. Each individual session was presented by two PGY1 residents, with oversight and assistance from PGY1 leaders when needed. The sessions were primarily informational, with time built in for hands-on feedback, discussion, and question and answer depending on the subject matter. PowerPoint® presentations were created for each session. Due to the significant number of students interested in having their curriculum vitae (CV) reviewed in the 2016-2017 school year, the didactic CV preparation was restructured by adding a supplemental session. All students were invited to attend this supplemental session hosted by the PGY1 residents and the chief resident. Students were instructed to bring 15 copies of their CV for a round-robin style feedback session. Attendees (both students and residents) then rotated CVs and provided as much feedback as possible in 10 minute timed intervals over 90 minutes. This allowed students to have multiple reviewers in a short time frame. Faculty from both universities provided pizza and encourage attendance.

Table 1.

Topics and Student Perceptions of the Resident-Led Residency Preparatory Series

Session	Topics  Discussed	Goals/Objectives	Respondents Indicating Session Attendance	Respondents Indicating Very or Extremely Useful (%)
September	Introduction	Introduce the RPS Discuss types of post-graduate training and residency positions Review ASHP residency application timeline Provide time for resident/student meet and greet	28	21 (75)
October	Review of CV, letter of intent, and letters of recommendation	Discuss common “do’s” and “don’ts” to CV writing Identify strengths and weaknesses in good and bad examples of CVs and letters of intent Review the general goal and structure of letter of intent writing Review timing and ways to professionally ask for positive letters of recommendation	29	21 (72)
November	Navigating a showcase, applying for a residency, and PhORCAS	Summarize ASHP’s annual residency showcase offered at Midyear Discuss how to prepare, how to dress, what to bring, and general tips for success. Summarize the application process and timeline including PPS interviews, thank you notes, requesting transcripts, navigating PhORCAS, etc. Explain PhORCAS and illustrate the process looks like through navigating the website	29	22 (76)
January	Preparing for interviews	Discuss what to expect, how to schedule interviews, how to prepare, and how to and how not to dress Describe professional interview etiquette Provide sample interview questions Discuss methods of responding to interview questions	25	17 (68)
February	Ranking, match, and scramble	Explain the match process from an applicant and program perspective Discuss making and submitting a rank order list Explain Phase I, Phase II, and the scramble process and provide timeline for each Provide resources for students to gain more information	19	15 (79)

Abbreviations: RPS: residency preparatory series; ASHP: American Society of Health-System pharmacists; CV: curriculum vitae; PPS: personnel placement service; PhORCAS: Pharmacy Online Residency Centralized Application Service

Data Collection and Analysis

To ensure the series was meeting student needs, a post-RPS evaluation was developed. Students indicating an interest in pursuing residency and attending at least one residency preparatory seminar were eligible for inclusion in the evaluation. Participants indicating no interest in pursuing residency were routed to survey completion and excluded from the analysis. Data was collected and analyzed using Qualtrics© (2018, Qualtrics, Provo, Utah, USA). Descriptive statistics were used to summarize survey response data. This study was approved by the institutional review board at WFBH.

Results

A total of 84 students completed APPEs in the geographic region and were invited to attend the RPS. The evaluation was electronically distributed to all eligible students and responses were received from 77% (n=65). Of those responding to the evaluation, 65% (n=42) indicated an interest in applying for residency. A total of 33 students indicating interest in applying for residency attended at least one RPS session, providing our study population. Thirty two students submitted at least one residency application, and 27 (84%) of respondents who applied for a residency program matched.

Attendance at the five individual sessions decreased over time, with the final session having the lowest attendance (Table 1). Forty-two percent (n=14) attended all available sessions while 73% (n=24) attended over half of the RPS sessions. A total of 21% (n=7) attended a single session. Cumulatively, over two-thirds of all respondents found every session extremely or very useful (Table 1). The majority of respondents agreed or strongly agreed with all confidence statements (Table 2).

Table 2.

RPS Participant Confidence in Residency Process

Confidence Statement	Respondents	Respondents Indicating Strongly Agree or Agree (%)
I understood the residency application process overall	31	30 (97)
I felt prepared for the residency application process overall	31	25 (81)
I felt confident in finding pertinent residency program information (resource retrieval)	31	27 (87)
I felt confident in asking for letter(s) of recommendation	30	16 (53)
I felt confident in preparing my CV for a position	31	29 (94)
I felt confident in navigating a nationwide showcase	31	24 (77)
I felt confident in submitting my residency application to PhORCAS	31	23 (74)
I felt confident in interviewing for a residency position	31	18 (58)
I felt confident in evaluating a residency program post-interview	31	26 (84)
I felt confident in ranking residency program(s) (n=31)	31	26 (84)
I felt confident in navigating a statewide showcase	29	24 (83)
I felt confident in participating in ASHP’s PPS	7	4 (57)
I felt confident in participating in PhORCAS’ Phase II or Scramble	6	4 (67)

Abbreviations: CV: curriculum vitae; ASHP: American Society of Health-System Pharmacists; PPS: personnel placement service; PhORCAS: Pharmacy Online Residency Centralized Application Service

Respondents reported submitting an average of 11 applications (SD ± 4.6), receiving an average of six interviews (SD ± 2.8), and ranking an average of five programs (SD ± 2.6). Likewise, all participated in outside preparatory activities (e.g. independent reading, online activities, mentoring, national programming).

Critical Analysis

The resident-led RPS was well received by the majority of students over the course of two years, with 30 respondents (97%) stating they felt confident in their understanding of the overall residency process (Table 2). A resident-led, rather than a traditional faculty-led approach provides students with practical and relatable teachers. Residents have firsthand experience with the recent changes in the application process, including navigating PhORCAS and understanding the match process (Phase I, Phase II, and Post-Match). Furthermore, this RPS provided teaching and mentorship opportunities for residents as new practitioners.

Fewer students participated in the ASHP personal placement service (PPS) or Phase II of the Match. This contributed to a smaller respondent number for those questions. Confidence in residency interviews demonstrated the lowest score in our survey results. In two published programs providing residency preparation to students, the opportunity for mock interviews yielded the highest satisfaction rating or confidence rating.[5,6] Additionally, 92.6% of students in a stand-alone mock interview program felt that these interviews helped them with their residency interviews.[7] Survey data from a national mock interview session reports that 80% of their participants found the session useful for their residency preparation.[8]

The match rate for our survey respondents exceeded 75% in both years, above the national average of 64% in 2019.[3] This is consistent with published literature from other programs showing an increased match rate for students participating in a preparatory program.[5-8]

Individual session attendance declined over time, although those that attended later sessions still found them valuable. The last session, “Ranking, match, and scramble” had the lowest attendance both years. This session was held in February, falling in the middle of interview season, which may explain the decreased attendance. All respondents also indicated participation in other residency preparation activities.

Next Steps

A resident-led RPS resulted in positive student perceptions and increased confidence in the residency application process. The unique administrative model had several benefits, including participants receiving advice from near peers. Further improvement could include expanding the program to include mock interviews. Future analysis could compare outside programming to the value students reported receiving from the RPS, as well as match rates between students who did not complete the RPS and those who did. Lastly, coordination of efforts with state, regional, or national preparatory programming could lead to less competition, repetitiveness, and thus higher attendance for all programs.