Novel Use of Text-Bot Automation for Residency Recruitment.

Purpose. To describe the employment of an automated text messaging text-bot during the 2019 American Society of Health-System Pharmacists (ASHP) Midyear Clinical Meeting Residency Showcase and its impact on the number of applications received for the postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) pharmacy residency programs at a medium-sized community hospital. Methods. Visitors at the residency showcase booth were asked to text a code word to a program number. The text-bot collected the visitor's contact information and program of interest (PGY1 or PGY2). A series of automated messages were sent to all visitors following the showcase and up until the residency application deadline. Results. 71% (20/28) of visitors to the program's showcase booth registered with the text-bot and of these, 65% (13/20) submitted applications to the residency program in phase I of the match. Both the PGY1 and PGY2 programs saw an increase in the amount of applications received compared to previous year. Conclusion. A text messaging text-bot may be a useful residency recruitment tool. © Individual authors.

Background

Despite a continuous increase in the number of applicants participating in the match over the past years, unmatched positions are concerns in residency programs. In 2019, approximately 8% (302/3530) of PGY1 positions and 16% (135/829) of PGY2 positions were unmatched after phase I of the National Matching Services (NMS) resident matching program.[1] Programs with filled positions ranked 2.4 more applicants per position than programs with unfilled positions after phase I.[1] Additionally, 6% (19/302) of the unmatched PGY1 positions and 20% (26/135) of the unmatched PGY2 positions failed to rank a single applicant.[1] To increase application numbers, residency programs have traditionally relied on in-person and virtual recruitment efforts; however, utilization of social media platforms to engage with prospective candidates has recently been described as an effective recruitment strategy.[2] While recruitment via social media has merit, the effort to provide content and continuous engagement on one or more platforms may be burdensome for residency programs.

A recent study of 2nd and 3rd year pharmacy students identified attendance at the American Society of Health-System Pharmacists (ASHP) Midyear Clinical Meeting (MYCM) as a predictor of intention to apply to residency programs.[3] Another study of pharmacy students found a correlation between positive fear of missing out (FOMO) scores and intention to apply.[4] Although neither of these studies followed students longitudinally to determine actual residency application rates, utilizing strategies that capitalize on FOMO during the ASHP MYCM may be an effective opportunity for residency programs to solicit more applications.

A text robot, or text-bot, is a computer program that provides automated responses and collects user data based on responses to algorithm questions. When used in social media and short messaging systems (SMS), or text messaging, a text-bot can be a useful marketing tool.[5] The text-bot can be designed to ask participants specific questions in order to gather data as well as efficiently communicate messages to a specific audience.[6] Promoting registration with a text-bot as an exclusive opportunity to receive communication about the residency program may take advantage of the FOMO phenomenon.

Purpose

The purpose of this quality improvement project was to describe the innovative employment of an automated text messaging text-bot during the 2019 ASHP MYCM Residency Showcase and its impact on the number of applications received for the postgraduate year 1 (PGY1) and postgraduate year 2 (PGY2) pharmacy residency programs at a community hospital in rural Minnesota.

Methods

The residency site studied offers two PGY1 positions and one Ambulatory Care PGY2 position annually. The program site is located at a 154-bed community hospital and primary care clinic in southern Minnesota.

A text-bot was designed for the residency program using the SMS messaging platform, Shout®, to gather the following data: name, email address, phone number, and position of interest (PGY1 or PGY2). During the 2019 ASHP MYCM Residency Showcase, all visitors to the residency’s booth were asked to register by texting a trigger word to a given number. The text-bot was promoted at the residency booth with a poster asking visitors to register, and through direct conversations with the booth’s representatives. However, visitors were not required to register with the text bot, nor with an alternative method. The residency booth’s representatives kept a simple tally of all the people that visited the booth. Following the showcase, automated messages were sent nine days after the showcase and on the application deadline to remind visitors to submit their application. Figure 1 outlines the text-bot algorithm used.

Figure 1

Text-Bot Algorithm

The primary measure of this quality improvement project was the percentage of visitors to the showcase booth that registered with the text-bot. Secondary measures included: percentage of showcase visitors that applied to the program and total number of applications received by the program.

Results

Visitors at the residency program’s showcase booth were not tracked prior to 2019. The number of applications received for each position in the previous year were 11 per PGY1 position and 6 per PGY2 position during phase I of the match.

During the 2019 ASHP MYCM Residency Showcase, a total of 20 (71%) of the 28 visitors who stopped at the program’s booth registered using the text-bot. Of those who registered with the text-bot, 16 were interested in the PGY1 positions, 3 were interested in the PGY2 position and 1 did not specify interest. Of those who expressed interest in PGY1 positions, 10 (63%) submitted an application during phase I of the match, accounting for 36% of the total applications received for that program. All 3 (100%) of those interested in the PGY2 position submitted an application during phase I of the match, accounting for 33% of the total applications received for the PGY2 program. Additionally, one person who did not register with the text-bot, but who did visit the residency showcase booth, also applied to the PGY2 position.

A total of 37 applications were received in phase I of the match. 28 applications were for PGY1 positions (14 per position) and 9 applications were for the PGY2 position, representing a total increase of 27% for the PGY1 positions and 50% for the PGY2 position.

Figure 2 displays the number of positions, total number of applications, and the ratio of applications per position from 2018 to 2019 for both the PGY1 and PGY2 program.

Figure 2

Number of positions, total number of applications, and ratio per position

Discussion

The SMS text-bot was an effective tool in capturing visitor’s information during the residency showcase. A high percentage of visitors to the booth registered with the text-bot. The high number of participants who then submitted applications to the program is consistent with the previous study correlating ASHP MYCM attendance and intention to apply to residency.[3]

Both PGY1 and PGY2 programs saw growth in the number of applications from the previous year that roughly correlated with the number of text-bot participants. However, this quality improvement project was not designed to assess statistical significance for this outcome. Nor is it possible to attribute the increase in applications solely to the use of the text-bot as this was not a controlled study and many confounding variables may have influenced the results. One such variable of note is that the PGY1 program increased the number of available positions from one in the previous year to two, and this may also have influenced the number of applications received. However, the number of applications per position still saw an increase. A lack of baseline data for visitor participation in the previous year makes it difficult to compare results.

Showcase participants accounted for less than half of the applications received for either the PGY1 or PGY2 positions. This suggests other recruitment efforts, such as local residency exhibits or Personnel Placement Service (PPS), likely contribute a majority of applications. Nevertheless, the residency showcase is a powerful recruitment forum as approximately one third of applications did come from visitors to the residency booth.

One limitation of using a text-bot for gathering information is the possibility for visitors to input incorrect information. However, this could be true for other forms of registration such as a written log; and, would not be limited to the text-bot technology. One way to control for incorrect data entry would be to ask visitors for their ASHP membership number, or to scan the QR code on their name badge.

Specific feedback about the use of the text-bot itself was not formally gathered. Anecdotally, several of the residency booth’s visitors commented on the uniqueness of the text bot and that it was not something they had encountered at other residency booths. However, a survey to gather participant’s impressions on the text-bot tool may be an opportunity for future research.

Feasibility and cost of employing a text-bot is a valid concern for residency programs. The Shout® platform described in this study does require an annual fee, ranging from $12 to $20 per month, billed annually, depending on the type of account (customer service or marketing) and number of text messages managed.[7] The text-bot for this study was developed by the assistant residency program director with a customer service account managing up to 750 text messages per month, and took an estimated 20 minutes to create initially. The text-bot is easy to modify, making it relatively simple to update for other events. This suggests the text-bot may be a feasible tool for many residency programs.

Conclusion

An automated text messaging text-bot may be a useful residency recruitment tool to capture visitor information during ASHP MYCM Residency Showcase, or similar events.