Substance use screening, brief intervention, and referral to treatment training for emergency medicine trainees.

INTRODUCTION: Emergency medicine (EM) providers are in an opportunistic position to identify and intervene with patients at risk for alcohol misuse and related problems. However, alcohol screening, brief intervention, and referral to treatment (SBIRT) services are underutilized within the emergency department. Providing SBIRT training to trainees may help to increase utilization of these valuable services in the future. An SBIRT training program for EM faculty and trainees was developed and delivered to increase trainees' skills and practice of SBIRT services.
METHODS: The SBIRT training program included unique tracks for medical students, physician assistant (PA) students, EM residents and faculty preceptors. Faculty and trainees completed performance measures at the end of each training session, 30 days post-training and 12 months later.
RESULTS: SBIRT training was provided to 238 trainees and 65 faculty members. At all follow-up time points, satisfaction of training and usefulness varied by trainee type with PA students rating constructs higher than both medical students and EM residents. At the 12-month follow-up survey, there was no significant difference in ratings of sharing the information (χ2 (2)=0.38, P=0.33) between these trainees, with the majority of all trainees (96% of PA students, 83% of residents and 68% of medical students) responding that they had applied what they learned in the training to their work.
CONCLUSION: An SBIRT training curriculum for EM trainees was delivered successfully and utilized 12 months after implementation.

Introduction

From 2006 to 2014, there was an increase of nearly 50% among acute and chronic alcohol-related emergency department (ED) visits.1 Because of this the ED is an ideal setting for providers to screen for alcohol use problems. According to the American College of Emergency Physicians (ACEP), emergency medicine (EM) providers are positioned and qualified to mitigate the consequences of alcohol abuse by utilizing screening, brief intervention, and referral to treatment (SBIRT).2 Unfortunately, only 27% of all EDs and 22% of Level I/II trauma center EDs reported routinely screening and counseling patients presenting with alcohol-related complaints.3 While research supports SBIRT utilization within the ED,4,5 training barriers continue to exist which prevent the integration of SBIRT into routine clinical care.

Medical students and EM residents often have limited experience with SBIRT training. To address this lack of training, the Substance Abuse and Mental Health Services Administration (SAMHSA) provides SBIRT resources to adopt into training curricula. To date, training for SBIRT has been implemented in several residency programs nationally, mostly in primary care specialties but also within some EM programs.6,7 While some work has been conducted with medical students, residents and faculty, training for other future ED providers, ie, physician assistant (PA) students, has not been evaluated.

The primary purpose of this project was to utilize the SAMHSA resources to create an SBIRT training program for future EM providers. The training program objective was to increase the use of SBIRT among future EM providers to meet the specific needs of a patient population, which, compared to other patient populations, has increased health risks associated with alcohol and other substance misuse. We hypothesized that the proposed training would increase trainees’ (medical students, PA students and EM residents) skills and practice of SBIRT services.

Methods

We utilized SAMHSA resources to develop SBIRT training curriculum for EM trainees. Curriculum materials are available on our website.8 The curriculum included unique tracks for clinical faculty, EM residents, medical students and PA students. Faculty and trainees completed performance measures at the end of each training session, 30 days post-training and 12 months post-training. This study was reviewed by our institutional review board (IRB) and determined to be exempt from the Federal Regulation 45 CFR 46 as meeting the criteria in 45 CFR 46.101(b)(1).

Clinical faculty training

To prepare the clinical learning environment for the trainees, initial work was done with EM faculty who supervise trainees in the clinical setting. A 1-hour web-based training on SBIRT was developed for EM faculty. It included a review of SBIRT best practices, documentation tips for the electronic health record and a description of available local resources. A 15-minute video using simulated cases to describe best practice for precepting trainees in common ED SBIRT scenarios was also developed. The five cases covered in the video depicted trainees at various stages of training, including a novice learner and an overwhelmed resident. This online module was distributed to 105 physician faculty members in January 2016, who were compensated for 1 hour of work upon completion of the training module.

EM resident training

Starting in summer of 2016, EM residents attended a 1-hour session consisting of a 20-minute didactic lecture, a very brief video with best practice example for delivering SBIRT to an ED patient and small group role-playing exercises that were precepted by experienced faculty trained in SBIRT. Additionally, EM residents participated in a 1-hour SBIRT simulation training session at an affiliated simulation center led by experienced simulation faculty. The goal of the simulation center educational activity was to demonstrate conduct of best practices for ED delivery of SBIRT. During the simulation session, EM residents were divided into two groups. One resident from each group would take the role of the physician and would role play with a simulated patient and nurse. With a scripted 15-minute case, they were given a scenario where alcohol/drugs were not the primary reason for the ED visit. If EM resident screened properly it was discovered that alcohol and drug use played a major factor. In a separate room, the other residents with two faculty preceptors viewed the simulation case on a monitor. After the case concluded, residents and faculty engaged in a critique and debriefing of the residents’ actions and case-specific necessary SBIRT actions.

Medical student training

Starting in the spring of 2016, medical students during their EM clinical rotation were required to complete an SBIRT online module consisting of a substance abuse overview, SBIRT’s role in EM and skills necessary for conducting SBIRT with patients. The module also included a video clip of a simulated ED-based brief intervention. After completing the module, students met with an EM faculty member who led a 1-hour didactic group session on SBIRT practices in the ED as well as facilitated three role-playing cases where each student took a turn playing different roles (observer, provider and patient).

PA training

Similar to the EM resident training, PA students participated in a 1-hour session delivered during their required EM module at the participating PA school starting Spring 2017. It consisted of the same brief SBIRT clinical example video and a 20-minute didactic session specifically tailored to PA students. The remaining 40 minutes were dedicated to small group role-playing exercises where each student rotated through different roles (observer, provider and patient) in three different cases. EM faculty members with SBIRT experience oversaw and provided feedback during the role-playing exercise.

Evaluation

We collected evaluation data from SBIRT training participants from March 2016 until June 2018 using the requirements of the Government Performance and Results Modernization Act of 2010 (GPRA)9 at end of each training event (baseline), 30 days post-training and 12 months post-training. Self reported data were collected using the Center of Substance Abuse Treatment (CSAT) GPRA Baseline Training Satisfaction Survey (Version 2.0)10 and the CSAT GPRA Follow-Up Training Satisfaction Survey (Version 2.0)11. The CSAT surveys include questions that evaluate the participants’ satisfaction with the training and about expected or actual use of the training when working with patients, using a five-point scale (1= very satisfied/strongly agree to 5= very dissatisfied/strongly disagree).

Self-reported baseline surveys were collected in-person at the completion of each SBIRT training event. To contact participants at 30 days and 12 months, emails were collected at baseline. Using RedCap™ as a survey platform, trainees were emailed up to four times over the span of 1 week to complete the survey. Data collected were entered into SAMHSA’s Common Data Platform (CDF) within 7 days of completion. EM residents received a $5 gift card incentive for the 30-day survey. All participants received a $15 gift card incentive at 12 months upon completion.

Data analyses

The main training constructs assessed in the baseline five-point scale TSS questions included the following: training satisfaction (eight items), training usefulness in working with patients (seven items) and two items scored independently: rating of current effectiveness in working with this topic area (SBIRT) and ratings of recommendation of training to a colleague (Tables 1 and 2). The items for each of these constructs were summed and for ease of interpretation the items were reverse scored to indicate that higher scores corresponded to higher values of these constructs. At the baseline survey the maximum scores were as follows: training satisfaction =40 points, relevance =35 points, SBIRT effectiveness =5 points and training recommendation =5 points. At the 30-day and 12-month follow-up surveys, we focused on the six items that reflected the usefulness of the training (maximum score =30 points), and the three items concerning sharing/use of the training materials (maximum score =3 points). For the faculty survey, we focused on the six questions about the usefulness/relevance of the training (maximum score =30 points) at the baseline and 12 months post-training survey. We also report on their overall satisfaction with their training at baseline.

Table 1

Post-training survey constructs in analyses

Constructs	Post-training
Satisfaction with trainingItem scale =1–5	1.How satisfied are you with the overall quality of this training?2.How satisfied are you with the quality of the instruction?3.How satisfied are you with the quality of the training materials?4.Overall, how satisfied are you with your training experience?5. The training class was well organized.6. The instructor was knowledgeable about the subject matter.7. The instructor was well prepared for the class.8.The instructor was receptive to participants’ comments and questions.
Usefulness/relevance of trainingItem scale =1–5	1. The material presented in this class will be useful to me in dealing with substance abuse.2. The training enhanced my skill in this topic area.3. The training was relevant to my career.4.I expect to use the information gained from this training.5.I expect this training to benefit my clients.6. This training was relevant to substance use treatment.7.How useful was the information you received from the instructor?
Current effectivenessItem scale =1–5	I am currently effective when working in this topic area.
Recommend trainingItem scale =1–5	I would recommend this training to a colleague.

Table 2

30-day and 12-month survey constructs in analyses

Constructs	30 days and 12 months post-training
Usefulness/relevance of trainingItem scale =1–5	1. The material presented in this class has been useful to me in dealing with substance abuse.2. The training enhanced my skill in this topic area.3. The training was relevant to my career.4. This training was relevant to substance use treatment.5. The training has enables me to serve my clients better.6.How useful was the information you received from the instructor?
Sharing/using informationItem scale = Yes/No	1.Did you share any of the information from this training with others?2.Did you share any of the materials from this training with others?3.Have you applied any of what you have learned in training to your work?

Descriptive analyses were conducted using SAS (version 9.4, SAS Institute Inc., Cary, NC, USA). Mean values with SDs are reported for each group trained and across the three time points that data were collected. Analyses of the distribution of the composite scores across the time points of data collection revealed that the data were not normally distributed (Kolmogorov–Smirnov test P<0.05). To accommodate the non-normal distribution of these data, a Kruskal–Wallis test was conducted, with a priori alpha set at <0.05 to determine statistically significant differences between the trainee groups across these outcomes of training ratings.

Results

In total, 238 trainees and 65 EM faculty members received SBIRT training and completed a post-training GPRA survey. Medical students were the most frequent level of trainee. Across trainees most self-reported their race as white, being non-Hispanic and most were males except for the PA students who were mostly female (Table 3). The 30-day follow-up assessment was completed by 99 (78%) medical students, 35 (83%) residents, and 60 (87%) PA students. Forty medical students (31%), 26 residents (62%), and 25 PA students (63%) responded to the 12-month assessment.

Table 3

Characteristics of trainees

	EM residents (n=42), n (%)	Medical students (n=127), n (%)	Physician assistant students (n=69), n (%)
Female	17 (42)	58 (46)	56 (81)
Race	White: 36 (85)Asian: 5 (12)Not reported: 1 (2%)	White: 79 (62)Asian: 32 (25)African American: 8 (6)Not reported: 8 (6%)	White: 63 (91)Asian: 3 (4)African American: 1(1)Not reported: 2 (3%)
Hispanic/Latino	2 (5)	9 (7)	2 (2)

Abbreviation: EM, emergency medicine.

Immediately after training, there was a significant difference among the trainees on satisfaction with the training (χ2 (2)=19.8, P<0.001) and on the usefulness of the training (χ2 (2)=22.4, P<0.001), with the PA students rating both of these constructs significantly higher than both medical students and EM residents. The medical students had significantly higher ratings than the EM residents. The evaluation of current effectiveness on SBIRT was not significantly different between the three trainee groups (χ2 (2)=3.7, P=0.16), but was in ratings of how likely to recommend the SBIRT training to a colleague (χ2 (2)=20, P<0.001), being significantly higher for the PA students but no significant difference between the medical students and resident trainees (Table 4).

Table 4

Trainee ratings of SBIRT training

Training level	Satisfaction with training Mean (SD)	Recommending training to a colleague Mean (SD)	Relevance/usefulness of training to career Mean (SD)	Sharing information Mean (SD)
Medical student	Baseline = 36.6(3.4)	Baseline = 4.3 (0.6)	Baseline = 31.9 (2.9)30 days = 20.5 (3.3)12 months = 19.5 (3.4)	30 days = 0.95 (0.77)12 months = 1.07(0.88)
EM resident	Baseline = 34.6(5.1)	Baseline = 4.1 (0.8)	Baseline = 30.1 (5.8)30 days = 19.7 (2.8)12 months = 19.7 (2.4)	30 days = 0.57 (0.65)12 months = 1.19(0.69)
Student physician assistant	Baseline = 37.7(2.8)	Baseline = 4.8 (0.4)	Baseline = 33.4 (2.3)30 days = 22 (2.7)12 months = 22.3 (2.2)	30 days = 0.68 (0.75)12 months = 1.12(0.79)

Abbreviations: EM, emergency medicine; SBIRT, screening, brief intervention, and referral to treatment.

Table 4 also shows the findings of the follow-up surveys for the trainees. At the 30-day follow-up survey, PAs (n=60) continued to report significant higher scores on the usefulness of the training compared to the residents (n=26) or medical students (n=99) (χ2 (2)=13.5, P=0.002), and residents reported significantly higher scores on sharing the information with others (χ2 (2)=11.1, P=0.001). At the 12-month follow-up survey, there was a continued significant difference in the ratings of training usefulness (χ2 (2)=11.5, P=0.002), with the PA students (n=25) rating significantly higher than the medical students (n=40) or EM residents (n=26), but no significant difference in ratings of sharing the information (χ2 (2)=0.38, P=0.33) between these trainees. Of those who completed the 12-month follow-up survey, 96% of PA students, 83% of residents and 68% of medical students responded that they had applied what they learned in the training to their work.

The faculty mean score of their evaluation of the usefulness of the training immediately after completion was 26.1 (SD=3.27). At the 12-month follow-up, 49 (75%) EM faculty completed the survey. Their mean rating of the usefulness of the training was 23.9 (SD=4.4), with 71% of the faculty reporting that they had applied what they had learned about SBIRT to their work.

Discussion

Our educational model was unique in several ways. It initially engaged faculty with training to improve their SBIRT skills before starting educating the trainees they supervise on this topic. Faculty training was important to exemplify behavior in the ED, acknowledge the importance of curriculum and provide appropriate feedback to trainees. Our curriculum focused on multiple types of EM trainees’ SBIRT skills in the ED and was embedded into the existing educational programming for each type of trainee.

We found differences in satisfaction with the curriculum, with both medical students and PA students being more satisfied than residents. The reason for this is unclear but potentially it could be that residents have experienced frustration with dealing with substance abusing patients and felt the curriculum did not adequately address that. It is interesting that although there were some differences in curriculum satisfaction, at follow-up, all groups of trainees (96%, 83%, and 68%) and faculty (71%) reported utilizing the skills they obtained. It appears that SBIRT training for EM trainees may be an important step in adopting SBIRT services into routine clinical care within the ED.

Limitations

Interpretation of our data has several potential limitations. This was only done in a single locale by the same faculty group and might have different satisfaction and uptake within another setting or if delivered by other faculty. Additionally, training methodology differed between professional groups, making comparison challenging. Some of the 12-month follow-up responses were low, specifically for medical students. This may have resulted in a response bias with respondents not being representative of medical students who participated in the curriculum.

Conclusion

A training curriculum for SBIRT skills can be successfully delivered to faculty and various types of trainees. Furthermore, an SBIRT curriculum can result in utilization of SBIRT skills in clinical care 12 months after training. Future studies should further examine the effect of EM SBIRT training on patient substance misuse.