Impact of a medical student alcohol intervention workshop using recovering alcoholics as simulated patients.

BACKGROUND: Alcohol screening and brief intervention (SBI) reduces drinking among at-risk drinkers. Lack of training and negative attitudes represents a barrier to SBI performance. This study evaluates the impact of a medical student workshop using recovering alcoholics in simulated patient interviews to teach SBI skills.
METHODS: Third-year students (n=94) were surveyed before and after a 3-hour alcohol SBI workshop regarding their perceived importance and confidence in performing eleven SBI behaviors. Students were also asked to list factors increasing and decreasing motivation to conduct SBI. Students completing off-campus rotations (n=71) served as controls, completing surveys during the same time period but without attending the workshop.
RESULTS: Analysis of variance found a significant interaction effect between the students participating in the workshop and control students on both importance scores [F(2,174)=3.34] and confidence scores [F(2,174)=9.13], indicating higher scores for the workshop students at the follow-up time periods. Commonly listed motivators for performing SBI included clinical experience with alcohol misuse and the impact of alcohol on health and relationships. High relapse rates and patient reactions to questions about alcohol use decreased the motivation to perform SBI.
CONCLUSION: SBI workshops that include recovering alcoholics as simulated patients can produce long-term improvements in students' perceived importance and confidence in performing SBI.

Introduction

Despite widespread evidence of the health impact and economic importance of alcohol misuse worldwide1–3 and evidence that simple brief interventions by physicians can reduce alcohol misuse and related health care consequences,4–9 physician screening and brief intervention (SBI) is infrequently performed,10–14 with many physicians reporting inadequate training in this area.15–22 For several decades, key national and international organizations including the World Health Organization, American Medical Council on Mental Health, Medical Council on Alcoholism, and Office of National Drug Control Policy have called for increased education about alcohol misuse for students in medical schools.23–28

Surveys of medical students have revealed frequent negative attitudes toward patients with drug and alcohol problems,29–32 with one study indicating that the attitudes of medical students worsen during medical training.32 Because a strong correlation has been identified between physicians’ attitudes, confidence, and role legitimacy on reported screening and treatment practices with patients who misuse alcohol and other drugs,32,33 alcohol training programs typically target clinicians’ attitudes, as well as their knowledge and skills. The goal of such training is to dispel any existing biases against these patients and increase clinicians’ commitment to addressing patterns of misuse, as well as their medical or surgical consequences.30,34,35 Numerous programs designed for practicing clinicians indicate that a combination of didactic education and skills training increases clinicians’ confidence in performing alcohol SBI and increases rates of alcohol intervention.34,36–42

A 2000 review of the existing literature by El-Guebaly et al concluded that improving medical students’ knowledge and skills was easier to obtain than attitudinal shifts.43 Two studies from the early 1990s reported changes in student attitudes using a model substance abuse curriculum and an intensive interactive seminar, including seminars, 12-step meeting attendance, and role plays with standardized patients; however, changes were not sustained during clinical training.44,45 Two clinical training experiences from the last decade report positive attitudinal change. Silins et al found that a clinical rotation including 2 weeks on an addiction treatment unit resulted in decreased dislike of problem drinkers, a greater sense of responsibility towards providing intervention, and less anticipation of discomfort working with these patients.30 Similarly, Christison and Haviland found that 1-week in addiction medicine as part of a psychiatry rotation changed students’ perceptions of working with patients with substance abuse problems. Following the rotation, students were more likely to describe the patients as treatable, and worthy of medical resources.35

Similarly, workshops have been utilized to teach SBI skills and improve student attitudes toward patients with substance use problems. Two trials of 3-hour interventions in Australia found that both simple didactic training and a combination of didactic plus experiential training improved student attitudes toward patients misusing alcohol, although neither was sufficient to provide students with adequate brief intervention skills.42,46 Kahan et al found that a 3-hour skills-based workshop resulted in increases in knowledge of drinking reduction strategies, increases in assessment and management scores, and increases in self-efficacy, although commitment to treatment returned to near-baseline levels at the 4-month follow-up.47 Comparison of a brief intervention lecture versus a rich media web module resulted in increases in confidence to perform brief intervention in both groups, with web-trained students achieving higher brief intervention scores on observed structured clinical examinations; however, no information on attitudes or long-term outcomes is available.48

Standardized and simulated patient interviews (SPs) are a preferred strategy for teaching skills because they provide a safe learning environment, opportunity for feedback, and increase learners’ self-confidence.49,50 Use of recovering alcoholics as SPs to teach alcohol interview skills offers unique potential advantages: practice interviews with emotionally authentic SPs in stable recovery might increase student confidence in alcohol brief intervention while also increasing their sense of the importance of engaging patients in conversations about their alcohol misuse. A 2012 literature search found no prior studies that examined the attitudinal impact of utilizing recovering alcoholics as SPs in alcohol intervention workshops.

The following analysis describes a 3-year evaluation of an alcohol training workshop designed to produce long-term increases in students’ perceived importance and confidence in performing the specific components of alcohol SBI and to explore factors which increase or decrease students’ motivation to perform brief interventions.

Materials and methods

In this study, approved by the Mercer University institutional review board, third-year medical students participating in their family medicine rotation provided appropriate written consent and then completed a baseline attitudes survey which measured their perceived importance of and confidence in executing eleven behaviors related to alcohol screening, intervention, and referral. The behavior list was developed based on components of effective SBI intervention studies described in the US Preventive Services Task Force’s recommendation statement advocating alcohol screening and behavioral counseling for all adult primary care patients.51 Students were asked to rate these Likert scale items on a scale of 1 to 10, with 1 representing low importance/confidence and 10 representing high importance/confidence. Evaluation also included two open-ended questions: “List any aspect(s) of your medical training that has (have) increased (or decreased) your motivation to screen and intervene with patients who are problem drinkers.”

Of 172 students invited to participate, 165 were consented and completed the baseline survey. Students at the medical school’s main campus (n=94) then participated in an alcohol intervention workshop, while students completing their family medicine rotation at off-campus sites (n=71) served as study controls. While students were allowed to select the location of their rotation, with the exception of the alcohol intervention workshop, the education experience is similar across sites. All graded components, readings, testing, and student evaluations were identical and coordinated from one campus by a single coordinator. Although not randomized to these locations, there should not be any systematic differences between students electing local rotations and those in other locations.

The skills-based intervention workshop included a 1-hour lecture on the components of alcohol SBI, a 15-minute practice interview with a recovering SP that allowed them to practice the skills covered in the lecture, then a debriefing feedback session for all students and patients. A typical workshop would include five to eight recovering SPs. Over the duration of the study, approximately 35 recovering persons served in SP roles. While some SPs participated in a 2-hour training session which included practice interviews, most participated only in a half-hour orientation using a two-page description of the role they were requested to depict, ie, that of an active alcoholic on a first visit to a physician for a complaint with some relation to alcohol use (eg, accidents, abdominal pain, depression, insomnia). SPs were given liberty to provide their own details regarding their medical and social history. They were asked to admit to drinking alcohol, but be vague and evasive at first, giving more details if the doctor was persistent in asking about quantity and frequency. In response to brief intervention, they were asked to be somewhat resistant at first then agree at least to consider the student’s suggestions and come back for a follow-up visit. Specifically, they were told that one purpose of the exercise was to give students the message that at least some alcoholic patients are open to talking about recovery, and that it is worth taking the time to talk with other patients in the future. For that reason, they were asked not to portray a patient in total denial, but rather an ambivalent patient in some degree of denial who begins to open up with a physician demonstrating concern and investing time in conversation about drinking. After the interviews, all students and patients participated in a debriefing session in which SPs were asked to give positive feedback on something the student did well, one suggestion on something that might improve future interviews, and spend 5 minutes each sharing their recovery stories.

The baseline survey was readministered to all students 1 month and again 9 months after the workshop during the final year of medical school training to determine if confidence in and perceived importance of the various components of alcohol SBI increased as a result of the workshop and if changes would be retained over time. Follow-up rates were 97.5% at 1 month and 63% at 9 months, with slightly better rates for the intervention group (68%) than for the control group (55%). To test for possible response bias, independent-samples tests were performed on baseline importance and confidence items comparing those completing the 9-month follow-up with those who did not respond. The results showed significant differences on only two of the eleven importance items and none of the confidence items.

Individual items were summed to create importance and confidence scale scores (possible range 11–110) at each of the three time periods (baseline, 1 month, and 9 months). Analyses were conducted using a mixed analysis of variance design. Significant interactions indicate a significant difference between the workshop and control groups. Free text answers were entered into an Excel file and independently grouped into areas with common themes by two of the study’s authors.

Results

Comparison of workshop students with control group

Paired-samples t-tests of individual importance and confidence items showed that 18 of 22 items, including each of the eleven confidence items, increased significantly over baseline scores. At the 9-month follow-up, 12 of 22 individual means continued to be higher when compared with baseline.

At baseline, the mean score on the importance scale was 84.6±15.7 with a range of 39–110, while the mean score on the confidence scale was 92.1±12.9 with a range of 38–110. Changes in the mean scale scores resulting from the workshop were tested using separate mixed analyses of variance. The results are shown in Table 1.

Table 1

Changes in mean aggregate confidence and importance scores for workshop and control group at baseline, and 1 month and 9 months post intervention

Aggregate scores	Mean scores
	Baseline (SD)	1 month (SD)	9 months (SD)	F-statistic for main effect (P)	F-statistic for interaction (P)
Confidence
Workshop	95.3 (10.1)	100.5 (8.1)	98.2 (9.0)	2.13 (0.12)	9.13 (<0.00)
Control	92.2 (12.1)	88.7 (13.8)	93.7 (13.9)
Importance
Workshop	84.7 (16.5)	94.6 (14.4)	95.3 (10.1)	14.35 (<0.00)	3.34 (0.04)
Control	84.1 (15.4)	85.7 (10.7)	90.4 (13.0)

Abbreviation: SD, standard deviation.

There was a significant interaction effect between confidence and group [F(2,174)=9.13], indicating that the overall level of confidence in SBI differed significantly between students in the workshop group and those in the control group. The F-statistic for the main effect of the intervention was not statistically significant [F(2,174)=2.13], suggesting that the workshop did not significantly increase students’ overall confidence in performing SBI.

Contrasts were performed comparing 1-month and 9-month scores on confidence with baseline scores for the workshop and control group. These contrasts revealed a significant interaction when comparing the two groups’ scores at 1 month with baseline [F(1,87)=14.89]. Contrasts comparing 9-month scores with baseline were not statistically significant [F(1,87)=0.420], as shown in Figure 1.

Figure 1

Mean confidence scores for baseline, 1-month, and 9-months post intervention.

For the importance measure, there was also a significant interaction effect with group [F(2,174)=3.34], indicating that the overall level of importance of SBI differed significantly between students in the workshop group and those in the control group. In addition, the results showed a statistically significant main effect [F(2,174)=14.35], suggesting a significant increase in the perceived importance of SBI among those completing the workshop. As with the confidence measure, contrasts revealed a significant interaction when comparing the two groups’ scores at 1 month with baseline [F(1,87)=5.80], and a nonsignificant interaction when comparing 9-month scores with baseline [F(1,87)=1.70], as shown in Figure 2.

Figure 2

Mean importance scores for baseline, 1-month, and 9-months post intervention.

Free text responses

Results from the two open-ended questions revealed common themes for both the intervention and control groups at baseline. The two most common responses regarding factors that increased students’ motivation to conduct SBI were the students’ clinical experience with alcoholism and the effect of alcohol on patients’ physical health. These responses occurred about three times as frequently as the next most common response, ie, the effects of alcoholism on relationships and the family. Clinical experience and physical health responses continued to be the most common response for both groups at the 1-month and 9-month follow-ups. At the 9-month follow-up, however, the alcohol workshop emerged as the third most common response among the intervention group, mentioned by eleven of 64 respondents.

There were less than half as many open-ended responses regarding factors that decreased motivation to screen and intervene. The most common responses for both the control and intervention groups were the difficulties associated with treating patients with alcohol disorders, the high incidence of relapse, and patients’ negative reactions when asked about their alcohol use. These themes remained the most common responses for both the control and intervention groups at both the 1-month and 9-month follow-up.

Discussion

These results are consistent with previous studies in showing that brief alcohol workshops can improve medical students’ confidence or self-efficacy in addressing alcohol-related problems through brief interventions.36,37 The results also indicate that these types of workshops impact student attitudes (ie, importance of performing SBI), and that changes can persist over time. Recent research has shown that implicit measures (eg, importance and confidence) are often more predictive of actual behavior than explicit measures.52,53 When compared with a control group, workshop participation was associated with higher scores on 20 of 22 importance and confidence items related to performing alcohol SBI tasks 1 month after training and 12 of 22 items after 9 months. When these items were combined into overall measures of importance and confidence, mixed analysis of variance showed statistically significant differences between the workshop group and the control group, with scores increasing for the workshop group after training but remaining static or decreasing for the control group. After 9 months, differences between the two groups converge slightly, but both confidence and importance scores remain higher for the workshop group.

The lack of a significant main effect for confidence scores is likely the result of already high mean scores on this measure (92 out of a possible 110), leaving little room for significant change. Prior to training, medical students may perceive the skills required to address patients’ alcohol use as less difficult than the skills needed to address other acute or chronic conditions, resulting in inflated scores on confidence. Mean confidence scores did increase for the workshop group, but the increase was not enough to reach statistical significance.

Students’ free text responses on factors that increased their motivation to screen and intervene with problem drinkers are instructive. Comments regarding factors that motivated students to intervene outnumbered comments on factors that decreased motivation by a ratio of 4:1. Seeing the clinical impact of problem drinking, particularly its physical sequelae, appears to be a strong motivator for students to screen and intervene. Clinical experiences with problem drinkers were the predominant motivating factor for both the intervention and control groups across all phases of the study. Mentions of the impact of alcohol on physical health were also common in both groups throughout the study. Students’ free text responses regarding factors that decreased their motivation to screen and intervene with problem drinkers are also instructive, although the overall number of comments was small. Students in both the control and intervention groups most commonly listed the difficulty in treating problem drinkers and high relapse rates, factors that would appear to relate to the “treatment pessimism” described in previous studies.54 Similarly, patients’ negative responses to questions about alcohol decreased students’ motivation, perhaps reflecting many physicians’ desires to have positive patient encounters that leave both patients and physicians feeling good after the encounter. Future studies may wish to explore whether these factors decline as students become more proficient in interview techniques, such as motivational interviewing, that focus on collaborative approaches equipping students to assist patients in evoking their own reasons and rationale for change.55

The most innovative aspect of this study is the use of recovering alcoholics as simulated patients. Using brief predetermined scenarios, the patient–actors have the freedom to use details from their own personal experience to make the interview more realistic. Students are not told in advance that the SPs are recovering alcoholics. During the debriefing period at the end of the workshop, students often make positive comments about the authenticity of the answers and emotional reactions of SPs. This suggests that the SP approach may combine positive aspects of both “real” patient and standardized patient encounters by providing both authenticity and the opportunity for feedback.49 As SPs reveal their recovery status at the close of the workshop, students may recognize the “hidden” nature of alcohol misuse and realize the importance of screening everyone in their future practices. One potential area for future study would be to attempt to measure the impact of such an experience on students’ commitment to routinely screen all patients for alcohol misuse and intervene as indicated (so-called “therapeutic commitment”) and on their actual SBI practice behaviors.

The study has several limitations. Students were not randomized to the control and intervention conditions, although there is no indication that there are systematic differences between the two groups that would affect the study results. Nonetheless, there were baseline differences in the control and intervention groups that limit our ability to attribute all these changes to the workshop alone. Reasons for these differences are unclear. Surveys were administered by office personnel during the family medicine rotation orientation, making it unlikely that these responses reflect a social desirability bias.

While the use of recovering SPs is a unique addition to the workshop, it is not possible to separate the effect of this aspect of the workshop from the workshop’s other components. Future studies should randomize students into workshops with and without recovering SPs to determine if there is an independent effect of this workshop component.

The follow-up rates at 9 months may also have impacted the study results, particularly the low follow-up rate (55%) for control group participants at the 9-month follow-up. This could impact both the statistical power necessary to identify statistically significant results as well as introduce possible nonresponse bias.

Finally, there was variability in the training of SPs, with some receiving more extensive training than others. This variability could have been reflected in the SP/student encounter. We are unable to link individual SPs to student responses and therefore unable to account for difference in training in the study results.

Overall, the study indicates a very positive impact of a brief alcohol workshop. Future studies should attempt to link the impact of such a workshop to alcohol SBI behaviors after students move into residency.