Novel Integration of Administrative Pharmacy Residents in a Management Course.

BACKGROUND: Management skills are an essential component of a pharmacy graduate's abilities for successful practice. Although pharmacy education standards require that students have a working knowledge of management principles, students often do not see the value in management and business courses. One innovative approach is restructuring course content using case examples and real-world experiences to improve student understanding of finance and management principles. INNOVATION: Two specific changes were implemented in a second year (P2) management and finance course to improve the relevance of business principles. Course content was organized around current pharmacy service cases from a variety of practice settings and supported by the value of problem-based learning. Post-graduate year 1 (PGY-1) administrative pharmacy residents were engaged as course teaching assistants (TAs) who brought real-world experiences into the class. An analysis of pre- and post-course voluntary surveys, course evaluations, and TA evaluations assessed the impact of the course redesign.
FINDINGS: The course redesign achieved its intended goal of improving student-perceived course relevance. This was shown through statistically significant improvements in course evaluations that were intended to measure student perception of pharmacy management and its relevance in their future career. Student completed TA evaluations showed that those who reported their TA shared real-world applications had higher confidence in applying course concepts and greater understanding of course materials.
CONCLUSIONS: Administrative pharmacy residents were successfully integrated into a pharmacy management course redesign, resulting in improved student perceptions of course relevance. © Individual authors.

DESCRIPTION OF THE PROBLEM

Management skills are an essential component of a pharmacy graduate’s professional practice[1-2], because after all, there is “no mission without the margin.” The Accreditation Council for Pharmacy Education has recognized this in Standard 2.2, Medication Use Systems Management, which sets the expectation that doctor of pharmacy graduates are “able to manage patient healthcare needs using human, financial, technological, and physical resources to optimize the safety and efficacy of medication use systems.”[3] The field of pharmacy education has recognized that pharmacy students tend to be less than enthusiastic about management and business courses and do not fully comprehend the importance of such classes.[4-6]

STATEMENT OF INNOVATION

This lack of enthusiasm was experienced by the instructors of the 2-credit required course Managing Pharmacy Systems for Patient Care, offered by the University of Wisconsin-Madison School of Pharmacy. Course evaluations from the 2019 spring semester showed that students did not recognize the relevance of the course content to their future practice, (for example, one student wrote “I do not feel that any of the material that was used will be applicable to when I become a pharmacist.”) prompting the instructors to redesign the course for the 2020 spring semester.

The instructors reviewed the pharmacy literature to determine how others have improved relevance in their business courses and sought input from the administrative pharmacy team at a local partner academic teaching hospital. The instructors supplied the administrative team with the syllabus, objectives for each lecture and the course evaluation comments from students. As a result, the instructors elected to implement a two-pronged approach to increase course relevance for students:

(1) Organize course content around current pharmacy service cases across a variety of practice settings, supported by evidence of the value of problem-based learning with progressive cases.[7]

(2) Engage PGY-1 administrative pharmacy residents as course TAs, supported by evidence that guest lecturers with real-world experience can improve course relevance[8-10]

The instructors expected that sharing of real-world examples would underscore the relevance of management and finance principles, and that PGY-1 administrative pharmacy residents would be better able to demonstrate relevance and daily application of these principles in healthcare and pharmacy practice. This paper describes a partnership between a pharmacy management course and a PGY-1 administrative residency program, and reports on the process of the integration and research findings.

Process of Integration

The modular design of the course provided an overview of management and finance principles and discussed their application across three pharmacy settings (Table 1). With six discussion sections but only five resident TAs, one additional TA was recruited from the third-year pharmacy student (P3) class (the previous source of all TAs for this course). Each TA led one discussion section of 21-25 students which met weekly for 90 minutes. The instructors and TAs met in weekly staff meetings starting two weeks before the semester began to discuss and gather feedback on the plan for the upcoming weeks. Each TA was expected to include their own personal examples into the basic set of presentation materials and were encouraged to customize as needed to support their students. The COVID-19 pandemic required asynchronous delivery of the last three weeks of the course; subsequently, the last two weekly discussions were converted to remote, self-guided student activities.

Table 1.

Overview of Course Module Focus and Attributes

Module Number	Module 1	Module 2	Module 3	Module 4
Module Focus	Overview	Retail Pharmacy	Ambulatory Pharmacy	Hospital Pharmacy
Lectures and Discussions	5 Lectures 2 Discussions	7 Lectures 2 Discussions	4 Lectures 2 Discussions	10 Lectures 3 Discussions
Case Description	No Case	Case 1: Exploring the development of an injectable naltrexone service in a community pharmacy	Case 2: Interpreting and improving quality measures for an ambulatory geriatric hypertension clinic	Case 3: Manage a medication shortage of amiodarone vials with premixed infusion bags Case 4: Evaluating profitability of beside medication delivery programs
Topics Addressed in Lectures	Introduction and Entrepreneurship Operations Management Overview Financial Statements (2 lectures) Pharmacy Costs-Labor and Inventory	Retail Pharmacy Operations Retail Pharmacy Revenue Retail Pharmacy and Service Planning Case 1 Justifying and Planning Patient Care Services (2 Lectures) Cost of Service Break-even and ROI	Ambulatory Operations and Revenue Evaluating a Clinic Population Health Management Service Quality Improvement in Pharmacies Achieving and Measuring Patient Satisfaction	Hospital Pharmacy Operations Hospital Pharmacy Revenue Managing a Drug Shortage in a Hospital Pharmacy Managing to Improve Patient Safety: Errors, What errors? Role of Technology Evaluating an “Underperforming” Med-to-Bed Service Average Net Profit Comparison Differential Analysis and Pro Forma Analysis Advertising and Promotion
Focus of Weekly Discussion Sections	Entrepreneurship Financial Statements and Pharmacy Costs	Retail pharmacy service planning process (2 Sessions)	Service planning and evaluation Quality and patient satisfaction	Safety in Operations Profitability (2 Sessions)

In return for PGY-1 administrative pharmacy resident support, the school provided the residents with two credits of independent study and structured the experience as a teaching rotation within their residency program. One instructor served as the lead preceptor for the teaching rotation and had previous experience precepting residents in clinical and management rotations. The lead preceptor worked with the residency program director to establish the outcomes for the rotation and then established the evidence for achieving each outcome with the other course instructor. The outcomes and evidence were presented to the TAs at the beginning of the semester to ensure understanding of expectations. The instructors scheduled observations for each TA at the midpoint and end of the course to determine outcome achievement and submitted final evaluations on rotation performance at the end of the course. Resident TAs also completed an evaluation at the conclusion of the rotation and provided feedback on their experience. The P3 TA was compensated monetarily. She was observed and evaluated in the same manner as the residents, but did not complete a post-course evaluation.

CRITICAL ANALYSIS

The impact of the course redesign was evaluated via three outcomes: (a) a pre/post voluntary anonymous survey, that examined changes in student perceptions about management and financial principles; (b) the course evaluation; and (c) a modified TA evaluation. The last two outcomes are standardized across all courses in the School of Pharmacy. All analysis was conducted using SPSS 25 (IBM Corp. Armonk, NY). This course redesign and evaluation was certified as quality improvement/program evaluation by the University of Wisconsin-Madison Health Sciences Institutional Review Board (IRB).

Voluntary Survey

The survey was administered at the beginning of the course (paper-based) and at the end of the course (electronically via Qualtrics, Provo, UT). The survey used a five-point Likert scale (1= strongly disagree to 5=strongly agree) and consisted of 17 management and finance principles questions and ten additional questions asked only on the post-course survey. Using an independent sample t=test, the pre- and post-course voluntary survey results (response rate of 90% on the pre- and 75% on the post- survey) showed significant improvement in agreement with the statement Pharmacy management topics are relevant to my future pharmacy career, increasing from 3.5 (SD 1.17) at the beginning of the semester to 4.0 (SD 0.99) at the end of the semester (p < 0.001).

Course Evaluation

All courses at the school of pharmacy are evaluated by students using a standardized, web-based course evaluation. The evaluation includes 13 questions about the course (5=Extremely to 1=Not at all) and an overall score for course quality (7=A to 1=F). We compared responses to the 2019 and the 2020 course evaluations with an independent sample t-test that tested for homogeneity of variances to assess the impact of the course redesign on student perceptions of the course overall. Significant improvements existed in every evaluation item from 2019 (n=138 students, 98.6% response) to 2020 (n=133 students, 96.4% response). The letter grade students assigned regarding overall course quality also improved from an average BC grade (3.96 ± 1.8) to between a B and AB (5.64 ± 0.97, p < 0.001).

TA Evaluation

The last assessment was the TA evaluation, which included 12 questions used campus-wide to evaluate the quality of TA teaching and provide feedback on TA performance. Three additional TA evaluation questions, related to real world experiences, were included specific to this inquiry (Table 2). Each question was scored using a 5-point Likert scale (5=Extremely to 1=Not at all). Two groups were created – students with resident TAs and students with P3 TA - to determine if student perceptions varied by the type of TA given the greater management exposure of the resident TAs. The results from the independent sample t-test considering the results of the homogeneity of variance test (Table 2) found significant differences between the TA groups (resident vs. P3) for seven of the twelve evaluation questions. When significant differences existed, students rated the P3 higher than the hospital residents. For example, This TA explained material clearly (residents = 4.5 versus P3 = 4.9, p=0.035) and The topics discussed during my discussion section contributed to my understanding of the course material (residents = 4.2 versus P3 = 4.8, p=0.004). A Chi-Square analysis examined association between responses to the real-world applications question on the TA evaluation and responses to the two questions about understanding and confidence by TA group. Students who reported their TA shared real-world applications also reported higher confidence in applying course concepts (χ2 =125.49, p=<.0001) and greater understanding of course materials (χ2 =125.49, p=<.0001) regardless of TA type. The course and TA evaluation were worth 5 points (out of 375 points) for the course.

Table 2.

Comparison of Hospital Resident Versus P3 Student Teaching Assistant Evaluation Results (n=134)

	Hospital Resident TA	P3 TA
	M (SD)	M (SD)	t-test
1.  This TA made the objectives of each discussion clear.	4.5 (0.74)	4.8 (0.40)	2.67b
2.  This TA displayed enthusiasm when teaching.	4.6 (0.60)	4.8 (0.43)	1.29
3.  This TA encouraged student participation in discussion activities.	4.7 (0.54)	4.9 (0.29)	2.19b
4.  This TA explained material clearly.	4.5 (0.81)	4.9 (0.35)	3.49c
5.  This TA stimulated my thinking.	4.4 (0.84)	4.7 (0.57)	1.44
6.  This TA was able to help facilitate my understanding of the course material.	4.5 (0.78)	4.9 (0.35)	3.20c
7.  The topics discussed during my discussion section contributed to my understanding of the course material.	4.2 (1.01)	4.8 (0.40)	5.09d
8.  This TA shared real-world applications of course concepts.	4.7 (0.64)	4.8 (0.40)	0.92
9.  The real-world applications of course concepts shared by my TA contributed to my understanding of the course material.	4.4 (0.90)	4.8 (0.43)	2.98c
10.  The real-world applications of course concepts shared by my TA contributed to my confidence in applying the course concepts.	4.4 (0.92)	4.7 (0.57)	1.57
11.  This TA treated all students with respect.	4.9 (0.37)	4.95 (0.21)	1.55
12.  If you had to give the TA a letter grade e for the overall quality of teaching, what would it be?	6.7 (0.73)	7.0 (0.00)	3.31c

The five point Likert scale for the TA evaluation questions were 5=Extremely to 1=Not at all.

p<0.05,

p<0.01,

p<0.001.

Letter grades were coded using a reverse 7-point scale: A (7), AB (6), B (5), BC (4), C (3), D (2) and F (1). An AB is equivalent to an A- and a BC letter grade is equivalent to a B-.

KEY ISSUES

The course redesign with integration of resident TAs achieved its intended goal of improving student-perceived course relevance and resulted in significantly improved course evaluations. Students who felt more strongly that their TA provided real-world applications of course concepts also reported higher confidence in understanding and applying these concepts. This supported our underlying expectation that real-world applications would help improve student-perceived relevance. However, we did not find a difference by TA type in the perceived frequency of TAs providing real-world applications in class. Instructor observations of TAs leading their discussion sections supported the finding that all TAs incorporated real-world applications in their teaching; however, as students only interacted with a single TA throughout the semester, students are unable to compare the quality and quantity of real-world applications across all TAs.

The P3 TA significantly outperformed the administrative residents in ability to explain concepts clearly and support student understanding of course material. One explanation for this result could be that the residents assumed a higher knowledge among the students (which was directly observed by one instructor); the P3 TA may have been better able to determine the right level of explanation and how the content fit with the other courses the students were taking and/or the students’ past experiences. Another explanation could be that the P3 self-selected into the role and was more skilled at making relevant connections for this student cohort. Overall, students gave their TAs very positive evaluations, and a mix of residents and P3 students as TAs can be used going forward.

This analysis has several limitations. The anonymity of the pre- and post-surveys did not allow for comparisons for individual students. Fewer students completed the post-survey; if less-engaged students disproportionately did not complete the post-survey, this may have resulted in artificially higher scores. However, participation rate in both surveys was quite high, so this is less of a concern. A number of changes were implemented simultaneously, and students were not surveyed on every change that was implemented. Therefore, it is not possible to isolate which components of the course design led to positive outcomes.

NEXT STEPS

The persistence of necessary COVID-19 changes of the delivery approach and course structure for the Spring 2021 semester goes beyond changes that were planned based solely on the redesign results. Going forward, a mixture of asynchronous recorded lectures and synchronous learning opportunities will be used to deliver course content. Administrative pharmacy residents will again serve as course TAs. While a modular case study approach will be employed, student knowledge will be evaluated utilizing a new longitudinal group business plan assignment. In this semester, the course was offered online with no face-to face sessions. The business portfolio promotes student interactions and provides a realistic way to apply the concepts taught in the course to a real-world application. TAs will interact with their assigned student groups through this assignment instead of leading synchronous discussion sessions, and in this way, continue to share their relevant management experiences. The standardized course evaluations and a post-course survey will inform the impact of this revised structure, which continues to leverage real-world applications.