The impact of an immunization training certificate program on the perceived knowledge, skills and attitudes of pharmacy students toward pharmacy-based immunizations.

OBJECTIVE: To assess the impact of a national immunization training certificate program on the perceived knowledge, skills and attitudes of pharmacy students toward pharmacy-based immunizations.
METHODS: The study design utilized a pre- and post-survey administered to pharmacy students before and after the American Pharmacists Association's (APhA) Pharmacy-Based Immunization Delivery program. The primary outcome explored was a change in the perceived knowledge, skills, and attitudes of the pharmacy students. A five-point Likert scale (i.e. strongly agree = 5, strongly disagree = 1) was used for measuring the main outcomes, which was summated by adding the individual item scores in each section to form a composite score for each outcome.
RESULTS: The certificate training program resulted in a significant improvement in knowledge (38.5% increase in score, p<0.001) and skills (34.5% increase in score, p<0.001), but not attitudes (1% increase in score, p=0.210).
CONCLUSIONS: The national immunization training certificate program had a positive impact on the perceived knowledge and skills of pharmacy students. However, no change was observed regarding students' perceived attitudes toward pharmacy-based immunizations.

INTRODUCTION

Influenza causes an estimated 110,000 hospitalizations and 20,000 deaths every year in the United States.1 Influenza vaccination is the most effective way to prevent infection from the virus; therefore, public health initiatives have been set forth to increase vaccination rates for influenza among other vaccinations. Allowing pharmacists to administer immunizations is one potential solution to increase vaccination rates and ultimately improve public health. In response to this issue, the American Pharmacists Association (APhA) developed a certificate training program for pharmacists, Pharmacy-Based Immunization Delivery, over a decade ago.2 Importantly, the Center for Disease Control and Prevention’s (CDC) National Immunization Program has officially recognized APhA’s program. Many leaders in the profession agree that allowing pharmacists to administer immunizations has been one of the biggest movements towards improving direct patient care in recent years.3

With this clinical opportunity comes the need to determine the most appropriate method for delivering pharmacy-based immunization education to pharmacy students. The APhA certificate training program has been utilized at various pharmacy schools; however, few studies have assessed the impact of such a program on pharmacy students.4,5 A recent article by Welch et al. describes one approach to incorporating an immunization certificate training program into a pharmacy curriculum. The program was delivered over 3 weeks in the spring semester of the third professional year (P3) as part of a required Pharmacy Care Lab sequence. The study found that more than 44% of students provided positive feedback regarding the course with about 10% reporting that the course should not be mandated in the curriculum.5 This study is limited by the fact that it was completely qualitative in nature and did not extensively evaluate the program. Given this background, the objective of our study was to provide a comprehensive evaluation of an immunization training certificate program by determining its impact on the perceived knowledge, skills and attitudes of pharmacy students toward pharmacy-based immunizations.

METHODS

As a brief background, Butler University College of Pharmacy & Health Sciences pharmacy students complete 4 years of professional pharmacy academic training, with the final year (P4) comprised mostly of advanced pharmacy practice experiences (APPEs). However, no APPE rotations are scheduled in the month of December at which time the pharmacy students may enroll in the APhA Pharmacy-Based Immunization Delivery program. The immunization program consists of 20 hours of continuing education credit obtained via self-study (12 hours) and a live training session (8 hours).2 The live course seminar for this study was held at Butler University over 2 days in December 2007 and was taught by two APhA certified immunization trainers.

A self-administered, pre- and post- electronic survey was developed to meet the study objective. The pre-survey was distributed through e-mail to the students enrolled in the immunization elective prior to the live course segment and prior to receiving the self-study materials. Based on the results of the pre-survey, collaborative practice agreements was determined to be an unfamiliar topic for most students. Therefore, an additional lecture on collaborative practice agreements was developed and presented by the author during the live course seminar. After the live training session, the participants were provided the post-survey, which consisted of the same questions as the pre-survey as well as three additional questions assessing the overall program. Participation in the immunization elective course and this study was optional for all current P4 pharmacy students at Butler University, which included 160 students at the time of this study. The student certificate fee was paid by a University grant.

Survey Development and Completion

An electronic survey software program (Survey Monkey©) was used to develop and administer the survey. The pre-survey consisted of a cover letter and four sections: demographics, knowledge, skills and attitudes. The demographics section included age, gender, and previous injection experience, including prior subcutaneous and intramuscular injection instruction and/or hands-on experience. The knowledge section consisted of six items that assessed the student’s knowledge relating to immunizations. The skills section consisted of six items that assessed the skills of the student related to immunizations. The attitudes section consisted of six items that assessed the attitudes of the student toward pharmacy-based immunizations. Three additional questions were included with the post-survey to assess the student’s level of confidence in their ability to provide immunizations, to implement an immunization program, and an open-ended program feedback question. The participants were asked to indicate their level of agreement with the survey items in each section using a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree). A composite score for each survey section was derived by awarding points for each survey item reply and summing the total points for each section. Points were assigned as follows: 5 points (strongly agree), 4 points (agree), 3 points (neutral), 2 points (disagree), and 1 point (strongly disagree). Therefore, the maximum composite score for each section was 30 points (i.e., 5 points—strongly agree—for all 6 items = 30 points), and the minimum composite score for each section was 5 points (i.e., 1 point—strongly disagree—for all 6 items = 5 points).

Outcomes

The primary outcome was a change in the pharmacy students’ perceived knowledge, skills and attitudes toward pharmacy-based immunizations. Secondary outcomes included the perceived level of confidence the pharmacy students had toward providing immunizations and implementing immunization programs.

Statistical Analysis

Descriptive statistics were performed on the demographic data. Since data provided by Likert scales is ordinal and tends to be non-normally distributed, non-parametric tests should ideally be conducted for evaluation. However, all statistical findings were the same for both the parametric (t-test) and non-parametric (Mann-Whitney U test) tests; therefore, the results are reported based on parametric t-tests. All statistical analyses were performed using Statistical Package for Social Sciences version 13.0 (SPSS Inc., Chicago, IL). The study was approved by the Butler University Institutional Review Board.

RESULTS

Response Rate

A total of 57 students enrolled in the Immunization Elective out of 160 total students in the P4 class. The response rate for the pre-survey was 100% (57/57). The response rate for the post-survey was 94.7% (54/57).

Baseline Characteristics and Additional Post-survey Questions

The majority of participants in the Immunization Elective were female (Table 1). The mean age of the participants was 23.8 (SD=1.7) years. Overall, 89.5% of the participants had been instructed how to administer a subcutaneous injection to a patient while only 28.1% had actually administered subcutaneous injection. Prior to the immunization training program, 33.3% of the participants had been instructed how to administer an intramuscular injection to a patient while only 7.0% had actually administered an intramuscular injection. This prior experience was obtained primarily through a Pharmacy Care Lab earlier in the curriculum in which students voluntarily practiced subcutaneous injection technique during the diabetes care lab.

Table 1

Baseline Demographic Data (n=57)

Gender – n (%)
Male	15 (26.3%)
Female	42 (73.7%)
Age
Mean (SD)	23.8 (SD=1.7)
I have been instructed how to administer a subcutaneous injection to a patient. – n (%)
Yes	51 (89.5%)
No	6 (10.5%)
I have been instructed how to administer an intramuscular injection to a patient. – n (%)
Yes	19 (33.3%)
No	38 (66.7%)
I have administered a subcutaneous injection to a patient. – n (%)
Yes	16 (28.1%)
No	41 (71.9%)
I have administered an intramuscular injection to a patient. n – (%)
Yes	4 (7.0%)
No	53 (93.0%)

After completing the immunization certificate program, more than 98% of the participants felt confident (strongly agree or agree) in providing immunizations to patients (Table 2). Furthermore, about 85% of the participants felt confident (strongly agree or agree) in implementing an immunization program in their practice setting after their training. Most participants felt that the hands-on practice with the injection techniques and learning the CDC immunization schedules were the two most beneficial components of the certificate training program.

Table 2

Additional Post-survey Questions (n=54)

Upon completion of the immunization certificate program, I am confident about providing immunizations to patients. – % (n)
Strongly Agree	63.0% (34)
Agree	35.2% (19)
Neutral	1.9% (1)
Disagree	0% (0)
Strongly Disagree	0% (0)
Upon completion of the immunization certificate program, I am confident about implementing an immunization program in my practice setting. – n (%)
Strongly Agree	27.8% (15)
Agree	57.4% (31)
Neutral	9.3% (5)
Disagree	5.6% (3)
Strongly Disagree	0% (0)
Provide at least two things that were the most beneficial about completing the immunization certificate program.
Most common responses:
▪ Hands-on practice with injection techniques (subcutaneous/intramuscular) ▪ Learning the CDC immunization schedules

Knowledge

The primary outcome results can be seen in Table 3. The overall mean composite score in the knowledge section increased by 38.5% between the pre- and post-survey (18.47 SD=3.29 vs. 25.59 SD=2.58, respectively; p<0.001). All the items in the knowledge section showed statistically significant differences between pre- and post- scores (p<0.001 for all items; Table 4).

Table 3

Main Outcomes – Knowledge, Skills, and Attitudes

Outcome	Pre-Survey Mean (SD) (n=57)	Post-Survey Mean (SD) (n=54)	P-value
Knowledge	18.47 (3.29)	25.59 (2.58)	<0.001
Skills	19.33 (3.87)	26.01 (2.72)	<0.001
Attitudes	26.91 (2.01)	27.42 (2.27)	0.210

Table 4

Individual Items for Main Outcome Sections

Item	Mean Pre-survey Score + SD (n=57)	Mean Post-survey Score + SD (n=54)	P-value
Knowledge
I am knowledgeable on the current CDC guidelines for adult immunizations.	3.21 + 0.881	4.35 + 0.520	< 0.001
I am knowledgeable on the mechanisms of action and contraindications of vaccines.	3.23 + 0.824	4.24 + 0.612	< 0.001
I am knowledgeable on vaccine-preventable disease.	3.60 + 0.863	4.43 + 0.536	< 0.001
I am knowledgeable on biohazard waste disposal and blood born pathogen regulations.	3.91 + 0.931	4.48 + 0.540	< 0.001
I know how to obtain reimbursement for providing adult immunizations.	1.77 + 0.708	3.65 + 0.805	< 0.001
I know how to manage a vaccine adverse reaction.	2.75 + 1.005	4.44 + 0.538	< 0.001
Skills
I know how to communicate effectively with other health care professionals regarding immunization schedules.	3.56 + 0.824	4.41 + 0.533	< 0.001
I know how to communicate effectively with patients regarding immunization schedules and recommendations.	3.61 + 0.796	4.43 + 0.536	< 0.001
I know how to obtain informed consent from a patient.	3.91 + 0.830	4.50 + 0.541	< 0.001
I know how to prioritize patients based on their risk for contracting a vaccine-preventable disease (i.e. high-risk group vs. low-risk group)	3.53 + 0.889	4.57 + 0.536	< 0.001
I know how to create and implement immunization protocols.	2.39 + 0.901	4.11 + 0.604	< 0.001
I know how to establish collaborative practice agreements with community physicians.	2.33 + 0.913	4.00 + 0.727	< 0.001
Attitudes
Pharmacists are capable of safely and effectively administering immunizations with proper training.	4.77 + 0.535	4.91 + 0.351	0.116
I will look for future jobs that will allow me to administer immunizations.	4.12 + 0.683	4.20 + 0.711	0.542
The APhA immunization training program will prepare me to become an immunizer for the community.	4.56 + 0.501	4.76 + 0.473	0.035
Pharmacy students are capable of administering vaccines to patients with proper training.	4.30 + 0.680	4.39 + 0.656	0.477
Allowing pharmacists to administer vaccines will improve job satisfaction.	4.40 + 0.623	4.33 + 0.727	0.585
Allowing pharmacists to administer vaccines will improve public health.	4.75 + 0.474	4.83 + 0.423	0.357

Skills

The overall mean composite score in the skills section increased by 34.5% between the pre- and post-survey (19.33 SD=3.87 vs. 26.01 SD=2.72, respectively; p<0.001). All the items in the skills section showed statistically significant differences between pre- and post- scores (p<0.001 for all items; Table 4).

Attitudes

The overall mean composite score in the attitudes section increased by 1% between the pre- and post-survey (26.91 SD=2.01 vs. 27.42 SD=2.27; p=0.210). The only item in the attitudes section that showed a statistically significant difference between pre- and post- scores was the following statement: The APhA immunization training program will prepare me to become an immunizer for the community (p=0.035; Table 4).

DISCUSSION

Our study appears to be some of the only published evidence to date on the impact of an immunization training program on the perceived knowledge, skills and attitudes of pharmacy students toward pharmacy-based immunizations. The results of this study indicate that the national immunization training certificate program had a significant positive impact on the knowledge and skills of P4 pharmacy students. However, its impact on their attitudes was not significant. One possible explanation for this finding is that the students who enrolled in the immunization elective course were those students who already had positive attitudes toward immunizations. In fact, the overall mean score for the attitudes section was the highest of all three sections at baseline. Therefore, there was less room for improvement in this section because the students who enrolled in the class were those students who had positive attitudes toward pharmacy-based immunizations, representing a ceiling effect. Moreover, it is useful to see that the APhA training program had a positive impact on the knowledge and skills of students enrolled in the program.

Establishing collaborative practice agreements was an unfamiliar area for most students prior to the training program. In order to address this issue, a 20-minute lecture on collaborative practice agreements was developed and presented during the live training, which made a significant difference on this skill item. Specifically, the mean pre-survey score for this item was 2.33 (SD=0.913), and the mean post-survey score was 4.00 (SD=0.727) (p<0.001). The APhA certificate training program has been developed in such a way that allows for tailoring of the program to the individual audience’s needs. This individualized approach to teaching the certificate training program should remain in future sessions to help provide individualized and optimized education.

In looking at the recent study by Welch et al., they recommend incorporating immunization training into curricula based on the professional needs of the students, student scheduling, and faculty workload.5 They found success in placing the required course in the P3 year as part of a Pharmacy Care Lab while our study was an elective course offered during winter break of the P4 year. Since almost every participant felt confident in providing immunizations to patients after completing the training, the P4 year may be a logical time for immunization training due to its proximity to graduation. Under some state’s current legislation (including Indiana), pharmacy students are not allowed to provide immunizations; therefore, providing this education earlier in the curriculum would not be optimal in such states. Additionally, the temporal effects of retention of knowledge could potentially diminish such high confidence levels among the students. The elective nature of the course is also debatable. The immunization certification being optional is advantageous to students who know they will not be in a pharmacist position that allows them to administer immunizations and can, therefore, opt out of taking the course. However, making the course required would increase the number of potential immunizing pharmacists, which would ultimately help improve public health. Also, offering the training over a 2-day period allowed students a focused and intensive period of time to complete the training. In this way, the program did not interfere with any of the semester curricular workload. Other methods of implementation should certainly be evaluated and assessed at other Colleges/Schools of Pharmacy.

In order to help achieve meet the American Association of Colleges of Pharmacy Center for the Advancement of Pharmaceutical Education (CAPE) 2004 Educational Outcomes regarding public health and to meet the current and future demands of patients, immunization education has a clear place in the pharmacy curriculum.6 Therefore, allocating appropriate funding should be considered a top priority for pharmacy schools to help meet this objective and ultimately improve public health. Finally, at a time when an influenza pandemic is a major public health concern, it is promising to know that pharmacists can play an active role in helping combat this potential crisis and other future crises. Increasing the number of certified pharmacist immunizers will certainly place pharmacists in a position to aid in such efforts.

There are limitations to this study which deserve mention. First, because the study was a self-administered survey, the data were self-reported. It should be noted that previous research has shown a mismatch between pharmacy student’s ability to self-assess as compared to external assessors (i.e., pharmacists) in the following global rating scale areas of patient interaction: verbal communication, nonverbal communication, degree of focus, logic, coherence in the interview, empathy, and overall performance. The differences were the largest in the areas of empathy, and logic/focus/coherence of interviewing.7 Conversely, research has shown that pharmacy student self-assessments were similar to faculty ratings in the context of a pre- and post-survey for a drug information APPE.8 Because our study relied on student self-assessment, there is the potential for inaccuracy in their actual knowledge and skills relating to immunizations. However, all of the students passed the final exam and, thus, passed the course. Regardless, pharmacy student self-assessment should be an area of future research as it is a critical skill for career and personal development. Also, although the survey instrument was constructed with the assistance of a pharmacy practice and a social and administrative pharmacy professor, this measurement instrument did not undergo rigorous reliability and validity testing prior to being used in this study. Therefore, the items used to assess the knowledge, skills and attitudes may not have truly captured the entire universe of the construct that was being measured. Furthermore, a common limitation with survey instruments is the potential for misinterpretation of survey items. Specifically, items in the knowledge and skills sections may have some overlap; however, particular attention was paid to this issue in the development of the survey to minimize any potential confusion. A potential threat to internal validity was that the lecture on collaborative practice agreements was developed and presented as a result of the low mean score on the pre-test. By directly incorporating this component into the training program, the internal validity of the study was threatened; but, this potential threat is difficult to avoid when using an intervention such as a training program. Since approximately 36% (i.e., 57 of 160) of the P4 class enrolled in the elective program, the potential for selection bias does exist in that those students already knowledgeable on immunizations may have been less likely to enroll in the course, leading to a more impactful group of students enrolling in the course. Finally, it is not known whether the results of this study are generalizable to other Colleges/Schools of Pharmacy.

CONCLUSIONS

A positive impact on the perceived knowledge and skills of P4 professional pharmacy students was observed with the implementation of a national immunization training certificate program. However, the perceived attitudes of the pharmacy students did not significantly change in this study. Increasing the number of certified immunizing pharmacists is essential to meet the current and future demand for increasing vaccination rates. As the scope of pharmacy practice continues to expand and more legislation is approved for pharmacists to provide immunizations, ongoing assessment is necessary to identify the most effective instructional methods for the delivery of immunization education for pharmacy students.