Immunization training for pharmacy students: a student-centered evaluation.

BACKGROUND: Persistent and emerging public health challenges mean Pharmacy students require training in immunization services. Curtin University, Australia, integrated blended-delivery immunization training into the final-year Bachelor of Pharmacy (Hons) and graduate-entry Master of Pharmacy curricula in 2019 and 2020, utilizing materials licenced from the Pharmaceutical Society of Australia.
OBJECTIVE: This study evaluated changes in students' attitude, confidence, self-perceived knowledge and self-perceived skills pre- and post-training delivered in 2020.
METHODS: Pre- and post-training questionnaires featured 42 opinion statements grouped under headings 'Attitudes', 'Confidence', 'Self-Perceived Knowledge' and 'Self-Perceived Skills', and answered using five-point Likert scales (1 = strongly disagree to 5 = strongly agree). Completed pre- and post-training questionnaires were matched using respondent-generated codes. Data were subjected to descriptive and multivariate regression analysis to test pre-post changes and associations and changes in mean scores.
RESULTS: 128 (95.5%) and 132 (98.5%) students completed the pre- and post-training questionnaires, respectively. Immunization training resulted in significant (p<0.05) improvement in students' mean Confidence score (3.33 vs 3.96), Self-Perceived Knowledge score (3.08 vs 4.47) and Self-Perceived Skills score (2.81 vs 4.55). Improvement in students' mean Attitude score was also statistically significant (4.45 vs 4.61), yet more positively skewed pre-training. No significant pre-post differences were found between the Bachelor and Master students. Post-training, all respondents agreed that the training program increased their attitude, confidence, perceived knowledge and perceived skills, rating the training experience as either Excellent (91.6%) or Good (8.4%).
CONCLUSIONS: Immunization training integrated into final-year BPharm (Hons) and MPharm curricula improved Attitudes, Confidence, Self-Perceived Knowledge and Self-Perceived Skills, all key to further role development in public health. This method is recommended to other Pharmacy schools to determine the impact and acceptability of immunization training programs amongst students. Copyright:

INTRODUCTION

Despite robust evidence supporting immunization, underutilization of immunization programs remains a public health concern.1,2 Multifaceted efforts have included increasing the immunization workforce capacity, and pharmacists are well placed to contribute.2,3 Immunization was recognized as within the scope of practice for pharmacists by the Pharmacy Board of Australia in December 2013, with the state of Western Australia being the first Australian jurisdiction to legislate pharmacist-administered (injected) influenza vaccinations for adults.4 The Pharmaceutical Society of Australia and the Pharmacy Guild of Australia subsequently introduced training programs for registered pharmacists, approved by the state Department of Health.5-7 An evaluation of Western Australian pharmacist-led immunization services in 2015 demonstrated successful implementation and delivery.6 Evidence of positive consumer experience, safety and impact of pharmacist-administered influenza immunization has emerged over six influenza seasons, resulting in expansion of scope for pharmacist-administered vaccinations.5,6,8-11 Further to this, pharmacists are integral to the immunization workforce against SARS-CoV-2 (COVID-19).12 The COVID-19 pandemic highlighted pharmacists’ crucial role during a public health crisis, and their inclusion in the deployment of COVID-19 vaccines is expected to help achieve immunization targets in Australia.13

In Australia, the National Immunisation Education Framework for Health Professionals (National Framework) specifies standards for immunization training for health professionals.14 The Australian Pharmacy Council, responsible for accreditation of Pharmacy degree programs, published Standards for the Accreditation of Programs to Support Pharmacist Administration of Vaccines, recognizing the need for immunization education for pharmacy students.15 To date, three studies have evaluated pharmacy student immunization programs in Australia. In 2014, Hope et al. reported positive self-evaluations from 29 Master of Pharmacy students after completion of vaccination training in a blended-learning mode and incorporating the Australasian Society of Clinical Immunology and Allergy anaphylaxis training.16 Mean agreement about improved knowledge and confidence ranged from 4.52 to 4.83 out of 5 across the various training components. Carroll et al. evaluated the experience of pharmacy interns from The University of Sydney who completed Pharmacy Guild of Australia vaccination training immediately after gaining provisional registration.17 The authors reported significantly improved confidence, skills and knowledge of influenza immunization amongst 57 respondents post-training, and recommended embedding of vaccination training in Pharmacy curricula.17 Subsequently, Bushell et al. introduced and evaluated a bespoke immunization training program for pharmacy students at the University of Canberra, reporting significant pre-post improvements in mean knowledge scores of 52 respondents, along with positive feedback about value of the training.18 However, these studies did not report matched pre-post responses, and were relatively broad in scope and analysis. An international literature review confirmed the willingness of pharmacy students to become immunizers, with an estimated 90% of American Pharmacy Schools providing immunization training by 2019.19-21 Similar to the extant Australian research, international studies were either limited to post-training evaluation or did not enable matching of pre- vs post-training responses, and therefore, intra-student change.20,22,23 In one study, participants were assigned an indirect identifier to monitor changes in students’ confidence, which improved over various time points.24 Some studies reported data from knowledge tests about immunization; while outcomes were positive, the reliability and validity of test answers cannot be assured in an anonymous survey.20,22

To equip students to meet contemporary roles, Curtin University, Australia, introduced immunization training in 2019 to students enrolled in their final year in the Bachelor of Pharmacy (Honours) (BPharm (Hons)) and graduate-entry Master of Pharmacy (MPharm) programs. The MPharm is a two-year intensive professional qualification requiring a Bachelor qualification (typically in Biomedical Sciences) and completion of foundation science pre-requisites. Unlike an American study comparing the timing of immunization training in the first vs third year of a Pharmacy program, our immunization training was deliberately provided in the final year of Pharmacy studies for currency of knowledge and skills upon graduation.25 The training utilized the Western Australian Department of Health-approved materials developed by the Pharmaceutical Society of Australia. The blended delivery involved students undertaking an online module (six to eight hours) and passing a pre-learning knowledge assessment, followed by a one-day face-to-face workshop and demonstrating competence in a practical assessment. This training was a must-pass component of a relevant unit of study. Immunization concepts, including communication and service implementation, were also embedded in relevant units of study to facilitate consolidation and application of knowledge. The introduction of our training in 2019 enabled logistical refinement of its delivery in 2020. Western Australian legislation does not prohibit pharmacy students from undergoing immunization training, although it does not allow students to administer (inject) vaccines. As such, students who complete immunization training can apply for recognition of prior learning from the Pharmaceutical Society of Australia upon graduation to obtain a Statement of Attainment, enabling administration of vaccines during their internship under direct supervision of a pharmacist immunizer.5 Understanding and evidencing students’ experience with learning activities is key to ongoing improvement in curriculum design; therefore, it is valuable to undertake student-centered evaluation of the integration of immunization training into the pharmacy curricula.

This paper aims to evaluate the attitudes, confidence, self-perceived knowledge and self-perceived skills of Curtin University pharmacy students before and after completing immunization training in 2020.

METHODS

This was an observational study involving BPharm (Hons) and MPharm students undertaking immunization training as part of their final-year curriculum. The study was approved by the Human Research Ethics Committee of Curtin University (HRE2019-0804).

Pre-training questionnaires were self-completed in hard-copy format during orientation sessions for the respective units of study, whilst post-training questionnaires, also in hard copy, were administered immediately after completion of the face-to-face practical immunization workshop and assessment (Figure 1). This was one to eight and eight weeks following the orientation session for the BPharm (Hons) and MPharm students, respectively. Students were verbally briefed and provided a Participant Information Sheet. Return of the questionnaire implied voluntary consent to participate. Students were reminded that participation would have no influence on their assessment in the unit of study.

Figure 1

Schematic representation of the study design

The questionnaires were drafted and modified with reference to Australian and international research, and modified based on the APC Standards for the Accreditation of Programs to Support Pharmacist Administration of Vaccines.15,19,26-28 As there is no nationally consistent approach to pharmacist-administered immunizations in Australia, to encompass the legislation surrounding immunization specific to Western Australia, the questionnaires were purposively designed. The draft questionnaires were refined following review by three academic pharmacists for content and face validity.

The pre-training questionnaire comprised:

Part 1: Student information, including instructions for generating a participant code that could not be linked to the student’s enrolment information, to anonymously facilitate matching of each participant’s pre- and post-training questionnaires.

Part 2: 42 opinion statements relating to attitudes, confidence, and perceived knowledge and skills, answered using five-point Likert scales from 1 (strongly disagree) to 5 (strongly agree), and grouped under the headings ‘Attitudes’, ‘Confidence’, ‘Self-Perceived Knowledge’ and ‘Self-Perceived Skills’.

The post-training questionnaire comprised:

Part 1: Student information and the participant-generated code to be regenerated for linkage to the pre-training questionnaire.

Part 2: Repeat of the 42 opinion statements from the pre-training questionnaire.

Part 3: General feedback on the training experienced, answered using five-point Likert scales.

The data were analyzed using IBM SPSS version 26 software and SAS version 9.4. For the purposes of descriptive and pre-post analysis, data from the BPharm (Hons) and MPharm cohorts were merged, given that the structure of the training was identical. Demographic data were expected to differ between these cohorts. Opinion data were tested for normality using the Kolmogorov-Smirnov and Shapiro-Wilk statistics. Intra-cohort differences (before and after training) and inter-cohort differences (BPharm (Hons) and MPharm) were analyzed using parametric tests (paired Student’s t-tests) or non-parametric tests (Wilcoxon signed-rank tests) as appropriate. To avoid habitual responses, it is recommended that questions shift between positively and negatively worded statements, so three opinion statements in the questionnaire were deliberately negatively worded, and their scores reversed for the analysis.28 The responses to each opinion statement were summarised as medians, while an overall score for each section of opinion statements was generated as a mean of scores. Changes from pre- to post-training of these mean scores were computed and compared using Student’s t-test. Multivariate regression analysis evaluated associations between participant characteristics (enrolled program, gender, age and pharmacy work experience) and their mean Attitude, Confidence, Self-Perceived Knowledge and Self-Perceived Skills scores in the pre- and post-training questionnaires. Further multivariate regression analyzed associations between participant characteristics and the mean change in scores (post-training minus pre-training) for each section of opinion statements between pre- and post-training questionnaires. A p-value of <0.05 indicated a statistically significant association in all tests.

RESULTS

In 2020, 134 students were enrolled in the final year of the BPharm (Hons) (n=103) and MPharm (n=31) programs. Of these, 128 (95.5%) students completed the pre-training questionnaire, and 132 students (98.5%) completed the post-training questionnaire (Figure 2). Pre- and post-training responses from 122 participants (91 BPharm (Hons); 31 MPharm) were able to be matched based on the participant-generated codes, and were therefore included in the comparative analyses (Figure 2). The 10 respondents to the post-training questionnaire who did not complete the pre-training questionnaire were all BPharm (Hons) students.

Figure 2

Study response rates

From the combined cohorts, 92.4% (122/132) of post-training questionnaires could be paired with the respective pre-training questionnaires according to the participant-generated code, and this was similar for students in both programs, and for both genders (Table 1). The reported ages of the participants ranged from 20-59 years, with a mean of 24.5 (SD=4.9) years and no significant difference between pre- and post-training cohorts (p=0.509). While the six non-respondents to the pre-training questionnaire were, on average, slightly younger than the respondents (mean: 23.3 vs 24.5 years), largely because the non-respondents were all from the BPharm (Hons) program, this difference was not significant (p=0.510). Four in five (80.5%, 103/128) students indicated they worked or volunteered in a pharmacy at the time of completing the pre-training questionnaire. Of these, 21 students had been involved in assisting the delivery of immunization services (in capacities other than being the administrator), three students had been trained in injection techniques, and 10 students had experience in administering injections to themselves or another person.

Table 1

Demographic characteristics of study participants

n (%)	Pre-training N = 128	Post-training N = 132
Program
BPharm (Hons)[a]	97 (75.8)	101 (76.5)
MPharm[b]	31 (24.2)	31 (23.5)
Gender[c,d]
Male	39 (31.1)	40 (30.3)
Female	87 (69.0)	88 (66.7)

Bachelor of Pharmacy (Hons);

graduate-entry Master of Pharmacy

Non-response: n=2 (pre-training), n=4 (post-training)

No significant difference (p=0.504)

Review of the data distribution indicated that non-parametric analysis was appropriate for the 42 opinion statements, as data were positively skewed. Reliability analysis of the 42 opinion statements produced Cronbach’s alpha coefficients of 0.93 for the pre-training questionnaire and 0.91 for the post-training questionnaire. Comparison of the mean pre- and post-training opinion scores suggested students had a more positive perception of their confidence, knowledge and skills had improved after completing the immunization training (Table 2). However, means relating to the attitudinal statements were already high in the pre-training questionnaire. Notwithstanding the three negatively worded statements to discourage habitual responses, no students disagreed or strongly disagreed to any of the statements in relation to their attitudes, confidence, knowledge or skills.

Table 2

Mean (SD) score for each section of opinion statements in the pre- and post-training questionnaire

Section	Pre-training	Post-training	p-value[a]
Attitude	4.45 (0.45)	4.61 (0.34)	<0.001*
Confidence	3.33 (0.88)	3.96 (0.54)	<0.001*
Self-Perceived Knowledge	3.08 (0.75)	4.47 (0.49)	<0.001*
Self-Perceived Skills	2.81 (0.81)	4.55 (0.46)	<0.001*

p-values obtained from the paired samples t-test

Consistent with the findings reported in the , comparison of the mean scores for each section of opinion statements revealed a statistically significant improvement between pre- and post-training for all four domains (p=0.001) (Table 2). The mean Attitude scores pre- and post-training, whilst found to be significantly different, were more positively skewed, with a mean change between pre- and post-training of 0.161 (p=0.001).

Click here for additional data file.

In the regression analyses testing the association between student characteristics and mean scores for each section of opinion statements, the only significant association in the analysis of pre- and post-training responses was students’ work experience in a pharmacy, associated with higher mean scores for Self-Perceived Skills before and after the training (p=0.011, p=0.009). The mean Self-Perceived Skills score pre-training was 2.96 for those with work experience, compared with 2.46 for those without, and this difference was statistically significant (p=0.017) (Table 3). Gender was also associated with improved post-training Attitude scores (p=0.036); the mean score for Attitude pre- to post-training increased from 4.43 to 4.56 respectively for males, and from 4.31 to 4.64 respectively for females (p=0.008).

Table 3

Association between demographic variables and changes in mean domain scores

Demographic Variable	Attitude		Confidence		Self-perceived knowledge		Self-Perceived Skills
	Pre-training	Mean change[a]	Pre-training	Mean change[a]	Pre-training	Mean change[a]	Pre-training	Mean change[a]
Program
BPharm (Hons)	4.36	0.18	2.35	1.45	3.04	1.34	2.66	1.81
MPharm	4.37	0.28	2.22	1.61	3.03	1.26	2.76	1.80
Gender
Male	4.43	0.13	2.41	1.39	3.12	1.24	2.83	1.74
Female	4.31	0.33[b]	2.16	1.67	2.96	1.36	2.60	1.87
Age
≤22 years	4.37	0.28	2.21	1.60	2.88	1.41	2.79	1.70
23-25 years	4.27	0.27	2.29	1.49	3.12	1.19	2.63	1.83
>25 years	4.46	0.15	2.37	1.50	3.11	1.30	2.72	1.88
Work Experience
Yes	4.47	0.20	2.35	1.61	3.16	1.38	2.96	1.72
No	4.25	0.26	2.22	1.45	2.92	1.22	2.46[c]	1.89

Post-training minus pre-training (five-point scale); positive number indicates improvement

Significant improvement in Attitude post-training amongst female students (p=0.008)

Significant difference in Self-Perceived Skills pre-training between students with and without work experience (p=0.017)

At completion of the training, all respondents (n=132) rated the training either as ‘Excellent’ (91.6%) or ‘Good’ (8.4%). Post-training, 96.2% of students (n=127) either strongly agreed or agreed that they would be confident operating an immunization service and administering injections. Furthermore, 98.5% of students (n=130) either strongly agreed or agreed that they had increased knowledge about administration of vaccines, and 97.7% of students (n=129) either strongly agreed or agreed to statements relating to their self-perceived skills to immunize post-training.

DISCUSSION

In this study, the most impactful changes following integrated immunization training were in students’ confidence, self-perceived knowledge and self-perceived skills. Attitudes towards immunization were already high prior to training, and this may have confounded the overwhelmingly positive feedback about the learning opportunity. Our two cohorts of final-year Pharmacy students had been exposed to evidence supporting immunization throughout the entirety of their program, with cognizance of their duty of care as health professionals and awareness of the expanding role of pharmacists administering vaccinations. As such, it is unsurprising that the mean Attitude score was positively skewed pre-training.

The findings support the argument that formal integration of immunization training as part of an accredited Pharmacy degree is feasible and effective. This recommendation is in line with findings reported in numerous Australian and international studies in the past decade, all varying in design.16-18,21-26 Our study is the first to analyze intra-student changes in four domains – Confidence, Self-Perceived Knowledge, Self-Perceived Skills and Attitudes – and we recommend our use of a respondent-generated, replicable code to enable matching of pre- and post-training questionnaires while retaining anonymity. Although the training curricula in previous Australian studies differed, they all aligned with the National Framework, which may explain this general consistency and build on the national and international evidence that students can administer vaccines safely.14,17,18,21,29

There were no significant differences in any of the four domains in the pre- or post-questionnaire between BPharm (Hons) and MPharm students, suggesting the training received in each program was consistent. Whilst it was not possible to prevent sharing of experiences between BPharm (Hons) and MPharm students yet to undergo the training, the questionnaire was designed to elicit individuals’ reflections and experiences, rather than being a knowledge or skills examination that may benefit from prior insight into the content.

Despite students being reminded that completion of the questionnaires was voluntary, response rates to both questionnaires were high (95.5% and 98.5%), strengthening the credibility and representativeness of the study findings. Reversal of the three negatively worded statements revealed responses consistent with the positively worded statements. Hence, it can be concluded that students conscientiously completed the questionnaires. Bias from recall of pre-training responses was unlikely due to the spacing of one to eight weeks between the pre- and post-training questionnaires and minimal divergence in responses between cohorts who completed their training at different times.

Students with work experience in pharmacy had higher mean scores for Self-Perceived Skills pre- and post-training compared to those who had not worked in a pharmacy. Opportunities for students to observe and assist registered pharmacists in their provision of immunization services may explain this difference, and this finding could be used to enhance clinical placement activities during pharmacy studies. The association between gender and change in pre- and post-training Attitude scores, although significant, does not appear to have educational relevance for our future delivery of immunization training. Studies suggest that confidence is mirrored through attitude.30,31 Given that no other mean change between the categories pre- and post-training, particularly Confidence, was associated with gender, there is no suggestion that the training had a greater influence on the attitude of females. No other statistically significant association was determined from the regression analyses; therefore, the training provided was applicable to all students. The findings suggest that the existing training program and delivery were appropriate to meet the learning needs of students with varying demographics and experience; this could be confirmed through prospective monitoring of these cohorts of students upon graduation if they become pharmacist immunizers. Future studies could also examine the potential role of students who had worked in pharmacy to be peer mentors of others, considering their likelihood to have higher Self-Perceived Skills scores.

Other countries (for example, United States of America) have incorporated immunization training into pharmacy curricula for over 10 years, and many states now permit training to allow students to administer vaccines to adults in the community during clinical placement.21,29 Bushell et al. highlighted that similar jurisdictional change should be made in Australia to allow student-administered immunizations.18 Our pharmacy students had demonstrated the skills competencies specified in the Pharmaceutical Society of Australia training materials by virtue of the formal skills assessment in the training workshops, similar to that of a registered pharmacist undertaking the Pharmaceutical Society of Australia immunization program. The Australian Government is preparing the healthcare system to deploy one of the largest vaccination schemes to date, in which community pharmacies are under consideration to be a major provider for the COVID-19 vaccine.12,32 The inclusion of pharmacy students in the delivery of such services would be invaluable and further increase the immunization workforce capacity.17 No Australian jurisdictional legislation allows pharmacy students to inject vaccines, limiting this responsibility to registered, certified pharmacists; however, our study suggests students may contribute to the implementation and operation of immunization services via cold chain monitoring, directing immunization clinic flow, managing appointments, obtaining informed consent from patients, and assisting in data entry, counselling and other clerical duties.32 The topics covered in the Pharmaceutical Society of Australia training – including communication about immunization, service implementation and delivery, patient screening and legal boundaries – are relevant to holistic immunization service implementation. Given the logistical complexity of immunization services, future studies could evaluate the role and contribution of pharmacy students in strengthening the workforce capacity of pharmacist immunizers, and how their attitudes and attributes attained through immunization training can contribute to the implementation of an immunization service in a pharmacy setting.

One limitation of this study is that students’ experiences and views were constrained to quantitative data; therefore, an explanation behind the reasoning of students’ responses was not obtained. The other Australian studies collected qualitative data via follow-up telephone interviews or free-text survey responses.16-18 A mixed-methods quantitative stage followed by timely in-depth qualitative exploration of findings would be ideal. Our study was a point-in-time survey; hence data on the rates of students who become vaccinating pharmacists once registered to practice is unknown. This highlights the need for future research to determine the long-term impact of curriculum-integrated immunization training on pharmacists’ attitude, confidence, knowledge and skills when qualifying as a pharmacist immunizer. This could include a longitudinal design, monitoring future iterations of the current training and graduates’ immunization services in practice.

CONCLUSIONS

To prepare students for the expanding scope of pharmacy practice in Australia and internationally, students require knowledge, practical immunization skills and effective communication skills surrounding immunization. In addition, students should be cognizant of the principles of service implementation, so they can assist the implementation and delivery of immunization services and contribute towards the overall immunization workforce, especially amid the COVID-19 pandemic. Our immunization training integrated into the curricula of final-year BPharm (Hons) and MPharm programs increased students’ attitude, confidence, self-perceived knowledge and self-perceived skills. This study contributes to the emerging evidence around how pharmacy students can acquire knowledge, skills and attitudes to provide immunization services upon registration as pharmacists and attainment of appropriate certification. Our method is recommended to other Pharmacy schools to determine the impact and acceptability of immunization training programs amongst students.