Pharmacy students' knowledge and confidence of COVID-19 following an interactive didactic class.

Background: COVID-19 education for the pharmacy workforce is important to ensure pharmacists are optimizing patient care for the prevention and management of COVID-19. However, there are currently no reports to our knowledge of education and training experiences for COVID-19 prevention and management in the Doctor of Pharmacy (PharmD) curricula. Objective: To evaluate pharmacy students' knowledge and confidence regarding COVID-19 prevention and management before and after an interactive didactic class (IDC).
Methods: A multicenter, quasi-experimental, cross-sectional survey study was performed among pharmacy students before and after IDC on COVID-19 at two schools of pharmacy. The IDC on COVID-19 consisted of student-led presentations on a COVID-19 drug, an infectious disease pharmacist faculty-led interactive lecture on COVID-19 prevention and management, and clinical case vignettes to assess COVID-19 management strategies. An anonymous, voluntary, electronic survey was distributed to students (n = 85) before and after. The pre- and postintervention survey contained 10 COVID-19 knowledge-based questions and multi-step, 5-point Likert scale statements related to COVID-19 prevention and management confidence. The postintervention survey also evaluated students' perceptions of the COVID-19 IDC. Descriptive statistics were performed, and Student t test was used to compare pre- and postintervention responses.
Results: About 61 surveys were completed resulting in a response rate of 72%. COVID-19 knowledge scores (mean ± SD) increased overall following the IDC (5.9 ± 1.31 vs 8.6 ± 1.29). Pharmacy students' COVID-19 confidence scores (mean ± SD) also improved following the IDC (2.66 ± 0.75 vs 4.03 ± 0.53). Students performed well on the COVID-19 clinical case vignettes with a mean ± SD score of 22.41 ± 0.46 out of 25. Pharmacy students' perceptions of the IDC on COVID-19 were also positive overall.
Conclusion: Pharmacy students' knowledge and confidence of COVID-19 prevention and management improved following an IDC. This may be an effective strategy to provide COVID-19 education during the PharmD curricula.

INTRODUCTION

The COVID‐19 pandemic has impacted nearly all facets of life worldwide. Pharmacy education has been disrupted considerably with rapid changes in the approach, delivery, and assessment in both didactic and experiential education. , , , , , , Despite this, the role of pharmacists has increasingly expanded to further leverage the knowledge and skills of the pharmacy workforce during the COVID‐19 pandemic. , , , Pharmacists are on the front lines of the COVID‐19 pandemic in both the prevention and management of COVID‐19 in outpatient and inpatient settings. , Legislation has also expanded to further promote and allow pharmacists and pharmacy interns to administer COVID‐19 vaccines to adults and children. , , Furthermore, the Department of Health and Human Services added an amendment to the COVID‐19 Public Readiness and Emergency Preparedness Act allowing pharmacists who meet the criteria to order and administer select COVID‐19 therapeutics, including subcutaneous monoclonal antibodies.

Given the significant impact, pharmacists can have in the prevention and management of COVID‐19, appropriate education for pharmacy students during the Doctor of Pharmacy (PharmD) curricula is essential. However, there are currently no reports to our knowledge that describe education and training best practices for the prevention and management of COVID‐19 within the PharmD curricula. Historically, the use of active learning methods compared to lecture‐based approaches during didactic coursework has previously demonstrated improved retention of core content during advanced pharmacy practice experiences and potentially beyond. Pharmacy students' performance and confidence have also been shown to improve when applying literature evaluation skills with journal clubs to clinical patient case vignettes, which is particularly valuable when considering the rapidly emerging and evolving data for COVID‐19 prevention and management. Previously, implementation of an interactive didactic class (IDC) using active learning techniques and hands‐on approaches has previously shown to also improve pharmacy students' knowledge and confidence for penicillin allergy assessment.

OBJECTIVE

Given the positive educational impact previously observed with an IDC format, this study sought to evaluate pharmacy students' knowledge and confidence regarding COVID‐19 prevention and management before and after an IDC using active learning techniques and literature evaluation skills.

METHODS

This was a multicenter, quasi‐experimental, cross‐sectional survey study among pharmacy students before and after an IDC on COVID‐19 at Binghamton University School of Pharmacy and Pharmaceutical Sciences in Binghamton, New York, and Saint John Fisher College Wegmans School of Pharmacy in Rochester, New York. Pharmacy students were included in survey distribution if they attended and participated in the COVID‐19 IDC. Pharmacy students were excluded if they did not complete or had an incomplete pre‐ and/or postsurvey response.

This study was deemed exempt by the Institutional Review Boards at Binghamton University and Saint John Fisher College. A survey invitation letter with appropriate informed consent information was attached to the survey instrument and distributed electronically via email. By selecting to enter the survey, the respondent agreed to participate.

The 3‐h, COVID‐19 IDC occurred during February 2021 of the Spring semester for third professional year pharmacy students at both schools of pharmacy (n = 85). The IDC was part of an infectious disease (ID) elective course at Binghamton University School of Pharmacy and Pharmaceutical Sciences and as part of a required skills lab course at Saint John Fisher College Wegmans School of Pharmacy to have consistent delivery to only third‐year pharmacy students. Given the status of the COVID‐19 pandemic at this time, the IDC was delivered virtually, and breakout rooms were utilized for group work. COVID‐19 education was not delivered to pharmacy students as part of the PharmD curricula before the IDC; however, general literature evaluation skills were introduced as part of a drug information course before the IDC. The ID pharmacist faculty members delivering the IDC had ≥5 years of ID clinical practice experience, completed a postgraduate year two ID pharmacy residency program, and are board‐certified infectious diseases pharmacists (BCIDP) by the Board of Pharmacy Specialties.

Student learning outcomes for the IDC included: (a) describe the various COVID‐19 drugs including drug class and mechanism of action, (b) identify the appropriate place in therapy for COVID‐19 drugs, and (c) Explain the laboratory tests used to monitor both disease progression and drug therapy. Before the IDC, pharmacy students were arranged into groups and were randomly assigned a specific COVID‐19 drug to review using primary literature to further develop their literature evaluation skills. Pertinent journal articles were assigned for student groups to review and prepare a journal club activity to review the COVID‐19 drug and associated data to support or not support its use in the prevention or management of COVID‐19. A journal club template was created to include the following: the COVID‐19 drug's mechanism of action and potential notable adverse effects, study design, study population, methodology, results, limitations, and application to clinical practice. The student groups were also asked to identify the place or role in therapy for COVID‐19. Selected COVID‐19 drugs with accompanying primary literature for review included remdesivir, dexamethasone, baricitinib, tocilizumab, bamlanivimab, and hydroxychloroquine. , , , , ,

At the beginning of the IDC, each student group was allotted 10 min to present their journal club summary to the class and students were encouraged to ask questions. The ID pharmacist faculty also presented a one‐hour interactive lecture on COVID‐19 including COVID‐19 microbiology, epidemiology, pathophysiology, clinical presentation, diagnosis, treatment, and prevention. Prevention included general prevention measures (eg, handwashing, face coverings, and social distancing), but was primarily focused on the available COVID‐19 vaccine schedule recommendations as well as their efficacy and safety data. Instruction also reviewed awareness and usability of essential COVID‐19 management recommendations from key resources including the National Institutes of Health (NIH) and Infectious Diseases Society of America (IDSA) COVID‐19 Treatment Guidelines. , The ID pharmacist faculty provided a disclaimer that these recommendations may be eventually outdated based on emerging data, but current recommendations can be reviewed in these resources to provide updated management recommendations. The students were also instructed that the NIH and IDSA both provide guidelines with largely similar recommendations overall, but differences in recommendations may still exist. Real‐time question polling was utilized to reinforce key concepts as part of an active learning exercise and formative assessment.

ID pharmacist faculty used the same presentation materials at both schools of pharmacy to optimize consistency in content delivery. After this, students were arranged into their groups and tasked with completing five COVID‐19 patient case vignettes for a graded activity to reinforce content delivery and assess clinical application skills. Each patient case vignette represented a different clinical scenario and students were asked to provide appropriate prevention and/or management recommendations with supporting rationale based on primary literature and guidelines. , Once student groups submitted their case activity, the ID pharmacist faculty led a class debrief to review the correct answers and rationale in addition to holding a question‐and‐answer session to conclude.

A 20‐item preintervention survey and a 25‐item postintervention survey were designed to evaluate pharmacy students' knowledge and confidence of COVID‐19 prevention and management before and after the IDC. Survey questions aligned with IDC's learning outcomes. The survey was anonymous, and students were not required to complete this, but participation was encouraged. The survey was developed using the expert opinion of the ID pharmacist faculty members. Qualtrics (Qualtrics, Inc., Provo, UT) was used to design and collect survey responses.

The preintervention survey included 10 knowledge‐based COVID‐19 prevention and management questions including, but not limited to appropriate patient populations for COVID‐19 drugs, COVID‐19 vaccine data, and potential adverse effects of various COVID‐19 drugs. Survey questions aligned with the IDC's learning outcomes. The preintervention survey also included 10 confidence statements for various aspects of COVID‐19 prevention and management measured on a 1 (strongly disagree) to 5 (strongly agree) scale. The postintervention survey included the same knowledge questions and confidence statements as those on the preintervention survey as well as five additional items that measured students' perceptions of the COVID‐19 IDC, also measured on a 1 (strongly disagree) to 5 (strongly agree) scale. Reliability analysis of the pre‐ and postintervention scale score demonstrated that the scale had excellent internal consistency with a Cronbach's α value of 0.92 and 0.93, respectively.

The survey instrument was pilot tested by an ID pharmacist and ID pharmacy resident. Feedback was then incorporated before dissemination to ensure validity, logistical integrity, and question clarity. The preintervention survey was distributed electronically via email to pharmacy students 48 h before the start of the IDC and subsequently closed before instruction began. The postintervention survey was distributed electronically via email at the conclusion of the IDC and was left open for 48 h after. No reminder emails were sent given the relatively short time the survey remained open.

Survey responses were collected via Qualtrics survey software and statistical analyses were performed using R (R Foundation for Statistical Computing, Vienna, Austria). Survey responses were matched based on unique, anonymous survey codes to align pre‐ and postsurvey responses. Descriptive statistics were performed to characterize survey responses. Paired‐sample t tests with Bonferroni corrected P values were also used to compare pre‐ to postintervention means of the knowledge and confidence items. Although these scores were not normally distributed, the t test is generally robust to violations of the normality assumption when sample sizes and variances are similar in the pre‐ and postintervention data, which was consistent in this study. Furthermore, use of an equivalent rank‐based test yielded similar results. All statistical tests were two‐tailed, and a P value less than 0.05 was considered to indicate a statistically significant difference.

RESULTS

In total, 61 out of 85 pharmacy students completed both the pre‐ and postsurveys resulting in a response rate of 72%. Thus, 24 out of 85 pharmacy students were excluded because of incomplete or unsubmitted survey responses. About 28 (97%) of 29 and 33 (59%) of 56 pharmacy students responded from Binghamton University School of Pharmacy and Pharmaceutical Sciences and Saint John Fisher College Wegmans School of Pharmacy, respectively.

Overall mean knowledge scores on the 10 COVID‐19 prevention and management‐related questions significantly improved from pre‐ to postintervention (5.9 ± 1.31 vs 8.6 ± 1.29, P < .001). Table 1 displays the pre‐ and postsurvey COVID‐19 prevention and management knowledge scores by item. Knowledge scores increased by a mean of 2.7 points overall following the intervention (P < .001). All, but two questions demonstrated statistically significant improvement following the IDC. Students also performed well on the COVID‐19 clinical patient case vignettes with a mean ± SD score of 22.41 ± 0.46 out of 25.

TABLE 1

Pre‐ and postintervention COVID‐19 prevention and management knowledge scores by item (n = 61)

Question number	Question topic	Preintervention (percentage correct)	Postintervention (percentage correct)	P value a
1	Appropriate patient scenario to use dexamethasone	15%	85%	<.001
2	Appropriate patient scenario to use remdesivir	46%	77%	<.001
3	Laboratory parameters to monitor COVID‐19 progression	72%	95%	<.001
4	Appropriate patient scenario to use baricitinib	39%	89%	<.001
5	Appropriate patient scenario to use bamlanivimab	18%	70%	<.001
6	Identification of remdesivir as an antiviral	93%	93%	1.000
7	COVID‐19 vaccine data and administration considerations	80%	95%	.006
8	Remdesivir adverse effects and monitoring	67%	97%	<.001
9	Appropriate patient scenario to use tocilizumab	52%	62%	.310
10	Inappropriate use of hydroxychloroquine for pre‐exposure prophylaxis	84%	100%	.002

P values are from t test analyses of pre‐ to postintervention improvement.

Overall mean confidence agreement with COVID‐19 prevention and management statements using a 5‐point , , , , Likert scale significantly improved from pre‐ to postintervention (2.66 ± 0.75 vs 4.03 ± 0.53, P < .001). Table 2 shows the mean improvement in confidence agreement scores for COVID‐19 prevention and management following the IDC. Pharmacy students' confidence increased significantly in all COVID‐19 areas assessed. Confidence scores increased the most for describing the role and appropriate use of various COVID‐19 drugs including remdesivir, baricitinib, dexamethasone, bamlanivimab, and tocilizumab as well as identifying the appropriate phase of COVID‐19, identifying drugs that should be avoided in COVID‐19 based on lack of clinical data, and COVID‐19 monitoring parameters.

TABLE 2

Mean pre‐ to postintervention improvement in confidence agreement scores for COVID‐19 prevention and management following the focused interactive didactic class (n = 61)

Question number	COVID‐19 category or topic	Mean improvement a (95% CI)
1	Identify the appropriate phase of COVID‐19	1.48 (1.21‐1.74)
2	Describe the role and appropriate use of remdesivir	1.46 (1.20‐1.72)
3	Describe the role and appropriate use of baricitinib	1.69 (1.41‐1.96)
4	Describe the role and appropriate use of dexamethasone	1.46 (1.18‐1.73)
5	Describe the role and appropriate use of bamlanivimab	1.80 (1.56‐2.05)
6	Describe the role and appropriate use of tocilizumab	1.59 (1.32‐1.86)
7	Describe the role and appropriate use of venous thromboembolism prophylaxis	0.88 (0.65‐1.12)
8	Identify drugs that lack clinical data and should be avoided in patients with COVID‐19	1.30 (1.03‐1.56)
9	Identify monitoring parameters for patients COVID‐19 to monitor progression	1.36 (1.12‐1.59)
10	Describe the efficacy and administration of COVID‐19 vaccines	0.67 (0.46‐0.88)

Abbreviation: CI, confidence interval.

P values were derived from t test analyses of pre‐ to postintervention improvement and all were statistically significant (P < .05).

Table 3 displays pharmacy students' perceptions of the COVID‐19 IDC. Pharmacy students' perceptions of the class were positive overall. On a 1‐to‐5 Likert scale, the mean agreement for each statement was above 3.9. Pharmacy students believe that participation will better prepare them for their clinical rotations and future pharmacy practice. They found the class enjoyable and that it generally improved their knowledge and comfort level for COVID‐19 prevention and management.

TABLE 3

Postintervention perceptions of the COVID‐19 interactive didactic class (n = 61)

Item	Mean ± SD a
This COVID‐19 interactive didactic class with active learning improved my knowledge of COVID‐19 prevention and management.	4.16 ± 0.78
This COVID‐19 interactive didactic class with active learning was an enjoyable learning experience.	3.95 ± 0.97
I learned content during the COVID‐19 interactive didactic class with active learning that will be useful in my future practice as pharmacist.	4.20 ± 0.75
I am more prepared in performing an evaluation and providing recommendations for patients with COVID‐19 following the COVID‐19 interactive didactic class with active learning.	4.11 ± 0.78
Participation in this COVID‐19 interactive didactic class with active learning will better prepare me for my clinical rotations.	4.08 ± 0.90

Items are measured on a 1 (strongly disagree) to 5 (strongly agree) scale.

DISCUSSION

To the best of our knowledge, there are currently no reports to describe COVID‐19 education not only within the PharmD curricula, but also within medical and nursing educational programs. This study describes the implementation of a COVID‐19 IDC at two schools of pharmacy. This IDC was designed to educate pharmacy students on COVID‐19 prevention and management and provide them with the literature evaluation skills to hopefully keep them prepared for emerging published data and frequent guidance updates. , A statistically significant increase in scores on all, but 2 of the 10 questions on the COVID‐19 knowledge assessment was observed. Question 6 was identification of remdesivir as an antiviral, which in retrospect, was likely too easy of a question to include as demonstrated by both the high percentage correct preintervention and the same percentage correct postintervention. Question 9 was focused on the appropriate use of tocilizumab, which may have been too challenging particularly because students had not yet learned critical care pharmacotherapeutics and this agent is used for severe COVID‐19, most commonly in critically ill patients. , A statistically significant increase in pharmacy students' confidence in all included COVID‐19 prevention and management statements were also observed. Furthermore, students generally enjoyed this COVID‐19 IDC and found it useful to their practice as a future pharmacist. This study's findings were consistent with those observed in a similar active learning, didactic instruction approach of penicillin allergy management, where pharmacy students' knowledge and confidence improved following this instructional method. Based on these findings and overall positive impressions from the students, the ID pharmacist faculty continue to deliver the IDC in the same semester and format. However, the IDC is delivered in the classroom and utilizes the updated COVID‐19 therapeutics and guidance from the NIH and IDSA guidelines. ,

Given the broad responsibilities of pharmacists in different settings for COVID‐19, pharmacy graduates need to be prepared and educated about COVID‐19 prevention and management. However, it is unclear if COVID‐19 education is being delivered within PharmD curricula. The 2019 American College of Clinical Pharmacy Pharmacotherapy Didactic Curriculum Toolkit, which provides guidance for disease state and subject inclusion within PharmD curricula, was published before the COVID‐19 pandemic. However, influenza and lower respiratory tract infections are tier one subjects so it is plausible that COVID‐19 would also be included here, but it is still uncertain. The authors hope this experience will further encourage Colleges and Schools of Pharmacy to prioritize COVID‐19 prevention and management didactic education as well as relevant literature evaluation skills to keep current with COVID‐19 guidance updates given pharmacy students' potential future role in the COVID‐19 pandemic. It may also be beneficial for Colleges and Schools of Pharmacy to consider adding COVID‐19 as a “core” disease state for preceptors to review or teach during relevant advanced pharmacy practice experiences. This would provide students with not only a potential opportunity to reinforce those key concepts learned during didactic instruction, but also to become familiar with any important updates and new data for COVID‐19 prevention and management.

This study is not without important limitations to consider. First, pharmacy students were surveyed soon after the COVID‐19 IDC was delivered without a follow‐up assessment. Given that these were third professional year students in the Spring semester, a subsequent follow‐up assessment would have had to occur during the fourth professional year, which was not deemed feasible due to other competing responsibilities during experiential education. However, it was anticipated that fourth professional year students would have the opportunity to learn and reinforce key concepts during their relevant advanced pharmacy practice experiences when reviewing or encountering patients with COVID‐19. Therefore, it is unclear if students will retain the content and application upon graduation for future practice. Future studies should consider performing follow‐up assessments to evaluate if the content and skills learned were retained. Second, COVID‐19‐related data and recommendations are evolving rapidly. Therefore, some COVID‐19 content at the time of the IDC may be outdated. However, the literature evaluation activity may help students review primary literature and stay updated on COVID‐19 management and prevention guidance. Third, the surveyed population is only representative of pharmacy students from two schools of pharmacy and may not be generalizable to pharmacy students at other Colleges or Schools of Pharmacy. Fourth, there is a potential for response bias and social desirability bias with this survey instrument. However, students were well‐informed that there was no incentive or “grade” provided to students who did or did not complete the survey in an attempt to minimize these biases. Finally, given the limited number of knowledge‐based questions asked, there is a potential wide range for significance where small changes in the number of correct or incorrect responses could impact results.

CONCLUSIONS

This study demonstrated that delivery of a COVID‐19 IDC to pharmacy students led to significant improvements in knowledge scores and confidence for COVID‐19 prevention and management. Future evaluations of COVID‐19 education and training within the PharmD curricula are needed since it is unclear if and to what extent such exists. This structure and delivery can serve as a model for Colleges and Schools of Pharmacy to increase COVID‐19 education for pharmacy students.

FUNDING INFORMATION

There was no external funding for this research.

CONFLICT OF INTEREST

Wesley D. Kufel has received research grants from Merck and Melinta Therapeutics, and served on the advisory board for Theratechnologies, Inc. Lisa M. Avery's husband is an employee of Merck. All others have nothing to disclose.