Evaluation of the effect of video tutorial training on improving pharmacy students' knowledge and skills about medication reconciliation.

OBJECTIVES: This study aimed to describe the effect of using an innovative teaching strategy using a video tutorial on enhancing students' knowledge about medication reconciliation, and skills in identifying medication discrepancies.
METHODS: This is a one group pre-post interventional study that was conducted at the School of Pharmacy at Applied Science Private University. Sixty pharmacy students were invited to attend an educational sessions that involve watching a 6-minutes video tutorial. The first two levels of the Kirkpatrick's Model were used to evaluate the effectiveness of this training tool. Level 1 (Reaction) was assessed using a satisfaction questionnaire, while level 2 (Learning) was assessed using two criteria: 1) student acquired knowledge about medication reconciliation using a questionnaire and a knowledge score out of 13 was calculated for each student, and 2) student acquired skills in identifying medication discrepancies using a virtual case scenario. If the student was able to identify any of the four impeded discrepancies he/she rewarded 1 point for each identified discrepancy, but if they identified any incorrect discrepancy they scored a negative point.
RESULTS: Among the 60 students who registered to participate in the study, 49 attended the educational training (response rate 81.6%). The majority of them (n=44, 89.8%) were satisfied with the training process. Before the video tutorial, students showed an overall low knowledge score [4.08/13.0, SD 1.81], and low ability to identify discrepancies [0.72 identified discrepancies out of 4.0, SD 1.1]. Following the video tutorial, the overall knowledge score was improved (p<0.001), and students were able to identify more discrepancies after watching the video (p<0.001).
CONCLUSION: In conclusion, video education has shown itself to be an effective method to educate pharmacy students.. This visualized method can be applied to other areas within pharmacy education. We encourage the integration of videos within the learning process to enhance students' learning experience and to support the traditional learning provided by the teaching staff. Copyright: © Pharmacy Practice.

INTRODUCTION

Medication Reconciliation is a process where medications at home are reconciled with medications prescribed at admission and at every transition in care thereafter.1 Medication reconciliation is an identified patient safety goal by the Joint Commission International (JCI) and is endorsed by the Institute for Safe Medication Practices (ISMP) and World Health Organization (WHO) as a process that, if practiced effectively, can significantly reduce preventable adverse drug events and unnecessary harm to the patient.2-4 Its implementation has shown a significant reduction in number of discrepancies across transitions in care and adverse drug events not only during hospitalization but also after discharge from the hospital.5,6

Effective medication reconciliation requires the collaboration of healthcare providers in ensuring the accurate use of medications through transitions in care. Clarity of roles and processes is vital for its effective implementation. Being medication experts, pharmacists among other health care providers have a primary role and can add value to the successful implementation of the process.7

Despite the recognized value of medication reconciliation as a process that reduces medication errors and patient harm, health care providers generally, pharmacists no exception, receive little formal education during their college years on how to obtain a proper medication history and how to implement the necessary steps for its successful.8,9 This may be attributed to the lack of awareness and unclear understanding of who has the responsibility for providing reconciliation service.1,10,11

Since lack of awareness and understanding about the service was among the barriers for implementing reconciliation services, therefore, this study was designed to address this gap in teaching medication reconciliation at pharmacy school years by the development of an innovative learning tool. The impact of educational programs on improving student’s ability to obtain medication histories and to identify and resolve discrepancies has been evaluated in few studies.12,13 Both studies have focused on incorporation an interactive learning exercise using simulation to educate students about medication reconciliation, and the findings of both studies showed that students’ knowledge about medication reconciliation and skills in identifying discrepancies were increased following the educational exercise. In our study we are aiming to evaluate the effectiveness of another teaching method to educate pharmacy students about medication reconciliation through the use of a short duration video tutorial.

So, the aim of this study is to evaluate the effectiveness of video tutorial training on improving pharmacy students’ knowledge about medication reconciliation and their skills in identifying medication discrepancies. The effectiveness of the learning tool was assessed using Kirkpatrick’s model.

METHODS

Study Design and participants

This is a one group pre-post interventional study that was conducted at the school of pharmacy at Applied Science Private University, located in Amman-Jordan, during December 2018. During the study period, pharmacy students were invited to participate in the study, by sending an invitation announcement at the faculty official page at Facebook. Pharmacy students were allowed to participate if they met the following requirements: 1) Pharmacy students in their third, fourth or fifth year of studying, and 2) willing to participate in the study.

Sample size

A sample size calculation was performed using the following formula:

Where

N is the sample size for one group

σd is the Standard deviation of the outcome variable.

Zα/2 is the appropriate value from the normal distribution for the desired confidence interval (1.96 for α= 0.05)

Zβ is the critical value of the Normal distribution for the power β (0.842 for 80% power)

ES is the effect size (the difference in means)

Using Zα/2 = 1.96 (95% confidence level), Zβ= 0.842 (80 % power), and the identified discrepancies during reconciliation activities (0.979; SD=0.142 pre-intervention versus 0.862; SD=0.345 post-intervention) based on related previous study which was conducted in Pennsylvania, a minimum sample size of 43 pharmacy students was considered sufficient to obtain a significant difference between pre- intervention and post-intervention reconciliation score.14 A target sample size of 60 pharmacy students was decided to be approached to account for any drop-out during the follow-up evaluation.

The Kirkpatrick model

The first two levels of Kirkpatrick’s Model were used to measure the effectiveness of the novel training tool. The Kirkpatrick model was designed by Donald Kirkpatrick in 1954. It can be applied before, during and after training to demonstrate the value of the training to the organization or individuals. There are four levels in the model:15

Level 1: (Reaction) “to what degree participants react favorably to the learning event”.

Level 2: (Learning) “to what degree participants acquire the intended knowledge, skills and attitudes based on their participation in the learning event”.

Level 3: (Behavior) “to what degree participants apply what they learned during training when they are back on the job”.

Level 4: (Results) “the degree to which targeted outcomes occur as a result of the training and the support and accountability package”.

Since the assessment of both levels 3 and 4 are considered to be cost prohibitive and difficult to be done, only the first two levels (Reaction and Learning) were assessed during this workshop.

Level 1 (Reaction) was assessed using a satisfaction questionnaire that evaluated the degree to which pharmacy students reacted favorably to the learning event. The questionnaire was adopted from a study that have evaluated the effect of a training workshop using hospital training software on improving pharmacy students ability in implementing medication reconciliation service in a hospital in Atlanta, Georgia.16 Minor modifications were performed on the questionnaire and it was face and content validated by the researchers of the study to enhance the clarity and comprehensibility of the questions. The questionnaire consisted of 10 statements that have evaluated students’ satisfaction about the video tutorial workshop. A five Likert scale (strongly agree, agree, neutral, disagree and strongly disagree) was used to evaluate each statement.

Level 2 (Learning) was assessed using two criteria:

1) Students’ acquired knowledge about medication reconciliation. This part consisted of a quiz that was made to evaluate pharmacy students’ knowledge about medication reconciliation. The questionnaire was adopted from a study by Boissinot et al., with minor modifications performed on it.17 The questionnaire was face and content validated by the researchers of the study to enhance the clarity and comprehensibility of the questions. The questionnaire consisted from 9 multiple choice questions that covered different aspects including the concept of medication reconciliation, how to achieve it, the Best Possible Medication History (BPMH), the resources of the BPMH, who is responsible to provide medication reconciliation, and types of medication discrepancies. Also, the questionnaire contained an essay question which asked the student to mention the four basic steps of medication reconciliation. Regarding the first 9 questions, if the students answered correctly they scored 1 mark each question, if not they scored zero. Regarding the last essay question, the student should mention the four basic steps for medication reconciliation and they scored 1 mark for each correct step, so a total mark out of 13 was calculated for each student.

2) Students’ acquired skills in identifying medication discrepancies. In this phase, students were asked to analyze a virtual case about a 68-year-old female patient, a known case of diabetes and osteoporosis. She was presented to emergency department as a case of community acquired pneumonia. Students were asked to identify all the discrepancies impeded in the case (four discrepancies) by comparing the prior to admission medications list (the BPMH) mentioned in the file with the current admission medications list. If the student was able to identify any of the four impeded discrepancies he/she rewarded 1 point each, but if they identified any incorrect discrepancy they scored a negative point.

Preparation of the training video

A 6-minutes duration video was prepared by three of the authors using the VideoScribe software. The video covered several areas of interest including 1) what is medication reconciliation, 2) why do we need it, 3) who should implement medication reconciliation, 4) barriers to implement medication reconciliation, 6) how to practice medication reconciliation and 7) a case scenario on how to implement medication reconciliation. The content of the video was extracted mainly form the WHO report on the standards to operate medication reconciliation.4 Before preparing the video, the content was reviewed and validated by the rest of authors to ensure the clarity and accurateness of the information provided. Also, after preparing the video, the content was reviewed again.

Training workshop

Pharmacy students were invited to attend the educational sessions using an announcement published at the University official Facebook page. The number of invited students were limited to 60 students, and once the number of registered students reached 60, the registration was closed. Then, the 60 registered students were invited to attend the educational workshop which was held at the workshop room at Applied Science Private University. The workshop conduction and data collection were supervised by two authors. After obtaining students’ informed written consent, a pre-training data collection form was filled within 15 minutes to evaluate students’ knowledge about medication reconciliation and skills in identifying medication discrepancies as mentioned previously, followed by watching a 6-minutes video tutorial. Then, a post-training data collection form was filled by the students after watching the video (within 19 minutes). In this form, we re-evaluated students’ knowledge about medication reconciliation and skills in identifying medication discrepancies, in addition to the degree of their satisfaction to the training workshop.

Ethics Approval

Ethics approval was obtained from the Applied Science Private University ethics committee, number 2019-PHA-1. The study was conducted following the ethical standards outlined in the World Medical Association Declaration of Helsinki guideline.18 All pharmacy students were informed that their participation in the study would be voluntary and that their responses would be kept confidential.

Statistical analysis

Data was analyzed using the SPSS version 22 (SPSS Inc., Chicago, IL, USA).The descriptive analysis was done using mean and standard deviation (SD) for continuous variables and percentage for qualitative variables.

Checking for normality was carried out using Shapiro-Wilk test (with p>0.05 indicating a normally distributed continuous variable). Paired t-test was used to evaluate pre-post changes in the knowledge score and medication discrepancies identification scores (continuous data); if the data was not normally distributed, Wilcoxon sign rank test was used instead. McNemar’s test was used to evaluate differences in categorical variables between pre-workshop and post-workshop data. In the case where expected cell counts were found below 5 for more than 20% of the cells, Fisher exact test was used instead. For all statistical analysis, a p-value of less than 0.05 was considered statistically significant and all tests were two tailed.

RESULTS

Among the 60 students who registered to participate in the study, 49 attended the educational training (response rate 81.6%). Female represented 51.0% of the study sample (n=25) and the majority of students were in their third year of study (n=42, 85.7%). None of the students had ever attended a course/workshop about medication reconciliation (Table 1).

Table 1

Demographic characteristics of the pharmacy students (n= 49)

Age (years), mean (SD)	21.2 (1.2)
Gender, N (%)
Male	24 (49.0)
Female	25 (51.0)
Year of Study, N (%)
Third	42 (85.7)
Fourth	5 (10.2)
Fifth	2 (4.1)
Have you ever attended a course/workshop about medication reconciliation? N (%)
Yes	0 (0.0)
No	49 (100)

SD: Standard deviation

Before the video tutorial, students showed an overall low knowledge score [4.08/13.0, SD=1.81], and low ability to identify discrepancies [0.72 identified discrepancies out of 4.0, SD=1.1]. Following the video tutorial, the improvement in students’ knowledge was assessed and presented in Table 2. Results showed that there was a significant improvement in the percentage of correct answers for the majority of questions (questions 1-3, 5-8, and 10) (P-values <0.05). Students’ knowledge was worsened for only one question (question 9) where students were not aware that only unintentional discrepancies are considered as a medication errors (there was a 20.4% reduction in the percentage of students who answered the question correctly, p=0.013). Regarding question 4 which evaluated students’ knowledge about the best number of information sources needed to be used formalize the BPMH, results showed no significant difference between students correct answers between the pre and post training phases (30.6% versus 16.3%, respectively, p= 0.118).

Table 2

Evaluation of students’ knowledge pre and post the video tutorial training, (n= 49)

Click here for additional data file.

Question	Correct answer n (%)		p-value
	Pre-training	Post training
Medication reconciliation	24 (49%)	33 (67.3%)	0.049
Medication reconciliation can be achieved:	26 (53.1%)	46 (93.9%)	<0.001$
When collecting Best Possible Medication History (BPMH), we should collect information about	25 (51.0%)	43 (87.8%)	0.001$
To formalize a Best Possible Medication History (BPMH), it is best to consult	8 (30.6%)	15 (16.3%)	0.118$
Who can be involved in the medication reconciliation process	27 (55.1%)	43 (87.8%)	<0.001$
Ideally medication reconciliation process should be provided by	23 (46.9%)	45 (91.8%)	<0.001$
What are the sources of information during the reconciliation process?	27 (55.1%)	40 (81.6%)	0.007$
An unintentional discrepancy may correspond to:	26 (53.1%)	44 (89.8%)	<0.001$
Which of the following types of medication discrepancies is considered a medication errors	13 (26.5%)	3 (6.1%)	0.013$
List the four steps of medication reconciliation:	0.04 (SD= 0.29)	3.37 (SD= 1.42)	<0.001#

*Correct answers (see ), $based on McNemar test, #Mean improvement using paired t-test, significant when P-value <0.

Click here for additional data file.

Also, the overall knowledge score was improved (from 4.08 (SD=1.81) pre-tutorial video to 9.76 (SD=2.48) post the video, P-value <0.001), and students were able to identify more discrepancies after watching the video (from 0.72 identified discrepancies (SD=1.05) pre video tutorial to 2.49 identified discrepancies (SD=1.37) post video tutorial, P-value <0.001) (Figure 1).

Figure 1

Improvement of students learning outcomes

The overall knowledge score was improved (from 4.08 (SD= 1.81) pre-tutorial video to 9.76 (SD= 2.48) post the video, P-value <0.001), and students were able to identify more discrepancies after watching the video (from 0.72 identified discrepancies (SD= 1.05) pre video tutorial to 2.49 identified discrepancies (SD=1.37) post video tutorial, P-value <0.001). Analysis performed using Wilcoxon sign rank test)

Finally, the majority of students (n=44, 89.8%) were satisfied with the overall training process, where 91.8% (n= 45) believed that the educational video made them understand the concept of medication reconciliation, and 93.9% satisfied that educational video helped them to understand the importance of medication reconciliation (Table 3).

Table 3

Evaluation of students’ satisfaction with the training process, (n= 49)

Statements, n (%)	Strongly agreed	Agreed	Neutral	Disagreed	Strongly disagreed	Missing data
The educational video stimulated my interest in medication reconciliation.	27 (55.1%)	15 (30.6%)	3 (6.1%)	2 (4.1%)	0 (0.0%)	2 (4.1%)
Methods of medication reconciliation introduced in this educational video are beneficial.	22 (44.9%)	23 (46.9%)	2 (4.1%)	1 (2.0%)	0 (0.0%)	1 (2.0%)
I enjoyed participating in this education training using video	18 (36.7%)	27 (55.1%)	2 (4.1%)	1 (2.0%)	0 (0.0%)	1 (2.0%)
The educational video was easy to follow and understand	24 (49.0%)	17 (34.7%)	8 (16.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
The educational video made me understand the concept of medication reconciliation	24 (49.0%)	23 (46.9%)	3 (6.1%)	1 (2.0%)	0 (0.0%)	0 (0.0%)
The educational video help me understand the importance of medication reconciliation	31 (63.3%)	15 (30.6%)	3 (6.1%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
The educational video clarified what is BPMH	19 (38.8%)	22 (44.9%)	7 (14.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
The educational video clarified how to collect BPMH perfectly	20 (40.8%)	24 (49.0%)	5 (10.2%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
The educational training video helped me to be able to identify medication discrepancies.	20 (40.8%)	21 (42.9%)	8 (16.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Overall, the educational training video enhanced my learning.	27 (55.1%)	17 (34.7%)	2 (4.1%)	2 (4.1%)	1 (2.0%)	0 (0.0%)

BPMH: Best Possible Medication History

DISCUSSION

Recently, the role of pharmacists has developed dramatically, and it is expected that pharmacists will play an influential role in providing pharmaceutical care services like medication reconciliation. Although medication reconciliation is considered as important service in the patient’s care process, many studies have reported several challenges to implementing this process.1,10,11 Pharmacists have not been directly concerned with gathering a good medication history in most hospitals so far.19 This may be attributed to the poor knowledge about the process, the lack of awareness, unclear understanding of who has the responsibility for providing reconciliation services, and shifting responsibilities to other health care providers for implementing the service.1,10,11

Accordingly, pharmacists must receive adequate and comprehensive training and education to be effective healthcare providers and to be able to provide medication reconciliation correctly. Therefore, it is the responsibility of pharmacy educators to provide pharmacists with scientific activities and workshops to be competent in their responsibilities and to ensure their capability to deliver this service to their patients.

The Accreditation Council for Pharmaceutical Education (ACPE) considers pharmacy education provided at the school of pharmacy level as one of the most important elements to develop professional pharmacists.20 Thus, it is vital for pharmacy educators to supply students with the appropriate skills to make sure that they received appropriate experiences as part for their training, as well as to develop the opportunities to fully involve pharmacy student in pharmacy-related activities, such as medication reconciliation that positively impacts patient care, provides benefits to the experiential site, and also provides a strong learning experience for pharmacy student.12,21-23

Students should be able to implement medication reconciliation in an integrated manner and to recognize its steps correctly. New technologies are gaining attention in the world of education, such as the use of online communication, the use of social media and the use of videos.24,25 Some of these methods are visualized, and visual teaching methods have been proven to increase students’ understanding and learning.25-31 However, little is known about their effectiveness in pharmacy education.25,27,32,33 Thus, this study aimed to evaluate the effectiveness of using video tutorials as visualized teaching strategies in pharmacy education to educate pharmacy students on how to collect medication history and how to identify medication discrepancies correctly. The effectiveness of this learning tool was assessed using Kirkpatrick’s model.

The findings of this study demonstrate that this visual teaching method using video significantly improved the total knowledge scores about medication reconciliation from 4.08 (SD=1.81) to 9.76 (SD=2.48) and there was an improvement for the majority of questions included in the quiz, whereby more students performed better in the post-video assessment quiz, except for one question where they scored less after watching the video tutorial. Study results were consistent in certain aspects with the findings of previous studies conducted in the United Kingdom and New Zealand.25,34 In those studies researchers used visual teaching strategies including videos that demonstrate several practical skills and visual aids in the form of Magnetic Resonance Imaging (MRI) images, and results revealed that these visual methods of teaching were effective in improving students understanding about selected pharmaceutical science concepts and enhancing students learning experience innovatively and interactively.25,34

Regarding students’ skills in identifying medication discrepancies, students were able to identify more discrepancies after watching the video. This indicates that students were able to apply their gained knowledge into practice. Using visual learning could be thought to enhance the recall of information compared to the traditional auditory learning using lectures.35 This was supported by a previous study by Von Schantz et al. which investigated the effect of using videos on improving inhalers use technique. The study revealed that video technology could promote recall on inhaler using technique among participants.33 Moreover, the use of video tutorials as a teaching strategy had the advantage of being available at any time. Students can hear the information more than once and return to the video in case they missed any information, unlike traditional lectures.

Finally, when evaluating students perceived satisfaction about the use of video as a teaching strategy in their training, the majority of students were satisfied with the overall training process, and they believed that the educational video made them understand the concept of medication reconciliation. This finding was similar to that obtained by Zhang et al., where they showed that the learning process involved videos resulted in a higher level of students’ satisfaction than other used methods.36 So, we recommend incorporating visual educational videos into the learning process.

Study limitations

The results of this study should be understood taking into consideration the following limitations. First, the study represented a one group pre-post study design that has evaluated the incorporation of video education into student teaching about medication reconciliation without having a comparator group to compare this training method with the standard methods of training. Second, the influence of the training module using a video tutorial was studied immediately after conducting the workshop. This may not reflect the actual long term effect of using video as a teaching strategy. Third, the effect on real practice was not evaluated, as student evaluation was performed in school only. Forth, students’ skills in the identification of medication discrepancies were evaluated only using one case scenario that applies to hospital settings. And finally, our study was conducted only among one cohort of students from one university which may limit the generalizability of the results.

CONCLUSIONS

In conclusion, video education has shown itself to be an effective method to educate pharmacy students. We encourage the integration of this visualized method of education within the learning process to enhance students’ learning experience and to support the traditional learning provided by the teaching staff.