Development and evaluation of an online medication safety module for medical students at a rural teaching hospital: the Winchester District Memorial Hospital.

OBJECTIVE: To design, implement and assess an online learning module for third-year and fourth-year medical students addressing medication safety.
DESIGN: This study was a prospective, parallel, open-label, randomised controlled trial with two arms: (1) a control arm in which students were given five articles to read about medication safety, and (2) an intervention arm in which students were given access to an interactive web-based learning module on medication safety. Pretesting and post-testing were done online to evaluate change in medication safety knowledge.
RESULTS: Ten students completed the study in the intervention group (online module) and six students completed the study in the control group. The increase in score obtained on the post-test, relative to the pretest, was 15.4% in the group who completed the online module and 2.0% in the control group (difference=13.4%, 95% CI 0.5% to 26.2%, p=0.04).
CONCLUSION: Students who completed an online educational tool about medication safety demonstrated a significantly greater increase in knowledge than those who completed a few readings. Online learning modules can be a convenient and effective means of teaching safe prescribing concepts to medical trainees. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

Introduction

An adverse event (AE) is defined as an undesired and unplanned occurrence, directly associated with the care or service provided to a patient in the healthcare system. An adverse drug event (ADE) is an injury from a medication or lack of an intended medication, and includes adverse drug reactions (ADRs) and harm from medication incidents.1 In the first national study of AEs in Canadian hospitals, it was found that 7.5% of hospital admissions were accompanied by at least one AE, and that over one-third of these events were deemed preventable.2 Approximately one-quarter of the AEs were related to medication or fluid administration. In a patient safety study conducted at the Winchester District Memorial Hospital (WDMH) in 2004, the majority of reported AEs were caused by medication use (50%), of which half were preventable.3 In a subsequent study examining aAEs suffered by patients after discharge from the hospital, it was found that 72% of events were drug related.4 Preventable AEs were classified most frequently as therapeutic errors, which consisted of the concomitant use of medications known to interact, the use of a treatment known to be contraindicated in a specific condition or the failure to adequately monitor a treatment.4 The Institute of Medicine has reported that as many as 44 000–98 000 deaths occur annually in hospitals in the USA from some type of medication error.5 All stages of the drug delivery process are susceptible to error, including prescribing, transcribing, dispensing and administration.6

Inaccuracies made during the drug prescribing stage, however, are the most common type of avoidable medication error.7 Prescribing decision errors have thus become an important target for improvement in the patient safety movement.

There is a wide range of factors contributing to medication errors, including distracted staff, lapses in memory, slips in attention, errors of judgement, lack of knowledge, communication problems, inadequate pharmacological training and a culture that does not support safe prescribing.8–10 If a prescriber lacks knowledge of the drug or patient in question, or if handwritten orders are illegible, incomplete or ambiguous, the likelihood of an ADE increases.11 In order to curtail the incidence of medication-related AEs, hospitals across the nation are implementing various safeguards in the medication use system.

For instance, computerised physician order entry, computerised decision support systems, pharmacist participation in medical rounds, bar coding systems, preprinted order forms and provision of key reference materials to staff are but a few examples.6 However, strategies that consist of purchasing new hardware or software can become prohibitively expensive, whereas other cost-efficient methods can still have a significant effect on medication safety. Creating a culture within the hospital of patient safety, developing standardised protocols (especially in relation to high-risk medications) and developing an educational programme are examples of such strategies.12

Medication safety teaching represents a recognised strategy in risk reduction, and is perhaps most valuable in the foundational years of medical education. More prescribing errors occur in the first year after graduation than in all other years.13 As mentioned above, many factors contribute to the susceptibility of committing a medication error. For junior doctors in particular, the most likely factor is a deficiency in clinical pharmacology training, causing a decreased confidence in safe prescribing skills.9 11 The American College of Chest Physicians recently released evidence-based guidelines on the effectiveness of continuing medical education (CME) on physician knowledge.14 Based on the association between knowledge-based certification and improved clinical performance, it was recommended that multimedia CME interventions (eg, video, audio, print) with multiple instructional techniques (eg, discussion groups, case-based learning) be used to improve physician knowledge.14

There are relatively few published studies on the effectiveness of medication safety educational interventions in medical students. With a focus on medication reconciliation, Lindquist et al evaluated the effect of an interactive simulation on the improvement in student knowledge and comfort in obtaining medication histories.15 Students received a 30-minute lecture on the necessities of obtaining an accurate medication history and tools available for acquiring more information from pertinent sources, then participated in a simulation which included a physician, pharmacist and actor playing the role of the patient. Students rated their knowledge level as having increased by 27% and their comfort level as having increased by 20%. Garbutt et al conducted two 1-hour, small-group, interactive educational sessions for third-year medical students, held 2 weeks apart.11 Following the intervention, the average number of error-free prescriptions (evaluated through verbal transcription of 10 prescription orders) increased fivefold from 0.82 per student to 4.54 per student. Following the intervention, the average number of errors and dangerous errors per student decreased from 14 to 7.4 (p<0.0001) and from 4.8 to 2.7 (p<0.0001), respectively. Pharmacy team conducted a pharmacist-facilitated teaching session given to 40 final-year medical students.16 The teaching consisted of five stations: managing anticoagulation, preparing intravenous medication, taking a medication history and writing an inpatient prescription, writing a discharge prescription that includes a controlled drug, and teaching inhaler technique. One month later, a random sample of 16 taught and 16 non-taught students participated in an objective structured clinical examination (OSCE), which resulted in higher scores obtained by the taught group in eight out of nine OSCE stations.

Finally, Celebi et al conducted a randomised controlled crossover study assessing the effect of a 1-week training module on drug-related problems.17 Seventy-four year-5 medical students participated in a seminar on prescription errors, a prescribing exercise using both paper cases and real patients, and discussions with a lecturer. Following the training module, rates of potential prescription errors were reduced from 69% to 29% (p<0.001). Together, these four educational interventions resulted in students having an increased sense of knowledge and comfort in taking a medication history, a higher rate of error-free prescriptions and superior performance in clinical examinations. None of the interventions, however, consisted of an online module, nor did they address medication safety as a wide-ranging issue.

It is imperative to consider basic learning styles in order to maximise knowledge acquisition especially for medical students where a high burden of information is often presented. Every student has a unique learning style and with the meshing hypothesis, which states that instruction is best provided in a format that matches the preference style of the learner.18 The Fleming and Mills’ VARK model describes the four modes of learning which focus on visual, auditory, reading and kinesthetic learning styles.18 Visual learners learn predominantly through seeing and they often prefer visual displays, diagrams and video demonstrations.18 Auditory learners learn through listening and they often prefer verbal lectures, discussions and sometimes even background music.18 Readers learn and prefer when information is displayed as words, and they often prefer manuals, reports and essays.18 Kinesthetic learners learn through movement and they often prefer hands-on exercises.18 Thus, with different learning styles, different educational interventions may be used to maximise knowledge acquisition for medical students.

The educational intervention in this study addresses medication safety on the whole, and covers issues relating to obtaining accurate histories, writing clear and rational prescriptions, avoiding common prescribing pitfalls, learning the risks associated with dangerous abbreviations and high-risk medications, following hospital protocols, drug monitoring concepts, accessing drug information, and reporting AEs. The aim of this study was to evaluate the effect of an online educational module on medical students’ knowledge of medication safety.

Methods

Study design and population

This was a prospective, parallel, open-label, randomised controlled trial involving medical students at the WDMH. Eligible participants consisted of third-year or fourth-year students enrolled in the Doctor of Medicine programme at the University of Ottawa. We chose to recruit medical students from within the same school to ensure a consistent degree of prior exposure to medication safety concepts. Exclusion criteria included inaccessibility to an email account or the web-based module.

Upon receiving ethics approval from the WDMH Research Ethics Board, research participants were identified and contacted via class email listserv. The recruitment email contained information about the study as well as a link to the pretest. Prior to completing the pretest, students were asked to consent to study participation and provide some baseline demographic information. Students who did complete the pretest were then eligible for randomisation to either arm of the study (ie, control group or intervention group). In order to ensure equal numbers of subjects within both groups as well as even distribution of potentially confounding factors, block randomisation was used to allocate participants to control or intervention. Block sizes of 2 were used, and the randomisation sequence was created by an online sequencing website (www.random.org).

Intervention (online module)

Students randomised to the intervention group were granted access to a web-based learning module on medication safety and were required to complete it within a 2-week time frame.

This module was developed by a pharmacy resident at the WDMH using Microsoft PowerPoint to cover issues related to obtaining accurate medication histories, writing clear and rational prescriptions, avoiding common prescribing pitfalls, learning the risks associated with dangerous abbreviations and high-risk medications, following hospital protocols, drug monitoring concepts, accessing drug information, and reporting AEs. Interactive examples and case-based problem-solving were used to try to enhance the student’s learning experience and challenge them to think about such issues rather than passively commit them to memory. The module aimed to teach skills in avoiding ADEs, as well as learn how to manage and report such events should they occur. The module was reviewed by a physician, a professor and two pharmacists, all of whom are involved in medication safety initiatives at the WDMH.

Control (assigned readings)

The control group was emailed five reference articles to read.2 6 19–21 These readings address the importance, prevalence and causes of medication errors, as well as strategies to reduce them.

They were chosen by the study investigators as a representative sample of what a medical student would be assigned to read in school when learning about medication safety. The articles were emailed in portable document format.

Outcomes (multiple-choice tests)

The primary outcome was an increase in medication safety knowledge, determined by the change in score obtained from the pretest to the post-test. The tests contained 50 multiple-choice questions designed to specifically assess topics covered in the readings and online module. The tests were created and accessed online through www.surveygizmo.com. Following completion of the training module or the assigned readings by the intervention and control groups, respectively, students were asked to complete the post-test, which was identical to the pretest. The mean change in scores (post-test minus pretest) was compared between the intervention and control groups.

Statistics

Scores are expressed as mean±SD. Unpaired Student’s t-test was used to compare the difference in scores obtained in both groups. We considered p<0.05 as statistically significant.

Results

Twenty-six out of 300 possible medical students completed the pretest (8.7%) and were eligible for randomisation to the intervention or control study groups. Sixteen of those students proceeded to complete the assigned intervention as well as the post-test, thus, a total of 16 students participated in the entire study. Six students were in the control group (readings) and 10 students were in the intervention group (online module). Baseline characteristics focused on factors that could influence scores on the tests, that is, age, medical school year, number of years of postgraduate education and highest degree earned. These baseline characteristics were similar between both groups (table 1).

Table 1

Baseline characteristics

	Control group(n=6)	Intervention group(n=10)	P value
Age (mean, years)	31	24	0.65
Postgraduate education (mean, years)	7	4.2
Medical school year (%)
Third year	50 (3)	50 (5)	1
Fourth year	50 (3)	50 (5)	1
Highest degree earned (%)
Bachelor’s	83 (5)	80 (8)
Master’s	0 (0)	20 (2)
PhD	17 (1)	0 (0)

The average score obtained on the pretest was 64.3% in the control group and 61.4% in the intervention group. After the completion of the assigned readings, the score on the post-test increased to 66.3% in the control group and 76.8% in the intervention group (difference=13.4%, 95% CI=0.5% to 26.2%, p=0.04). Thus, the group that used the online medication safety module obtained a significantly higher score on the post-test, relative to the pretest, than the control group (table 2).

Table 2

Medical student scores on pretests and post-tests

	Control group	Intervention group	P value
Pretest score (mean, %)	64.3	61.4
Post-test score (mean, %)	66.3	76.8
Difference in score	0.02	15.4	0.04

Discussion

The online medication safety module used in this study represents a novel effort to train medical students on the important aspects of safe prescribing and ADE management. We demonstrated the effectiveness of an online tutorial on knowledge attainment regarding medication safety. The use of voice narration and interactive, case-based questions throughout the module, as well as the capability to complete the module at one’s own pace, likely contributed to the success of the learning experience. It is important for clinical clerks to understand concepts, such as obtaining accurate histories, dangerous abbreviations and high-risk medications, therapeutic drug monitoring, drug information resources and so on, at the beginning of their careers. This allows for a greater understanding of the importance of reporting any ADRs for analysis and quality improvement in order to optimise patient outcomes and safer practice.

The initial scores on the pretests were very close between the two groups, as were all baseline characteristics. The change in score after the post-tests was complete, however, demonstrated much greater knowledge acquisition by the students who used the online module. The use of online educational tutorials is growing among medical schools and this study demonstrates the value of such an educational strategy. Compared with traditional readings, an interactive online course keeps the student interested and engaged. It serves as a simple, low-cost method of enhancing medication safety in clinical practice. Furthermore, the online module allows for multiple modes of delivering information an--d includes texts, diagrams and narration. This variety of modes offers an advantage for learners whether their learning style is visual, auditory, reading or a combination of the different learning styles. Many physicians often use the internet to continue their medical education, although prior randomised controlled studies of web-based learning have shown mixed results of knowledge acquisition.22–24

Limitations

The study results need to be interpreted in light of many limitations. First, the assessment of gaining knowledge was based on written multiple-choice tests and not practical measurements of medication safety competence.

Second, the number of students who participated in this study was small. Recruitment only through email is likely not the best tactic as students may get overwhelmed with the number of emails received via the listserv every day. Expansion of this study to other medical schools in the province could have yielded a higher response rate and would have been feasible, since recruitment, participation and data collection all occurred online. The module, however, focused specifically on WDMH protocols and examples, therefore limiting the interest to the University of Ottawa medical students only. Another avenue to explore is to conduct the study as a multicentre approach which includes other sister hospitals with relatively similar number of beds. In addition, one other factor to consider is whether medical students are receiving training from different centres which can confound the educational intervention our study has explored.

Third, the immediacy of the post-test following the exposure (readings or online module) only demonstrated short-term knowledge acquisition. If another post-test were to be administered a few months after the first, the long-term impact on retention of knowledge gained by the exposure could have been assessed in order to determine applicability to clinical practice. Due to time constraints, this was not a feasible approach. However, we highlight that this is an area which can be explored further in order to optimise efficient and effective learning.

Despite these limitations, the outcome of this study shows that this educational tool has a significant utility in improving knowledge of medication safety in students. As such, this tutorial will become part of the mandatory orientation process for all new medical residents at the hospital and will also serve to address a fundamental hospital accreditation standard on providing training and education on safe medication use.