Is online case-based learning effective in helping undergraduate medical students choose the appropriate antibiotics to treat important infections?

BACKGROUND: Medical students are frequently confused about indication for and choice of antibiotic. We developed an online learning resource that focused on antibiotic stewardship and important infections where medical students could practise their antibiotic decision-making skills safely.
METHODS: The resource was made available to third-year undergraduate medical students via their virtual learning environment. It covered the theory and fundamentals of antibiotic stewardship and five clinical cases covering important infections. We assessed the number of attempts taken to achieve the required level of understanding to pass each activity and surveyed a selection of students for their feedback.
RESULTS: Of 310 students, over 80% engaged with the theory-based components, with an average score exceeding 90% (range 93.4%-99.7%). Eighty-three percent (258/310) engaged with the first two cases (Clostridioides difficile infection and pyelonephritis) but only 61% (189/310) of students completed the fifth case on bacterial meningitis. Only 49.4% (153/310) of students completed all five cases, with 48% (73/153) of these achieving ≥90% on their first attempt of the associated quizzes. Fifty-nine percent (23/39) agreed or strongly agreed that the quality of the learning resource was excellent. Seventy-two percent (28/39) agreed or strongly agreed that the objectives of the resource were relevant to their needs as undergraduate medical students. Only 33% (13/39) reported the resource would change their practice.
CONCLUSIONS: Student feedback was positive but engagement with the cases needs improvement. Highlighting the utility of case-based technology-enhanced learning as a safe place to practise antibiotic decision-making skills among students may improve this.

Introduction

Antibiotic resistance is a global health concern. The emergence of antibiotic resistance and MDR organisms (MDROs) has arisen from the indiscriminate use of antibiotics and poor prescribing and dispensing practices. Conserving these medicines and preventing the emergence of new MDROs is now a global health priority. Nationally, most countries have published antibiotic prescribing guidelines to direct prescribers. For example, in the UK, NICE has evidenced-based guidelines to manage common infections. In Ireland, the Health Service Executive has similar guidelines available, as does Australia, to name but a few.

In recent years, the use of technology-enhanced learning (TEL) in the training of healthcare professionals has been used to provide better access to guidelines. Many healthcare institutions have their own localized antibiotic guidelines accessible online and/or through applications accessible via portable smart devices. Having information that is easily accessible improves patient care but knowing how to use that information correctly, without structure, curation or context to the relevant scenario, makes the practical application for the user more challenging. Choosing the appropriate antibiotic to prescribe empirically, or even in a directed way, can be difficult as there are a number of variables that can influence the choice.

Our own experiences with undergraduate medical students suggest they find it difficult to choose an appropriate antimicrobial agent and providing a clinical context is key to helping them understand and make appropriate choices. The overall aims of this study were to: (i) develop a case-based TEL resource that focused mainly on the management of clinically important infections with antibiotics; (ii) to assess student engagement with the resource; and (iii) to examine whether the resource improved the ability of the students to choose an appropriate antibiotic to treat these infections. We also aimed to gauge the student learning experience as student motivation is known to drive learning.

Methods

Study design

This was an observational intervention study designed to assess whether undergraduate medical students made use of the TEL resources available, and the ability of the students to choose the appropriate antibiotics to treat important infections. The online resource was designed to complement face-to-face teaching, which consisted of lectures and case-based tutorials, and other online content in an already integrated system-based blended medical curriculum. This information was captured through the Royal College of Surgeons in Ireland (RCSI) virtual learning environment (VLE), Moodle. Furthermore, we used an online survey to determine student satisfaction and attitudes to the online learning resource. An overarching conceptual framework was adopted for devising a practical evaluation process.

Student population

The student population included intermediate cycle (IC) (Year 3) undergraduate medical students in Dublin enrolled in a 5 year direct-entry medical degree programme at the RCSI (2013–18). Students at this stage in their education were chosen as clinical microbiology is a core subject at this phase in the curriculum. These students have already had some early patient exposure in the community setting and are about to begin a further stage of appropriate training in medicine and surgery in the clinical setting, which is followed by rotations in medical and surgical subspecialities, e.g. paediatrics, psychiatry and obstetrics and gynaecology.

Design of the online module

The online antibiotic learning resource was designed to support existing lectures and tutorials and to further highlight the importance of antibiotics and their appropriate clinical uses. The aim of the module was to improve the knowledge of undergraduate medical students around: (i) the classification of antibiotics; (ii) the mode of action of important classes of antibiotics; (iii) the principles of antibiotic stewardship; (iv) the mechanisms of antibiotic resistance; (v) the epidemiology of important resistant bacteria; and (vi) the adverse effects of antibiotics. We aimed to improve decision-making skills when choosing an appropriate antibiotic to manage a patient with a clinically important infection. The module had four learning outcomes. On completion of the online module, students should be able to:

i. Apply the principles of antibiotic stewardship to ensure antibiotics are used wisely

ii. Choose the appropriate empirical antibiotics to treat clinically important infections

iii. Interpret laboratory results, e.g. microscopy, culture and susceptibility results

iv. Rationalize empirical choice to an appropriate and more directed antibiotic

The module consisted of the six interactive tutorials with accompanying quizzes and feedback that covered the theory and knowledge-based components (referred to as theories from now on). The module was made available through Moodle over one semester (12 weeks) that consisted of three integrated system-based modules covering the gastrointestinal system, the renal and genitourinary system and the skin, bone and CNS. Five interactive online case-based tutorials were also developed using Articulate Storyline (UK) e-learning authoring software to align with the clinical microbiology content delivered in the three modules taken by students during the study period. This authoring software allows for the creation of question/answer-type activities that were scripted and based on a specific topic (Figure 1). The five clinical cases developed included: (i) Clostridioides difficile infection (CDI), (ii) pyelonephritis, (iii) cellulitis, (iv) septic arthritis and (v) bacterial meningitis.

Figure 1.

Example of case-based activity workflow. (a) All online cases had the same format and appearance with the opening case vignette giving context to the students and an introduction to the patient. (b) Relevant clinical information was then provided to the students. (c) Students made an initial diagnosis. (d) Students had to choose an appropriate empirical antibiotic. The case progressed over time with more information, such as laboratory results and susceptibility testing results, being provided. On this basis, the students were asked to rationalize their treatment.

To progress through the module, students had to obtain a goal grade of ≥90% in each quiz before accessing the next online activity. Each clinical case was only released on the same day but after the same topic was covered in the face-to-face teaching. Students had unlimited attempts to complete each online activity and they received immediate feedback after each attempt.

Assessment of engagement and the ability to choose the correct antibiotic

Data captured through Moodle (e.g. usage data, grades and log files) were used to assess student engagement with the online module. Student responses were anonymized. Descriptive statistics (frequencies and percentages, means and standard deviations) were used to summarize the data and explore the number of students who engaged with the resource. For those who engaged with the resource, the number of attempts it took to complete the cases and the percentage of students that completed the cases were recorded.

Assessment of student attitudes

Two different approaches were used to determine student attitudes to the online module. The TEL-evaluation framework of Cook and Ellaway and an online survey were used to collect the students’ opinions. These findings were further analysed to determine whether there was any correlation between students’ performances, perceptions and engagement (as determined from IT analytics).

Ethics

Ethical approval was sought from the RCSI Research Ethics Committee in 2014 to collect student data from VLE IT logs. In accordance with this ethical approval, all student data were anonymized and stored in an encrypted file on a secure network.

Results

Students’ understanding of basic knowledge and principles

There were 310 undergraduate medical students in this class. In most instances, over 80% of the class engaged with the theory-based components of the online module (Table 1). However, there was a slight fall-off in engagement from the first activity, which covered antibiotic classification [287/310 (92.6%)] to the last activity, which covered adverse effects of antibiotics [261/310 (84.2%)]. Overall, students performed very well in all the activities relating to basic knowledge (i.e. mode of action) or principles around the correct use of antibiotics. Of the students who completed the activities, the average score for all these activities was ≥90%, with a range of 93.4%–99.7% (Table 1). Despite doing best on the online activity relating to mechanisms of antibiotic resistance, students appeared to struggle most with the epidemiology and treatment of important antibiotic-resistant bacterial infections. Of note, only 51 students had an average score of less than 90% across all theories and, of these, all except seven students did not complete all the theories. The seven students who completed all theories had a score of less than 90% in one theory-based activity only, which pulled down their average score across the theory content.

Table 1.

Student performance in theory-based online activities

Theory activity	Score			Percentage of students scoring 100%
	n	mean (SD), %	median (range), %
Classification of antibiotics	287	96.1 (3.9)	95.5 (90.9–100)	44.6
Mode of action of antibiotics	281	97.6 (3.4)	100 (92.9–100)	66.1
Important antibiotic-resistant bacteria	271	93.4 (4.9)	95.0 (45.0–100)	19.2
Mechanisms of antibiotic resistance	269	99.7 (2.7)	100 (62.5–100)	99.3
Antibiotic stewardship	267	96.7 (3.6)	100 (85.7–100)	54.7
Adverse effects of antibiotics	261	98.0 (3.4)	100 (92.3–100)	74.0

Ability to choose the appropriate antibiotic

Engagement with the first two online cases was good, with 83.2% of the class completing the cases on antibiotic-associated diarrhoea/CDI and pyelonephritis (Table 2). However, as the semester progressed, engagement began to fall off, with only 61% (189/310) of the class completing the case on bacterial meningitis and 65.8% (204/310) of the class completing the last case of the semester, which covered septic arthritis. Overall, only 49.4% (153/310) of students completed all five cases. Of these, 48% (73/153) achieved the pass grade of ≥90% on their first attempt and a further 22% (34/153) took a second attempt on at least one case. Just over 9% (14/153) of students required at least two attempts on all five cases to achieve the ≥90% pass mark.

Table 2.

Attempts to achieve ≥90% pass mark in the antibiotic management of each infection

Case	Total completed (≥90%)	Completed on first attempt	Completed on second attempt	Completed on third attempt	Completed on fourth attempt	Completed on ≥fifth attempt
CDI	258 (83.2)	205 (79.5)	35 (13.6)	10 (3.9)	4 (1.6)	4 (1.6)
Pyelonephritis	258 (83.2)	226 (87.6)	23 (8.9)	7 (2.7)	2 (0.8)	0
Cellulitis	249 (80.3)	206 (82.7)	31 (12.4)	8 (3.2)	2 (0.8)	2 (0.8)
Septic arthritis	204 (65.8)	152 (74.5)	30 (14.7)	15 (7.4)	4 (2.0)	3 (1.5)
Meningitis	189 (61.0)	139 (73.5)	37 (19.6)	8 (4.2)	4 (2.1)	1 (0.5)

Values are presented as n (%).

Table 2 shows the total number of students that completed and passed each case and the number of attempts it took for the students to achieve the required pass mark. For each case, most students were able to work through the case, making the appropriate antibiotic prescribing decisions to achieve the ≥90% pass mark on their first attempt. Students performed best in the pyelonephritis case, with 87.6% (226/258) achieving ≥90% on their first attempt; this was followed by the case on cellulitis, with 82.7% (206/249) achieving ≥90% in this activity on their first attempt (Table 2). Students appeared to struggle most with the antibiotic management of CDI and septic arthritis, with 20.5% (53/258) and 25.5% (52/204), respectively, requiring two attempts or more to achieve the pass mark.

Students’perspective of the TEL

Thirty-nine students of 310 participated in the online survey. Of these students, 80% (31/39) agreed or strongly agreed that the online module was well organized. Seventy-four percent (29/39) agreed or strongly agreed that enough initial instruction was given before use of the online module and 72% (28/39) felt that the course objectives, expectations and requirements were made clear from the outset. From a design perspective, 59% (23/39) agreed or strongly agreed that overall the quality of the learning resource was excellent. Most students found the resource easy to navigate and experienced very few technical issues.

The majority of those surveyed (72%; 28/39) agreed or strongly agreed that the objectives of the learning resource were relevant to their needs as undergraduate medical students. Again, and importantly, 72% (28/39) of the students agreed or strongly agreed that the online resource encouraged achievement of the overarching module learning objectives (Figure 2). Interestingly, there was less agreement surrounding the links between the content delivered online and that delivered in class, with only 46% (18/39) agreeing or strongly agreeing that the online resource was effective at blending both elements. Similarly, only 54% (21/39) agreed or strongly agreed that classroom teaching helped meet the objectives of the online resource. However, 72% (28/39) agreed or strongly agreed that the assessments were appropriate for the resource objectives, the content and the activities completed. When asked if they felt the resource would change their practice, the responses were evenly split with only 33.3% (13/39) agreeing or strongly agreeing, 33.3% (13/39) remaining neutral and 33.3% (13/39) disagreeing or strongly disagreeing that it would (Figure 2).

Figure 2.

Student perceptions on learning and engagement. Thirty-nine students were surveyed using an online tool. Student perceptions of the online resource were gauged using a five-point Likert scale with a range of strongly agree to strongly disagree. Data represents the percentage of students with a specific opinion relating to the question posed.

Discussion

We developed an online learning resource for third-year undergraduate medical students, covering antibiotic stewardship theory followed by five interactive clinical cases where antibiotic decision-making skills could be practised safely. While over 80% of the students engaged with the theory-based components and gave positive feedback overall, engagement with the cases that convert theory into practice needs improvement. Interestingly, of the students surveyed, there was an even split in opinion, with some believing it would impact positively on their future ability to choose and use antibiotics wisely in their future practice and others feeling it would not.

Choosing an appropriate antimicrobial agent to theoretically manage an infection can be difficult for undergraduate medical students and other future prescribers. There are several factors that need to be considered and, without any clinical context or patient contact and experience, the ability to make the appropriate choice may only develop with experience and after they graduate. Indeed, most final-year European undergraduate medical students believe they need more training and education on the appropriate use of antibiotics to practise effectively as junior doctors.

The use of web-based learning interventions was associated with consistent and large positive effects in the education of healthcare professionals in a meta-analysis of 201 studies. A recent study found that the introduction of a mobile application, which focused on antibiotic prescribing and stewardship, resulted in an increase in compliance with antibiotic guidelines within medical and surgical specialities. Another study found 89% of prescribers used a smartphone or tablet in the emergency department when making antibiotic prescribing decisions. Often, anxiety about missing an infection, issues around the potential adverse effects of certain antibiotics and being influenced by supervising superiors impact how antibiotics are prescribed and are likely to contribute to inappropriate use of these medicines.

The teaching of the principles of antibiotic stewardship and the correct indications for use of important classes of antibiotics most often begins for healthcare professionals when they are undergraduate students. The use of e-learning and other forms of TEL are becoming more prominent in the curricula of health professional degrees, where a more blended approach to learning is favoured. In recent years, serious gaming and online clinical cases have been used to enhance teaching on antibiotics and improve student understanding.

Through an online module we aimed to support our face-to-face teaching around antimicrobial stewardship, antimicrobial pharmacology and the antimicrobial management of clinically important infections. Online case-based teaching and virtual patients have been developed and used by others to help students apply and integrate their knowledge of clinical microbiology and infection., Also, online case-based learning has been shown to give students a comparable learning experience to traditional face-to-face teaching.

Our findings suggest that students recognized the importance of the content of the online module as engagement with the activities was generally high and the feedback generally positive. This was a positive outcome as McCarthy et al. believe positive feedback such as this can be filtered down to students in earlier years, which in turn can influence their decision to engage or not with the material. However, feedback indicated that students did not think that the online material and the material delivered in class blended. This was somewhat surprising as the content online was mostly repetitious of that delivered in face-to-face case-based teaching on the same topic, but it often differed in context, e.g. the patient or the pathogen. The online cases were also layered with complexity that may have challenged those who took more than one attempt to complete the activity.

While most students were able to work through each theory-based activity and case-based activity competently in their first attempt, some topics were more challenging. Of note, students struggled with the epidemiology of important antibiotic-resistant bacteria. Complex infections such as antibiotic-associated diarrhoea/CDI, septic arthritis and bacterial meningitis also proved more challenging to our cohort of students. Encouragingly, students did not struggle with the principles of antibiotic stewardship. The latter is extremely important given the challenges presented to modern healthcare delivery by MDROs. A recent review suggests that healthcare professionals should have continuous training on antimicrobials to ensure changes in practice around the prescribing of antimicrobials.

Timing of the content release would also appear to be important. There was a fall-off in engagement with online material as the semester progressed. Engagement with the cases covering bacterial meningitis, cellulitis and septic arthritis was lower and this may be explained by the timing of their release online, which was close to the end of the semester and 2 weeks before the assessment period. It is well documented that assessment is a driver of learning, especially in high-stakes examinations. However, our findings would suggest these students were a little less engaged with the online resource closer to the assessment period. The authors of a recent study assessing the impact of TEL on anatomy learning and assessment outcomes suggests that students may engage with a learning resource for several reasons, such as their own enjoyment or to organize and consolidate their revision. However, they could not show that the TEL resulted in better examination performance. There is also much debate around the link between engagement with the content and activities in curricula and actual outcomes of learning.,

Unfortunately, we could not determine whether the online resources helped the students to retain the information beyond this academic cycle or whether they ultimately impacted their antimicrobial prescribing practices, which is a limitation of this study. Another evaluation later in our curriculum would be interesting but challenging for us to complete as we do not teach them beyond their third year. Only 39 students provided perspectives on the TEL, which is another limitation of this study as it may not be representative of the entire student body. Despite this, the students did feel that the online module would influence how they practise in the future. This again highlights the need for continued training of future prescribers in the appropriate use of antimicrobials.

Future generations of healthcare professionals need to know how to look for and interpret antimicrobial guidelines, which are often online or accessible through a smart device. Case-based TEL may prove very useful in doing this as a clinical context, with feedback and guidance on accessing and using policies, appears to help students make appropriate choices in the antimicrobial management of important infections. Ultimately, this will help preserve these medicines in the future, which is in the interests of patients.

Funding

This study was carried out as part of our routine work.

Transparency declarations

While this research was not directly funded, related research has been funded through a research grant from Astellas (Ireland) and the Heath Research Board (Ireland) (to H. Humphreys). H. Humphreys is also in receipt of research funding from Pfizer (Ireland) and has received payment for providing advice to Pfizer (Ireland), both in an unrelated area. All other authors: none to declare.

Click here for additional data file.