Introducing E-learning in Epidemiology Course for Undergraduate Medical Students at the Faculty of Medicine, University of Belgrade: A Pilot Study.

The aim of this study was to determine whether e-learning as a new teaching concept was acceptable for second-year undergraduates and to compare attitudes and exam results of students who followed electronic compared with classroom seminars. The electronic seminars (e-seminars) were developed several months prior to start of the epidemiology course for second-year students at the Faculty of Medicine, University of Belgrade. The students who applied for e-seminars accessed their content during summer semester (February-May) 2014. E-seminars were set according to the existing topics in practical workbook and designed using Moodle, a free, open-source, personal home page web application for producing modular internet-based courses. To evaluate the motives for enrollment and satisfaction with seminars, two surveys (pre- and post-course) were administered. Students' exam grades were registered over 4 exam sessions (June-October 2014) and compared according to seminar program. Out of 516 students in the second year, 60 (11.6 %) applied for e-seminars (mean age 21 years). Students considered the reason "It's easier to do assignments from home" as the strongest motive to participate. When compared to classroom seminars, students in e-seminars had significantly more fun (p = 0.003), thought that e-seminars were better mode to learn epidemiology (p = 0.030) and would recommend them to other colleagues (p = 0.001). There was no significant difference in average grade received at the oral exam in epidemiology (t = 0.071, p = 0.944). E-seminars in undergraduate epidemiology course add a novel, easy-to-follow and amusing mode of learning. Based on this pilot study, e-seminars in epidemiology will be available for next generations of students, while further improvement of e-seminars could include expansion of seminar syllabus and development of discussion fora.

Introduction

Over the last decade, online learning, or e-learning, has been increasingly implemented in medical education. It has been suggested that key features of e-learning include self-directed learning, flexibility and daily availability for learners and collaboration within the learning community to ensure further professional development [1]. The concept of e-learning was already adopted by numerious academic institutions, offering various courses both at preclinical and clincial levels [2-5]. Also, a recent expansion of online courses in epidemiology and public health has allowed that these scientific disciplines become available to broader public. Moreover, there is a remarkable number of accredited schools that offer online public health degrees [6]. Introducing e-learning may help the users not only to improve health literacy, but also to motivate them to do own literature search and improve their knowledge on the subject [7, 8].

The first online course at the Faculty of Medicine, University of Belgrade was organized in 1999, at the Institute of Histology and Embryology. The course consisted of additional e-material sent by e-mail, along with conventional lectures and microscopic practicals. However, given that growing number of students showed interest to enroll, the course required a more sophisticated medium for improvement and inclusion of all those who took interest. As of 2004, web-based platform Moodle (moodle.med.bg.ac.rs) has been used as a mean of e-learning in this course, because of flexibility and simplicity for navigation and creation of educational material (interactive concept, dynamic content, links). After pilot testing, the project “Reticulum” (moodle.med.bg.ac.rs) was founded in 2005 and has been used since for various e-courses including Physiology, Pathology and Social Medicine, both for under- and postgraduates at the Faculty of Medicine, University of Belgrade. It is also important to highlight that e-courses at our Faculty are additional tool along with conventional teaching methods, while students are required to follow lectures and practicals (i.e., there is no course that is entirely carried out in electronic form).

To respond to increasing number of courses at the Faculty that offer e-learning material, a novel e-course in epidemiology for undergraduate medical students was developed. The aim of this study was to determine whether this new teaching concept was acceptable for second-year undergraduates and to compare attitudes and exam results of students who followed electronic compared with classroom seminars.

Methods

General layout of the epidemiology course

The course of Epidemiology at the Faculty of Medicine, University of Belgrade is compulsory for medical students in the fourth semester of undergraduate studies (preclinical subject). Students who passed exams in Genetics, Anatomy, Histology and Embryology, Medicine and Society and English language, held in the first two semesters, entered this course. The course lasts for 15 weeks and is composed of lectures (1 class, lasting for 45 min), after which the students continue to join 90 min (2 classes) practicals in the classrooms, performed and supervised by Chair of Epidemiology. According to Faculty bylaws, the same topic lecture is held from Monday to Thursday. All students who follow epidemiology course on a certain day are divided in 5 subgroups. In this way, around 25 students take part in practicals in separate classrooms.

Practicals consist of exercises and seminars. In exercises, students are required to calculate specfic rates, effect measures, and/or answer questions related to study examples. In seminars, students are required to prepare oral presentations, lasting up to 10 min, in Power Point, where they give a talk in front of other students on a topic delegated by the teaching assistant in charge of practicals for that group. All seminar topics are printed in the practical workbook, which means that student has to read through the entire text and present it. The topics are discussed in class after corresponding presentations. Over the semester, there are 4 weeks (a total of 8 classes) dedicated to classroom seminars. At the end of school year each student was required to have prepared and presented one seminar topic.

Development of e-seminars

This was the first online epidemiology course organized for undergraduate medical students in Serbia. Electronic seminars were set according to existing topics in practical workbook and designed using Moodle, a free, open-source, personal home page web application for producing modular internet-based courses. This platform is integrated in moodle2.med.bg.ac.rs (entitled “Reticulum”) which is a part of online domain of the University of Belgrade. All parts of this website are closed and require a password for entering. E-seminar topics were developed over 2013. In January 2014, all the topics were organized and uploaded before the start of epidemiology course in summer semester. All the content was in Serbian language.

Seminar topics are presented in Table 1. All classroom seminars were extracted from national and international published data. For e-seminars, apart from the existing published data, some topics were fictious and set in Serbia (outbreaks of shigellosis, E.coli gastroenteritis, legionellosis, food poisoning caused by V.parahaemolyticus, Syndrome X). Depending on topic, e-seminars were either compact in one block (Disaster epidemiology, Sources of data) or had several different assignments (Table 1). Each assignment was organized in such a way that students were required to read through theoretical background and then move on to answering particular questions. Each question offered three answers, of which only one was correct. Participants were allowed to continue to the following page only after answering correctly to previous question. To help students answer correctly, each question was followed by a selected image related to topic, but not indicative of correct answer. The questions were related to application of various epidemiological study methods such as calculations of ratios, proportions, rates and risk ratios. For some calculations (use of descriptive method in investigation of communicable and non-communicable diseases, vital statistics) students were required to follow the links and find corresponding data in open year books, published online by the Institute of Public Health of Serbia (http://www.batut.org.rs/).

Table 1

Seminar program in epidemiology for second year medical students at the Faculty of Medicine, University of Belgrade

Classroom seminars	E-seminars
1. Investigation of communicable diseases  1.1. Descriptive study (overall rates, HIV and TB)  1.2. Case–control study (outbreak of cholera in Nigeria, 1996)  1.3. Retrospective cohort study (food poisoning with S. aureus, on a wedding in Serbia, 2005)  1.4. Experimental study (vaccine against HPV, 2006)	1. Investigation of communicable diseases  1.1. Descriptive study (using online year books)  1.2. Case–control study 1 (shigellosis in a restaurant)  1.3. Case–control study 2 (gastroenteritis caused by E.coli)  1.4. Retrospective cohort study 1 (legionellosis on a cruise ship)  1.5. Retrospective cohort study 2 (food poisoning with V.parahaemolyticus at the financial meeting)  1.6. Retrospective cohort study 3 (food poisoning with S. aureus, on a wedding in Serbia, 2005)  1.7. Retrospective cohort study 4 (SARS outbreak on a plane, 2003)  1.8. Experimental study (vaccine against HPV, 2006)
2. Investigation of non-communicable diseases  2.1. Descriptive study (multiple sclerosis)  2.2. Case–control study (exposure to cow milk in early childhood and risk of diabetes, Serbia 2001)  2.3. Prospective cohort study (education level and risk of malignant and cardiovascular diseases, Sweden 2001)  2.4. Cross-sectional study (health status in the population of Serbia, 2002)  2.5. Experimental study (exposure to calcium and risk of prostate cancer, 2005)  2.6. Tobacco as a risk factor for onset of non-communicable diseases	2. Investigation of non-communicable diseases  2.1. Descriptive study (using online year books)  2.2. Case–control study (Syndrome X, fictional example)  2.3. Prospective cohort study (Syndrome X, fictional example)  2.4. Experimental study (Syndrome X, fictional example)
3. Burden of disease  3.1. Calculation of YYL, YLD and DALY  3.2. Burden of disease and injury in Serbia part 1  3.3. Burden of disease and injury in Serbia part 2 (2003)	3. Burden of disease  3.1. Calculation of YYL, YLD and DALY  3.2. Burden of disease and injury in Serbia part 1  3.3. Burden of disease and injury in Serbia part 2 (2003)
4. Sources of data  4.1. Population data (census)  4.2. Morbidity data (theoretical framework)  4.3. Mortality data (theoretical framework)	4. Sources of data  Population, morbidity and mortality data with copies of official of forms (interactive, using website of the latest census in Serbia)
5. Environmental epidemiology  5.1. Air pollution and occurence of childhood asthma (2006)  5.2. Pesticide exposure and risk of congenital maformations (2006)  5.3. Climate change and health (2003)	5. Environmental epidemiology  5.1. Exposure to radon and risk of lung cancer (2012)  5.2. Climate change and health  5.3. Environmental pollution (dumping site in Kenya)
6. Disaster epidemiology  6.1. Overall disasters  6.2. Bioterrorism and biological weapon  6.3. Tsunami  6.4. Avian flu	6. Disaster epidemiology  Overall disasters and example of earthquake in Italy, 2009

Legend: HIV human immunodeficiency virus. TB tuberculosis, HPV human papilloma virus, SARS severe acute respiratory syndrome, YYL years of life lost, YLD years lived with disability, DALY disability adjusted life years

All topics were displayed in the main menu. However, only the scheduled seminar topic was accessible during 1 week, while others were visible but students were not able to access them. Students were required to answer all the assignments in the given topic to be able to validate the seminar.

Selection of participants in e-seminars

A total of 516 students were enrolled in the fourth semester in February 2014 and attended epidemiology course. During the first practical, teaching assistants explained to students in detail the course contents. Students had 5 weeks to consider whether or not to take participation (while they practiced health measurements and application of epidemiological methods). Those who applied for e-seminars were exempt from coming to classes when corresponding seminars were scheduled. To join e-seminars the students had to create own username and password.

Evaluation

To evaluate the motives for enrollment and satisfaction with the e-seminars two short questionnaires were administered. The first (pre-course survey) was related to several motives for enrollment and their initial expectations. Students had to answer this survey to acess the first seminar topic. The second (post-course survey) was related to overall satisfaction and impressions from the course. We had two version of the post-course survey: one for those who participated in e-seminars and one for those who were in classroom seminars. Students in e-seminars were required to answer this survey before they accessed the last seminar topic. This means that completion of all e-seminars included answering to post-course survey. Students in classroom seminars were randomly assigned post-course survey in the last class of the semester. Each answer was rated on a 5-point Likert scale, where rank 1 denoted “I strongly disagree” and rank 5 denoted “I strongly agree” with a given statement. To assess differences in the second survey, marks 4 or 5 on Likert scale were considered as positive attitude for the given statement. To measure potential difference in attitudes of students in e-seminars as opposed to classroom we performed Chi-square and Fisher’s exact test.

Because 40 % of students in e-seminars did not fill in the post-course survey, we sent them an e-mail to explain as to why the survey was not answered.

Follow-up

After completion of the course, students are required to take oral exam in epidemiology, which accounts for 70 % (maximum of 70 points) of the total grade. Oral examination consists of answering three questions printed on hand-size cards that students choose themselves out of a pile of cards turned on their backsides (in order not to see the questions before students pull out a card). The remaining 30 % (maximum of 30 points) of the total grade consists of points received in the written quizz in mid-semester (27 points) and points received for the oral presentation of seminar topic (3 points). Students who opted to follow e-seminars received seminar points based on whether or not they completed the topic scheduled for the corresponding week of the course program. In epidemiology exam sessions for 2014, students were examined by one out of seven professors (three full professors and four assistant professors). Main exam sessions after pilot e-seminars were June, July, September, and October, 2014. According to Faculty bylaws students are not obliged to present themselves at the exam to be able to enroll in the 3rd study year. Those who opt to take oral exam in epidemiology later may take it the following January at earliest. We registered grades of students who presented at the examination from June to October 2014 and compared their final grades in relation to type of seminars by means of t-test. According to grading policy of the University of Belgrade, students’ exam achievements when passing the oral examination are graded from 6 (51–60 points) to 10 (91–100 points).

Ethical approval for the study was obtained from the Institutional Review Board of the Faculty of Medicine, University of Belgrade.

Results

Out of 516 students who were enrolled in the second year, 60 (11.6 %) applied for e-seminars. Mean age of participants was 21 years. Scores of pre-course survey are presented in Table 2. Students considered the reason “It’s easier to do assignments from home” as the strongest motive to participate in e-seminars. They also had considerably high expectations from e-seminars, as all given statements were highly rated (Table 2).

Table 2

Median scores of students who applied for electornic seminars according to reasons for enrollment and expectations from the course

Item		Median	Interquartile range	Skewness	Kurtosis
Reasons for entering the course
#1	To better learn epidemiology	4.0	1.0	−1.60	3.13
#2	To avoid coming to classes	4.0	3.0	−0.51	−1.36
#3	It’s easier to do assignments from home	5.0	1.0	−0.81	−1.39
#4	To try a new mode of learning	4.0	1.0	−1.76	4.06
#5	To finish the assignments faster	4.0	2.0	−1.27	1.63
Expectations from the course
#1	To better learn the course material	4.0	1.0	−0.40	−0.89
#2	To have fun while learning	4.0	1.0	−0.52	−0.66
#3	To pick up and see something new	4.0	1.0	−1.76	6.06

1 I strongly disagree, 5 I strongly agree

At the end of semester, 36 students (60 %) completed the post-course survey (i.e., completed all e-seminars). Post-course survey was distributed to 149 randomly selected students in classroom seminars during last week of semester. We observed that students in e-seminars had statistically significantly more fun while working on seminars, thought that e-seminars were better way to learn and that they would recommend it to others (Table 3). Moreover, all students who attended e-seminars would recommend them to their colleagues. The lowest proportion of positive answers in post-course survey was observed for item “I feel ready to take oral exam in epidemiology soon”. We noticed that there was no statistically significant difference in relation to exam preparedness between the two groups (Table 3).

Table 3

Proportion of students who agreed or strongly agreed with given statements

Item		% students with positive attitude		p
		Electronic seminar	Classroom seminar
#1	Epidemiology seminars fulfilled my expectations	34/36 (94.4)	140/149 (94.0)	1.000
#2	After these seminars I understood epidemiology better	35/36 (97.2)	132/149 (88.6)	0.206
#3	I had fun while working on seminars	32/36 (88.9)	94/149 (63.1)	0.003
#4	This mode of learning is much better than the other one	34/36 (94.4)	117/149 (78.5)	0.030
#5	I would recommend this type of seminar to other students	36/36 (100.0)	113/149 (75.8)	0.001
#6	I feel ready to take oral exam in epidemiology soon	21/36 (58.3)	67/149 (45.0)	0.150

Because 24 students did not complete the post-course survey, we sent them emails to explain the motives for non-compliance. Of those, only 6 (25 %) replied. Three of them explained that they could not complete e-seminars because of June exam session, while other three students asserted that other non-faculty obligations resulted in skipping the last e-seminars.

From June to October 2014 exam session of 456 students who attended classroom seminars 183 passed the final oral exam in epidemiology (40.1 %). Similarly, 38.3 % of students who attended e-seminars passed this exam (23 out of 60). There was no statistical difference in proportion of students in these two seminar programs who passed the final exam (χ 2 = 0.071, p = 0.789). When average grades were assessed among students who passed the exam, we found that students in classroom seminars received an average of 9.33 and students in e-seminars obtained 9.35 (t = 0.071, p = 0.944).

Discussion

In this pilot study, we evaluated a novel e-learning program in undergraduate epidemiology course at the Faculty of Medicine, University of Belgrade. In the current body of literature, to our knowledge, there are no reports on assessment of e-learning in Serbia. Although e-learning concept at the Faculty of Medicine in Belgrade has been established and well-elaborated in Histology and Embryology course over the past decade, it was applied for the first time in 2014 in undergraduate epidemiology course. Our pilot study indicated that, prior to entering e-seminars, students had considerably high expectations from the course in terms of overall learning as well as experiencing novel methods of learning while being at ease. We also observed that process of learning, for our students, is the most convenient from own home, which accounted for the highest rank in the scale. Various in- and out-of-classroom techniques have been developed for teaching not only epidemiology, but for almost all medical branches as well [9-11]. Given the rise of communication technology, diverse distance learning courses in epidemiology have targeted both developing and developed countries, particularly when academic centers are not within easy reach [9]. Introduction of e-seminars in our undergraduate epidemiology course has also included the use of additional websites in search for appropriate data. In this manner, students were encouraged to step out of their classical learning practice, based on readings of recommended textbooks and reproduction of theoretical frameworks. Incorporation of web-based modules in epidemiology could demonstrate the relevance and presence of epidemiological data on the world wide web. Because internet represents an important source of health-related information to medical students [12], it is important to include e-based resources in epidemiology curriculum.

While working on e-seminars, students were required to be active participants and self-reliant, as opposed to students in conventional seminars, who were required to attend oral presentations while were sitting in the classroom. Although classroom seminars offered possibilities for peer interaction, we did not determine a degree of knowledge retention at the individual level in this group. Taking into account students’ impressions in the post-course survey, we believe that through these e-seminars we achieved the goal to present epidemiological topics in a well-balanced amusing and studious manner. Nevertheless, there was no difference in perception of readiness to take oral exam in the forthcoming exam session between the two groups. It has been highlighted that students by-and-large have difficulties understanding the key principles underlying epidemiological concepts [13]. Indeed, over the second year of undergraduate studies in medicine, students tend to be more oriented towards major preclinical subjects, such as physiology and biochemistry, which, according to Faculty bylaws, students are obliged to pass in order to enter the 3rd year of training. By contrast, students are not obliged to either take or pass exam in epidemiology before entering the next year of training, which likely affects students to feel at-ease when considering whether or not to take this exam during the ongoing Faculty semester. Again, another important aspect attributed to obstacles in adopting basic epidemiological principles is the “population-level thinking” [13], contrary to patient-oriented medicine, which seems to represent the greatest barrier in comprehension of study material.

Additionally, some authors have reported that after introduction of e-learning courses participants do not demonstrate significant improvement in knowledge compared to classical teaching methods [2, 14]. In this respect, students’ perception of being prepared for oral exam as well as the final grade they received after the exam could be considered as proxy for knowledge gain. The lack of significant knowledge gain in e-seminar group could have resulted from a small number of classes dedicated to seminar topics as detemined by the Faculty program, because approximately just one sixth of classes in the entire semester is assigned to epidemiology seminars. It is possible that further development of topics and more practical examples could lead to better subjective feeling of readiness to take the final exam. However, even though neither difference in knowledge gain between the two seminar groups was significant nor was the average grade at final exam, we believe that introduction of new learning module in epidemiology course has been justified given the expectations and overall impressions after completion of seminars.

In interpretation of e-seminar completion rate of 60 % several important factors need to be taken into consideration. The last two seminar topics (environmental and disaster epidemiology) were set for the last week in semester (May 26–31, 2014). However, in the third week of May 2014, record-breaking rainfall severely hit numerous municipalities across Serbia. High intensity floods and subsequent landslides resulted in total destruction of households, bridges and sections of roads both in rural and urban areas, causing estimated 1.8 billion Euros damage and loss. These extreme weather conditions affected around 1.6 million people of which 32,000 were evacuated from homes and lodged in temporary camps established by the Government and the Serbian Red Cross [15]. Because of this, Universities across country were closed and final lectures in the semester were postponed to beginning of month of June 2014. In response to this disaster many medical students were engaged in various emergency-related activities including collection of clothes, food and item personal hygiene products. It is likely that these emergency events as well as the start of exam session (following the end of lectures) caused 40 % of students not to complete e-seminars and subsequently not respond to post-course survey. As a result, answers given in the post-course survey may not entirely represent students’ impressions at the end of the course. Another limitation of our study is related to selection of participants for e-learning course, since students were not randomly selected, but rather they decided on their own whether or not to participate. Such selection could have favored those students who are more versatile with web browsing and those who own a personal computer.

Conclusion

In conclusion, e-seminars in undergraduate epidemiology course at the Faculty of Medicine in Belgrade seem to add a novel, easy-to-follow and amusing mode of learning. This, by no means, suggests that e-learning could substitute traditional lectures in our course. Instead, use of information technology has the aim to improve understanding of basic epidemiological principles and to enhance retention of acquired knowledge. Based on this pilot study, e-seminars in epidemiology will be available for next generations of students, while further improvement of e-seminars could include expansion of seminar syllabus and development of discussion fora.