Analysis of Web-based learning methods in emergency medicine: randomized controlled trial.

INTRODUCTION: In medical education, Web-based learning is increasingly used as a complement to practical classes. The objective of the study was to perform a comparative analysis of three different forms of e-learning course on emergency medicine in terms of an indicator of knowledge growth and students' satisfaction.
MATERIAL AND METHODS: For the purpose of the study, we developed and implemented a tool in the form of an online course: A - non-animated presentation, B - video, C - interactive video. The participants were undergraduate students of emergency medicine and nursing (n = 106). A pre-test and a post-test were carried out, and the resulting data were analyzed using parametric tests (t-test, ANOVA, post-hoc). Final questionnaires assessing six parameters of satisfaction were also evaluated.
RESULTS: A significant increase in knowledge in the experimental group which used an interactive video was observed (p = 0.04). Moreover, the number of replays of learning material was the highest in group C (2.09 ±2.48). The level of post-course satisfaction turned out to be comparable in all three groups (p = 0.62).
CONCLUSIONS: The study allowed the interactive course to be identified as the most effective method of distance learning among selected ones. Due to the limitations of the study, we conclude that there is a need for further studies on the effectiveness of e-learning in emergency medicine.

Introduction

Distance learning is currently undergoing intensive development and is widely used in the private and commercial sectors and in continuing education as well. Modern information technologies allow the use of innovative educational solutions online and telemedicine [1]. People are increasingly less likely to talk face-to-face, sending short text messages and graphics more frequently and using online multimedia publications in the form of podcasts [2].

e-Learning courses have become common-place, because they allow savings on costs of training materials, travel, and training room rental and reduce instructors’ working time [3]. When teaching practical subjects, e.g. resuscitation, so-called blended learning is more commonly used, in which students gain theoretical knowledge during online self-study while exercises are being performed using traditional methods together with the teacher [4]. There are numerous studies showing that e-learning is similarly or even more effective in comparison to traditional teaching [5]. Because the effectiveness of distance learning has already been proved, among the currently available techniques of e-learning one should search for the most effective one that satisfies the participant.

Education utilizing multimedia computers or portable computing devices enabling access to materials posted on a website is referred to as Web-based learning (WBL). During WBL one uses both synchronous mode real-time learning and asynchronous mode education at any time and any place, without the real participation of a teacher [6]. Digital resources are increasingly being used in higher education, including medical sciences. During self-study, students show more interest in acquiring knowledge from the Internet than from textbooks, especially through mobile applications [7]. Increasingly, different medical information is being sought; however, it does not always appear to be a reliable educational resource, particularly in the publicly available and free resources [8]. It is therefore necessary to care about high quality of educational resources shared online and prepare students for the changes likely to occur in the medical curriculum and in the marketplace [9].

There are few studies on the effectiveness of e-learning in emergency medicine, and for this reason, in the present study we attempted to analyze the effectiveness of three different forms of distance learning among students of emergency medicine and nursing. The objective of the study was to develop an e-learning tool which could significantly increase the indicator of knowledge growth among medical students, and which could also be assessed positively in terms of usefulness, participants’ satisfaction, ease of use, and substantial and technical quality [10].

Material and methods

Participants and enrollment procedure

The study was conducted in 2015 between April and June at the College of Health Sciences in Żyrardów (Poland). Participants (n = 106) were part-time undergraduate students of emergency medicine and nursing. Participation in the experiment was voluntary and students confirmed the willingness of anonymous participation by signing a consent form. Each participant was randomly assigned an individual number for the purpose of identification in the further course of the study.

Among the areas determining the effectiveness of e-learning courses, we could distinguish external factors (educational materials, forms of knowledge transmission, technologies) and internal factors (participant’s predisposition). In order to ensure the objectivity of measurements, verification of participants’ internal factors was made [11]:

level of knowledge (W),

declared level of competence in using information and communication technologies (ICT),

declared experience in e-learning environment (ELE),

declared interest in the field of the course (ZAI),

declared level of motivation to learn (MOT).

The level of knowledge of participants was determined based on a test (PRE-TEST) containing 30 single-answer questions with four variants of answers. Filling in the tests was supervised by a teacher ensuring self-reliance of students.

Electrotherapy included in the activity of emergency medicine performed by paramedics and nurses of the Polish Emergency Medicine System constituted the main research area.

Internal factors were analyzed by a five-point Likert scale: “Strongly disagree” (–2), “Disagree” (–1), “Do not know” (0), “Agree” (1) and “Strongly agree” (2). Among the participants’ predispositions, ICT and MOT were considered as crucial [12], and therefore the participants were excluded from further study as the average in these two aforementioned factors was lower than zero on the –2/2 scale.

After the preliminary verification, students enrolled in the study were randomly assigned to one of the following three groups: A – control group, B – first study group, C – second study group. Randomly assigned groups were compared using the t-distribution taking into account the internal predispositions consisting of 17 points: W – 1 point, ICT – 6 points, ELE – 1 point, ZAI – 5 points, MOT – 4 points. In none of the internal factors analyzed were there observed statistically significant differences between groups at p < 0.05.

Research tool

Three instructional materials were prepared according to the asynchronous WBL model, which is different in terms of the multimedia levels and interactivity. Each course was made available online in the “Zaption” system, which is an open source tool enabling the creation of e-learning courses, posting videos, adding interactive elements and analysis of participants’ progress. The minimum requirement for the user’s computer is a web browser, video player and soundcard.

The introductory part in each of the three forms of the course concerned motivation of the participant, outlining the benefits of implementing the proper procedure in cardiac arrest [13]. Course A constituted a presentation consisting of 45 slides containing text, tables, colorful graphics and pictures. No sound or motion graphics were used. Participants could return to selected slides using the navigation keys. Course B constituted video material containing slides, animations of ECG, demonstrations of medical operations performed by instructors and audio commentary of the narrator [14]. The film lasted 16 min and 30 s and had smooth navigation, a pause function and volume control. Course C was extended by the Learning By Asking (LBA) system to assess its learning effectiveness in comparison to non-interactive methods [15].

For this reason, we prepared video material included in course B, but extended by interactive elements requiring active participation during video replay. Twenty single-choice questions were developed with three variants of answers interrupting the replay, forcing the student to make a decision (Figure 1). After addressing the questions, the student immediately received the feedback with a comment. The time for addressing the question was unlimited.

Figure 1

Asynchronous Web-based learning (WBL) course with the Learning By Asking (LBA) interactive system

During replay, the Zaption system showed the previously selected answer, enabling its change so that all of the tasks could be solved correctly and remembered. The navigation panel enabled smooth scrolling of the film, volume adjustment, pause and switch to any questions in the video material.

Course of the study

When filling in the questionnaire and pre-test, each student received an individual participant number and web address in the process of random draw.

Moreover, respondents provided e-mail addresses to contact the teacher during the course. After the enrollment procedure and verification of internal factors, 103 individuals were enrolled in the study by random assignment to one of the three groups. On participants’ websites, randomly chosen courses were made available for 7 days.

Students could use educational materials many times, at any time of day or night. A week before the final test, access to the course was closed to ensure comparable conditions for the formation of declarative memory of individuals under study. The POST-TEST and questionnaire on course satisfaction were carried out in the traditional class in the presence of the teacher.

Statistical analysis

Based on the questionnaire and pre-test, demographic characteristics of students, self-assessment of internal factors and the level of knowledge within the scope of the course were defined. For further analysis, we selected the participants who met the inclusion requirements and who logged into the course at least once during its availability (n = 94). Comparison of study groups in terms of knowledge growth and satisfaction was performed by analysis of variance (ANOVA) and multiple comparison analysis (post-hoc) [16]. Satisfaction with the e-learning course was examined using six questions, the possibility of addressing the questions using a five-point Likert scale [17] and one open question allowing the student to comment freely. The POST-TEST was carried out analogously to the preliminary test, modifying the questions so that they were not identical but showed similarity. The development of knowledge was calculated from the following formula [18]: W = [(PRE – POST)/(MAX – PRE)] × 100%, where W – growth in the knowledge indicator, PRE – test result before the course, POST – test result after the course, MAX – maximum number of points possible to gain.

Data related to the number of unique logs and replays, hours of activity, and technical information of web browsers used were obtained from the Zaption platform and Gostats.com – publicly available supporting statistics.

Results

Users

The study consisted of 106 students, including students of emergency medicine (n = 72) and nursing (n = 34), 35 women and 71 men with an average age of 30 ±7.1 years. Demographic data show that 72.6% of respondents live in an urban area, and the vast majority (96.2%) have regular access to the Internet. From the selected 24 nurses and 70 paramedics (n = 94), we identified three groups which accounted for A = 30 (8 nurses and 22 paramedics), B = 32 (8 nurses and 24 paramedics), and C = 32 individuals (8 nurses and 24 paramedics). On course websites, we registered a total of 194 unique visits, usually in the afternoon (Figure 2).

Figure 2

Hourly distribution of participants’ activity

We analyzed 111 computer systems used by participants in unique sessions. The most commonly used operating systems were Windows 8 (43.2%) and Windows 7 (25.2%). We also reported logins from systems such as Android (12.6%), Windows 8.1 (10.8%), Linux (2.7%), Windows XP (1.8%), Windows Vista (1.8%), iOS (0.9%) and Mac OS (0.9%). Among four browsers used, Chrome with a resolution of 1366 × 768 was used in 75.7% of cases and browsers such as Firefox (15.3%), Safari (8.1%), and Internet Explorer (0.9%) were far less common. Cookie files and Java applets were accepted in 108 and 94 browsers, respectively.

Satisfaction

There were no statistically significant differences (at p < 0.05) between groups when evaluating the average satisfaction after completion of the course (Table I).

Table I

Results of ANOVA for satisfaction of study participants

Average satisfaction
Groups	N	Sum	Average	Variation
A	30	46.00	1.53	0.21
B	32	46.33	1.45	0.22
C	32	50.00	1.56	0.26
Analysis of variance
Source	SS	df	MS	F	P-value	F-test
Between subjects	0.23	2	0.11	0.49	0.62	3.10
Between measures	21.09	91	0.23
Total	21.31	93

Adopting Student’s t-test to compare each of six variants of answers individually demonstrated no statistically significant differences. Respondents who had to choose one out of five values (from –2 to +2) ever assigned a negative value. The highest average in each question was obtained in group C, while the lowest was reported for group B (Table II). The same average (1.63) was observed for groups B and C for the question concerning the ease of use of the course (PEUO), as indicated in the table with bold font. Noteworthy is the fact that the lowest scores in all three groups were assigned for teaching effectiveness (PTE), as underlined in Table II.

Table II

Average evaluation of participants’ satisfaction

A	1.57 ±0.50	1.57 ±0.50	1.63 ±0.49	1.53 ±0.51	1.50 ±0.51	1.40 ±0.62
B	1.53 ±0.51	1.50 ±0.51	1.47 ±0.57	1.41 ±0.56	1.44 ±0.50	1.34 ±0.55
C	1.66 ±0.48	1.69 ±0.47	1.63 ±0.61	1.63 ±0.49	1.63 ±0.49	1.53 ±0.57

Knowledge

Students replayed teaching materials 1.79 times on average, within each group: A – 1.53 ±1.55, B – 1.72 ±1.80, C – 2.09 ±2.48. In the vast majority of cases, the percentage indicator of knowledge growth was a positive value, and only in 12 cases did it turn out to be negative. In one-way ANOVA, a statistically significant difference between the analyzed groups was observed (Table III). Moreover, a statistically significant difference (p = 0.03) in knowledge growth between nurses (W = 38.18%) and paramedics (W = 27.16%) was reported.

Table III

ANOVA performed for indicator of knowledge growth among study participants

Indicator of knowledge growth
Groups	N	Sum	Average	Variation
A	30	7.79	0.26	0.07
B	32	7.49	0.23	0.08
C	32	12.77	0.40	0.09
Analysis of variance
Source	SS	df	MS	F	P-value	F-test
Between subjects	0.50	2	0.25	3.26	0.04	3.10
Between measures	7.01	91	0.08
Total	7.51	93

The highest average (39.92%) was obtained for the students from group C containing an interactive movie, and the lowest (23.40%) for those from group B containing a movie without interactivity. In order to determine the inter-group relationships, post-hoc analysis was performed and as a result a statistically significant difference between groups B and C at p < 0.05 was observed (Table IV).

Table IV

Post-hoc analysis performed for indicator of knowledge growth among study participants

Groups	A	B
B	0.70	–
C	0.05	0.02

Discussion

The objective of the study was development, implementation and evaluation of an e-learning course in the field of electrotherapy emergency. Teaching and learning materials were divided into four chapters: automatic external defibrillation, manual defibrillation, cardioversion, and electrostimulation. Workflow was based on the guidelines of the European Resuscitation Council and consulted with specialists in emergency medicine. The open source platform on which the courses were implemented was chosen based on the ease of use for the user and technical quality as well.

In the vast majority of cases, students used computers with the most recent Windows and Linux operating systems, and currently Android for example is the most widespread system dedicated to mobile devices [19]. In individual cases, one reported logging on to a particular course even from three different devices, which may indicate the continuation of studies beyond the place of residence of the respondents.

A total of four e-mails regarding the lack of access to the course were received. Technical problems were solved by recommending a change in the settings of web browsers. Statistics have shown that not all browsers accepted cookies and Java applets, so this fact can be considered as a possible cause of the occurrence of errors. The analysis of hourly distribution of activity of course participants confirms that one of the main advantages of e-learning is elasticity of the study period [20] as evidenced by the registered views even during late-night hours. Taking this into account, it can be concluded that the implementation of an e-learning course was successful.

Participants’ satisfaction was comparable in all groups; however, the indicator of knowledge growth was significantly different. The best result was achieved by students from experimental group C, who studied with interactive video. In this group, the most frequent replays of the course were also reported, which highlights increased interest among participants. This confirms the experience of other authors, who claim that people learn better when they independently discover knowledge and control the speed of learning [21]. The lowest growth in the knowledge indicator was demonstrated by participants from experimental group B, who learned from a non-interactive video. This result is 3% worse than the result obtained in control group A, in which students absorbed knowledge from slides without any elements of animation, sound or interactivity.

Probably, when replaying an instructional video, not all information contained in it was saved because of the lack of pauses and replays stimulating concentration. Therefore, one does not confirm the result observed by other authors which showed that the use of materials containing sound, graphics and animation causes increased knowledge absorption among medical students under study [22]. The statistically significant difference in the increase of knowledge absorption in a group of nurses and paramedics suggests the possibility of the occurrence of a confounding factor. However, even the distribution of nursing students ensured by randomization reduces the likelihood of such an effect.

The final questionnaires also provided interesting results. Among six sections of questions on the course, students gave the lowest scores for the effectiveness of teaching even in group C, where the progress of knowledge was reported to be the greatest. Among study participants, the low sense of efficacy in acquiring knowledge through e-learning is striking. When analyzing the questionnaires, attention is drawn to the fact that the proportion of self-evaluation of teaching effectiveness in particular groups is relative to the proportion of the obtained results from the POST-TEST. It proves a directly proportional relationship between actual progress of knowledge and self-evaluation of study participants. During further studies, it is worth assessing whether the inclusion of the post-test in an e-learning course online with feedback option, or the implementation of “blended learning”, would improve the sense of effectiveness of learning among participants, as evidenced in the literature [23].

Students gave only two comments when filling in the questionnaire: “More activating questions, and additionally the view of a patient” and “problems when trying to re-play”. In addition to the technical problem, we also suggested the expansion of the interactive course, which should be considered in further studies. In the literature, we could find evidence for positive intensification of the learning process for medical professionals when using dynamic learning environments, such as non-computer games [24].

The conducted study provides some empirical evidence on the importance of interactivity in WBL methods. The experiment was limited to a short-term course on electrotherapy in emergency medicine. The study group consisted of students of medical sciences, and was quite carefully selected in order to avoid the impact of factors determining the process of learning. We failed to avoid technical problems, which is one of the major barriers in e-learning. However, we implemented well-designed and standardized educational material at low costs of implementation [25]. Despite the promising results indicating the superiority of interactive forms of WBL, further studies are needed to eliminate the restrictions, which may affect the final result. Higher finances would allow to improve the quality of technical tools. The study group should be more numerous and include students of other medical faculties.

Undoubtedly, e-learning enables fast, efficient, and cost-effective creation and delivery of teaching materials [26]. However, the form of content transfer is extremely important; it needs to be tailored to participants’ needs. Interactive online courses could play a role in promoting health education programs and connect schools to current social reality [27]. The satisfaction of study participants estimated at 1.51 points on average (on a scale from –2 to 2) confirms the conclusions made by authors claiming that e-learning is not currently treated as a replacement of traditional forms in medical education, but as a complement to them, providing many research possibilities [28].

In conclusion, the conducted study confirms that individuals who possess appropriate computer skills and motivation show significant progress in learning when using interactive e-learning methods. In this method, there were also reported the most frequent replays of the course, which highlights increased interest among participants The lowest growth in the knowledge indicator was demonstrated by learning from non-interactive video. Non-animated presentations, audio-visual films and interactive video used in the context of asynchronous Web-based learning are not significantly different in terms of perceived satisfaction among medical students. Students benefit from Web-based learning usually in the afternoon and evening. There is a need for further objective research to determine the most effective method of distance learning.

Conflict of interest

The authors declare no conflict of interest.