Comparing live lecture, internet-based & computer-based instruction: A randomized controlled trial.

BACKGROUND: Comparing computer and internet based instruction with traditional giving lecture would provide enough evidence to identify best teaching practice. In this study, we compared lecture, interactive internet based and computer based learning regarding medical students' knowledge acquisition and satisfaction in teaching pathophysiology of hematology and oncology.
METHODS: Eighty four medical students were randomized into three groups and an identical faculty member conducted the instructions through the above mentioned methods. Students' knowledge was assessed one week before and immediately after the interventions by pre and posttest. Students' satisfaction was assessed using a validated 5-point Likert scale.
RESULTS: The results showed that students' satisfaction was significantly higher in interactive internet based group than other ones (p=0.05). There were a significant increase between pre and posttest scores in all groups (p=0.000). We used ANCOVA to compare score changes in the study groups, with posttest scores as the dependent factor and pretest scores as covariate and knowledge acquisition was significantly higher in interactive internet based group than other two groups (p=0.026).
CONCLUSION: The study showed that although interactive internet based instruction is a difficult and time consuming method, it is recommended to integrate this method to medical curricula.

RCT Entities:   Population Interventions Outcomes

Introduction

Cancer education is a growing field and has a great impact in patients' outcome. Thus, educators should apply the best teaching practices and methods in the field (1). Computer based learning (CBL), as a learner centered instructional method, refers to the strategies in which computer software is used to deliver educational contents to learners (2). This multimedia based strategy would provide opportunities for enhanced learning (3). Internet-based learning (IBL) is based on internet infrastructures and web applications and is being increasingly used in medical education (4 , 5). In line with that, educators also need documented evidence on CBL effectiveness and application (4). Many studies have compared IBL with traditional teaching either with or without intervention (4 , 6 , 7). Although meta-analysis studies have provided enough evidence for effectiveness of IBL, there is not enough guidance in adopting preferred method of cancer instruction to medical students (8). Thus, for providing this guidance, we compared IBL intervention against another CBL and traditional instruction (4).

In this study, we carried out a randomized controlled trial (RCT) to compare face to face lecture, CBL and IBL methods, regarding medical students' knowledge acquisition and their satisfaction in cancer education. By interactive IBL, an interactive case based scenario was delivered via web. By CBL, a multimedia containing lecture contents was synchronized with the instructor presentation. We hypothesized that interactive IBL and CBL methods will be at least as effective as lecture in increasing students' knowledge acquisition and satisfaction. This RCT is registered and published on the ClinicalTrials.gov public site (identifier No: NCT01269775).

Methods

We carried out the RCT in 2011. Educational Council of Medical School and Medical Ethics Committee of Tehran University of Medical Sciences approved this RCT and written consent was obtained from all students for participating in the study.

Study participants

Learners were third-year medical students and were investigated during pathophysiology of oncology and hematology course. Considering standard deviation of previous students’ exam scores, we needed 18 subjects in each group to have 80% power with a 5% type one error rate. For the purpose of over sampling, 91 medical students were randomly (using the random number table) assigned to 3 groups. Figure 1 shows randomization scheme and participation flow of the study groups.

Fig. 1

Educational design

Students in all groups participated in similar but not identical instructional activities simultaneously and were supposed to achieve the same educational objectives. The same faculty member conducted the instruction for the groups. He had received top scores from students’ evaluation of course lecturers in previous years. The instructed topics were core ones in the medical curriculum and new for third-year medical students (also confirmed by students’ pretest one week before the intervention). The teaching strategy for the lecture group was exactly the same as previous ordinary courses. The faculty member allocated 2 hours for delivering the lecture for each topic with slide presentation and question and answer.

We prepared multimedia CD-ROM for CBL group. The lecture of the faculty member was recorded in a studio and synchronized with the same slide presentation as in his live lecture. The students could either listen to the whole lecture or the lesson’s subtopics separately.

For interactive IBL group, the faculty member developed a case-based scenario for each topic which began with a study guide. Then a case was introduced followed by a multiple choice question. The students were given feedback based on their response to this question. This question and answer pattern continued until the end of the scenario. Learners could have experienced different individualized learning paths, based on their responses to the questions. Also pictures, graphs and guidelines were attached to the content. This e-content was delivered to the students via a Learning Management Systems (LMS). Both CD-ROM and e-scenario were presented to two other oncology and hematology faculty members to verify the coverage of learning objectives.

Satisfaction questionnaire

In order to probe the students’ satisfaction, an author-designed questionnaire was used which consisted of 16 questions. Four factors including "perceived usefulness", "academic performance", "self-regulation", and "service quality" (9)were taken into account. The questions were scored based on 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). The questionnaire was validated by ten experts and its Cronbach’s alpha was 0.89.

Knowledge assessment pre and posttest

This test consisted of 30 multiple choice questions and was presented to two other hematology and oncology faculty members to assess its face and content validity.

Intervention

The pretest was administered to the students two weeks before the intervention. All groups started their sessions at the same time. The lecture was delivered in a class similar to the previous ordinary courses. The two other groups attended in the computer center of Medical School, and for each student it was assigned a computer with either a headphone or the multimedia CD, or the internet access. Technical assistance was provided to these groups when necessary.

The lecture was delivered within two hours as previously scheduled. There was no time limitation for the CBL and IBL groups. The last student completed the learning process in 2.5 hours. The students completed the posttest and satisfaction questionnaire immediately after the intervention.

After the interventions, the CD-ROM, the e-content and the faculty member were accessible for all students in line with the study ethical guidelines.

Analysis

The statistics analyzer was blind to the intervention groups of the participants. Analyses included comparing the scores of pretests and posttests in three intervention groups and comparing the scores of pretest and posttest in each group which were analyzed using ANOVA and pair t-test respectively. Also we used ANCOVA to compare score changes in the study groups, with the posttest scores as the dependent factor and the pretest scores as covariate. Also ANCOVA was used to determine the effect of sex and satisfaction. Statistical significance was set at P<.05 and all analyses were conducted using SPSS version 17.0.0 (SPSS Inc.).

Results

Of 91 students (31 lecture, 30 IBL and 30 CBL), 84 (30 Lecture, 27 IBL and 27 CBL) completed all learning activities, tests, and satisfaction questionnaire (Fig. 1). None of them reported previous familiarity with the topics. Chi-square analysis indicated that the three groups were similar in sex.

Score means of knowledge test for the three study arms are shown in Table 1. There were significant differences between the pre and posttest scores in each group (p=0.000). On the other hand, although there was no significant difference among the scores of pretest in intervention groups (p=0.774), a statistically significant difference was observed among the scores of posttest in these groups (F=3.944, p=0.023). If considering pretest as covariate (ANCOVA) this difference is still significant (F=3.802, p= 0.026). Tukey post hoc test showed that this result is due to the high score of the "interactive IBL" group in comparison with other groups. The sizes of the differences from pretest to posttest were large, representing an increase in knowledge acquisition in all teaching methods (95% CI of difference, 9.2-10.1). We found a significant difference among satisfaction scores in the intervention groups (Lecture = 2.31, IBL = 2.58 and CBL = 1.93 out of 5; p=0.005). Considering sex as covariate had no effect in the above mentioned results.

Table 1

Descriptive data of three intervention groups and the differences of scores

Type		Pretest		Posttest
	N	Meana	Sd	Meana	Sd	Significanceb of difference
Face to Face Lecture	30	4.1	1.6	13.5	1.2	0.000
Interactive Internet Based	27	4.4	1.3	14.3	.91	0.000
Computer based	27	4.1	1.9	13.7	1.1	0.000
Significance b of difference		p= 0.774		p= 0.023

a Means are out of 15.

bThe mean difference is significant at the 0.05 level.

Discussion

Unlike other studies that have compared internet based learning either with no intervention or only with lecture in medical education (4 , 6), we, in this study, compared CBL and IBL methods with traditional lecture. Thus, the results of this study would provide evidence for effectiveness of CBL and IBL in medical education.

This study showed that knowledge acquisition is better in interactive IBL than lecture and CBL methods. Different studies performed in undergraduate or continuous medical education confirmed our results (10 , 11). Also, study participants’ satisfaction was less in CBL group than two other ones. The same was reported in a study in which medical students preferred live lectures to recorded ones (12).

The characteristics of this study that showed the effectiveness of interactive IBL are as follows: 1. The lecture was delivered by a very good lecturer based on students’ evaluation of faculty members’ teaching skill, 2. Although the students in IBL and CBL groups had no time limitation for studying the lesson, other university affairs may have made them to finish their study within maximum 2.5 hours, 3. We attempted to take special care to develop a high quality e-content and multimedia CD which were reviewed and validated by the experts of that field before interventions. This is of great importance according to the literature (13), and 4. We designed the RCT in such a way to avoid the challenges of educational RCTs (7) as much as possible. We allocated participants to intervention groups randomly, there was no contamination among intervention groups, the same faculty member conducted three teaching methods, the knowledge test and satisfaction questionnaire were validated and also were blinded to statistical analyzer, and the power of the study was sufficient to prove the results.

Potential limitations of this study should be considered. One of the advantages mentioned for CBL and IBL, is their flexibility in the time and place of learning (2 , 4 , 14). In this study to prohibit contamination of the groups, we had to start the intervention at the same time for three groups in the school’s camp. However study results showed a higher satisfaction and knowledge acquisition in IBL compared with two other groups.

We did not address participants’ long term knowledge maintenance for two reasons: firstly there may be post interventions cross contamination and secondly the participants’ long term knowledge may not be definitely attributed to our intervention, because of their further study of the topic. Studies that have measured long term knowledge maintenance have addressed this limitation (11).

Conclusion

Finally our results showed that students’ satisfaction is higher in interactive case based IBL than lecture and CBL and interactive IBL is the superior method to CBL and lecture regarding knowledge acquisition. Although developing interactive e-scenarios is a difficult and time consuming task (13 , 15), the results of this study suggests a wider adoption of this strategy in the oncology course in medical schools and recommends the IBL to be integrated with medical curriculum.

Acknowledgments

This research has been supported financially by Tehran University of Medical Sciences & Health Services, grant No. 10764-76-02-89.

Conflict of interest

The authors declare that they have no competing interests.