Comparing the effectiveness of blended learning and traditional learning in an orthopedics course.

INTRODUCTION: The ongoing COVID-19 pandemic is forcing medical schools to replace substantial parts of the traditional lecture method with online formats to maintain social distancing guidelines and reduce face-to-face contact in the classroom. To our knowledge, there have to date been few studies comparing the effectiveness of traditional teaching with blended teaching based on the students' final grades which the efficacy of online learning is still controversial, and this study aimed to compare the efficacy of blended teaching with conventional teaching in an orthopedics course.
METHODS: This study was a retrospective cohort study based on data collected from fifth-year medical students between April 2019 and March 2021. The students were divided into two groups which based on years of study. The summative assessment was based on summing the MCQs plus KFs, the MEQ plus oral exam, OSCE, simulated patient chart reviews, and OPD work. All students took the same end-of-course quizzes with no differences between the groups regarding the kinds of knowledge tested. The results of these quizzes were used to compare the effectiveness of the conventional teaching in 2019 and the blended teaching in 2020. The paired t-test was used to analyze the data.
RESULTS: A total of 252 students were enrolled in the study, of whom 128 and 124 students were in the traditional teaching group or blended teaching groups, respectively. The grade point averages of the students were 3.2 ± 0.4 and 3.3 ± 0.4 in 2019-2020 and 2020-2021, respectively, without significant difference (p-value = 0.06). The scores in the blended learning group were higher than in the traditional learning group in all assessment tools (MCQ, KF, Oral, and OSCE scores) except the MEQ.
CONCLUSIONS: Blended learning was not less effective than traditional learning for teaching medical students.

Introduction

The ongoing COVID-19 pandemic is forcing medical schools to replace substantial parts of the traditional lecture method with online formats to maintain social distancing guidelines and reduce face-to-face contact in the classroom. The most common teaching method traditionally in medical education is the lecture. The advantage of a lecture is that it is able to transfer information from medical educators to a large number of students at the same time, and it is a simple and effective way of conveying factual information core knowledge, explaining difficult concepts, generating increased learning, increasing student engagement, and activating self-directed learning [1]. However, in recent years computers and other electronic tools have been increasingly used in all types of education including medical education, and electronic learning has been found to meet the needs of medical students [3].

The major advantage of online learning is that students can learn anywhere and anytime. There are also several disadvantages of online learning, however, such as the lack of social support, especially peer support, and the lack of physical visibility of the instructor [4,5]. While online learning has been increasing, traditional classroom instruction in the form of didactic lectures has declined in recent years, facing criticisms such as failing to promote engagement and being less effective due to its one-way communication basis.

Blended learning has various definitions in the previous literature, and there is no widely accepted absolute definition [6]. It was defined as a combination of traditional and online learning was defined by Graham [7], while Lotrecchiano et al. [8] gave the definition as a blend of structured and unstructured learning. For medical students, the clinical skills are a necessary part of their education, which cannot be usefully taught online, and therefore, when traditional lectures are limited, blended learning is crucial for medical students. The effectiveness of online teaching is still controversial, with some studies [9,10] examining the effectiveness of online teaching compared with offline teaching and finding that the conclusions did not yield consistent results. The most recent systematic review and meta-analysis study was inconclusive in comparison of the acquisition of knowledge and skills between traditional learning and online learning [11]. Therefore, the aim of this study was to compare the effectiveness of traditional teaching and blended teaching based on the students' final grades. Our hypothesis was that there would be no differences between the methods.

Methods

Design and study participants

This was a retrospective-cohort study performed at the Department of Orthopedics of a medical center in South-East Asia based on data collected from the department's fifth-year medical students between April 2019 and March 2021 to compare the efficacy of blended teaching and conventional teaching. The study was approved by our institutional review board (REC 64-251-11-1) and registered with the Thai Clinical Trials Registry (TCTR 20211004006). This study was done following the STROCSS criteria [11]. Normally, our department teaches a one-month course in Health, Disease, and Rehabilitations in Orthopedics which is broken down into two parts. The codes of this course are 388–571, and 388–572. In 2019–2020, the course consisted of sixteen conventional lectures delivered by face-to-face instruction in the classroom, as well as other activities such as outpatient clinics, in-patient clinics, attending/assisting in the operating room, interactive meetings with case-based discussions, and practicing of basic orthopedics skills.

However, the COVID-19 pandemic began in 2020, peaking in March and April, and although the medical students began their new semester on May 1st as normal, new teaching practices were developed to ensure the safety of the students. Our faculty education team divided the 5th year medical students into 12 rotations: pediatrics, surgery, internal medicine, obstetrics-gynecology and family medicine for two weeks each, and orthopedics and emergency for one week each, and instructed every department to do online learning whenever possible. Each department was in charge of its own online course. Our orthopedics department decided on a blended learning approach, beginning with a one-week online course, followed by three weeks of traditional training in the outpatient clinic, in-patient clinic, and operating room.

The orthopedic online course included a variety of activities over the one-week period, including 16 online lectures, live meetings with medical educators through the zoom application, and shorter segments such as traumatic film interpretation, upper limb disease case-based discussion, lower limb disease case-based discussion, and an assignment about spine disease. Live meetings activities were developed in order to engage our students to access the online course and also to encourage social support from their medical instructors and peers. These activities comprised one and a half hour per sessions from Monday to Thursday. The onsite learning included the necessary face-to-face practices such as out and inpatient clinics, operating room attending/assisting, and basic orthopedics skills as taught during a normal semester.

E-learning program

During the online course, the medical students were graded on the online content using Moodle, an online learning management system created by our institution. Using the program available at the online website, both students and medical instructors evaluated the students' work. Course information, lecture notes, and other teaching materials were included in the teaching materials.

Assessment instrument

There are many common assessment tools available for assessing students grouped by Miller's levels of competence [12]. The assessment of this course comprised formative assessment and summative assessment. For the summative assessment, multiple choice questions (MCQs), key features (KF), modified essay questions (MEQ), oral exams, and objective structured clinical examination (OSCE) were used. For the formative assessment, the medical instructors evaluated professionalism and communication.

The formative assessment was divided into three gradings, good, pass, and fail, based on the interactive activities including case-based discussion, assignment learning, and attending/assisting in OPD, IPD, and OR. The summative assessment in the first part of the course was evaluated via summing MCQs plus KF (55%) and MEQ plus oral exam (45%) and the second section used OSCE (60%), simulated patient charts (30%), and OPD (10%). All students were subjected to the same end of course quizzes with no difference between the groups regarding the kinds of knowledge tested. The results of these quizzes were used to compare the effectiveness of the conventional teaching in 2019 and blended teaching in 2020.

Data collection and analysis

The marks of the quizzes were taken from the secretary's office in the form of an Excel sheet that did not contain any names only the results of the whole class. The data were analyzed using R program Version 3.4.5 (R Foundation for Statistical Computing, Austria). Descriptive statistics are given in percentages and mean ± SD. To evaluate the effectiveness of the two methods the paired-samples t-test was used. A P value of <0.05 was considered significant.

Results

A total of 252 students were enrolled in the Orthopedics online course during 2019–2021, 128 in 2019–2020 and 124 in 2020–2021. The participants' ages ranged between 22 and 31 years, with 81.8% aged 22–23 and a mean age of 23.2 ± 1.1 years. Two-thirds of the students (60.2%) were female in 2019–2020 and there was equal gender distribution in 2020–2021. Overall student baseline demographics were not different between the two groups. The grade point averages of the students were 3.2 and 3.3 in 2019–2020 and 2020–2021, respectively, a non-significant difference (p = 0.06) (Table 1).

Table 1

Comparing baseline characteristics between the video recording orientation and live orientation.

Characteristics	Traditional Teaching Group (n = 128)	Blended Teaching Group (n = 124)	P-value
Age (Year)	23.3 (±1.2)	23.1 (±1.0)	0.27
Sex			0.14
Male	51 (39.8%)	62 (50%)
Female	77 (60.2%)	62 (50%)
GPA	3.2 (±0.4)	3.3 (±0.4)	0.06

The scores of each assessment tool comparing between the blended learning group and the traditional learning group are shown in Fig. 1. The scores in the blended learning group were higher than in the traditional learning group in all assessment tools except the MEQ. The MCQ, KF, Oral, and OSCE scores were 65.7 ± 9.3, 67.7 ± 7.8, 80.1 ± 10.6, and 69.5 ± 6.3, respectively, for the traditional learning group while the same scores were 66.3 ± 8.3, 68.8 ± 8.6, 78.8 ± 10.9, and 70.8 ± 5.3 for the blended group, with all differences being non-significant (p-value = 0.55, 0.31, 0.36, and 0.07, respectively) (Table 2). However, the MEQ score was significantly higher in the blended group than in the traditional learning group (66.8 ± 6.2 versus 65.1 ± 6.3, p-value = 0.40).

Fig. 1

Comparing the mean scores in each assessment tools between the blended learning and traditional learning groups.

Table 2

Comparing effectiveness of purpose in each following topic of orientation between the video recording orientation and live orientation.

Assessment Tools	Traditional Teaching Group (n = 128)	Blended Teaching Group (n = 124)	P-value
Multiple choice score	65.7 (±9.3)	66.3 (±8.3)	0.55
Key feature score	67.7 (±7.8)	68.8 (±8.6)	0.31
Modifed essay question	65.1 (±6.3)	66.8 (±6.2)	0.04
Oral exam	80.1 (±10.6)	78.8 (±10.9)	0.36
Objective structured clinical examination	69.5 (±6.3)	70.8 (±5.3)	0.07

Discussion

This research reports on 5th year medical students' scores comparing the effectiveness of blended learning and traditional learning in an orthopedics course. The study found that the mean score of MEQ and OSCE scores were significantly higher in the blended learning group than in the traditional teaching group, a finding which is consistent with previously studies [[13], [14], [15], [16], [17], [18]], while the other score tools including MCQ and KF in the blended learning group were slightly higher than in the traditional group, however, without significance, except the oral examination test, for which the blended learning group had lower scores than the traditional group, but without significance.

Our department applied a mixed model to schedule the recorded lectures and live activities [19,20], while the rest of the course followed the traditional learning program including ward rounds, practicing in outpatient and inpatient clinics, assisting in the operating room, and also practicing the basic skills in orthopedics because the missing element in the effectiveness of online learning was clinical practice. Therefore, our blended learning was defined as a combination of traditional learning and online learning [7]. Overall, the scores from the students who participated in the blended learning program were higher than in the traditional group. There have been many theories to explain why a blended learning program seems to be more effective than the traditional learning. Our course aimed to utilize modern technology to enhanced educational outcomes. Therefore, the face-to-face lectures in the classroom were shifted to online, and fulfilled the requirements of adult learners were fulfilled by promoting active and student-centered learning [21], with the advantage that students could manage their time in a way that was effective and convenient for them to achieve their goals, while making allowances for married students and to improve their sleep pattern by not having to drive to classes in the morning [22]. Our online learning curriculum was designed based on a framework that focused on effective pedagogical principles and was further supplemented by an understanding of what makes online learning work for students as described in a study by Martin and Bolliger [23]. Using interactive meetings aids in engaging students in an online environment by providing them with intuitive interaction, enabling social learning connections with educators and their peers, and promoting active facilitation [24]. Our final test results indicate that using these strategies can achieve the same effectiveness of learning outcomes as the traditional learning.

The main barrier for our educators in designing this program was the availability of open access resources, which posed a teaching challenge in using these resources while maintaining traditional standards. However, due to the specific concerns around the COVID-19 pandemic, the necessity of these changes became unavoidable and our department was pushed into changing to the blended learning format; a few of our staff were resistant to these changes, but it was a department neccesary, and after using online learning for a while, they agreed it has some advantages [25]. After this experience, our department has decided that after the COVID-19 pandemic resolves, we will continue using the blended teaching method to teach our students as we think this method provides additional benefits that are useful adjuncts to the traditional method for both medical instructors and students.

The strength of this study was that using final examination scores to evaluate the effectiveness between the two methods gave more reliable results than using a survey-based assessment. An evaluation such as this of the effectiveness of blended learning for the health professions is timely and very important the COVID-19 pandemic for both health educators and learners. Another strength of this study was that this investigation was done in medical students in a low-income country, and demonstrated that even here our students could access the online lectures and use the zoom application for meetings, showing the internet is sufficient for online use in our country. Therefore, it can be assumed that the technology for enhancing learning has been growing. However, this study also had several limitations. The effectiveness of blended learning may be dependent on student characteristics, design features, and learning outcomes. As this study was a retrospective study, some factors that could have influenced the final outcomes were not assessed in this study. Therefore, further well-designed randomized controlled trials should be done.

Conclusions

Blended learning was as effective to teach medical students as traditional learning. However, well designed randomized controlled trials are needed to further analyze the educational structure and investigated the factors related to the effectiveness between these interventions.

Ethical approval

The present study was waived by the Prince of Songkla University Institutional Review Board, Faculty of Medicine, Songklanagarind Hospital, Prince of Songkla University (IRB number REC 64-251-11-1).

Sources of funding

No funding was involved regarding this study.

Author contribution

Sitthiphong Suwannaphisit — conceptual idea, data collection, data analysis, writing the paper. Pakjai Tuntarattanapong — data collection, data analysis, writing the paper. Chirathit Anusitviwat — data collection, data analysis, writing the paper. Chaiwat Chuaychoosakoon — conceptual idea, writing the paper.

Consent

The informed consents were not needed in this study.

Registration of research studies

This study is a retrospective study.

Guarantor

Chaiwat Chuaychoosakoon, M.D.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Declaration of competing interest

No conflicts of interest.