Analysis of satisfaction and academic achievement of medical students in a flipped class.

PURPOSE: The purpose of this study was to examine the satisfaction of medical students in flipped learning and analyzed academic achievement in comparison with lecture class.
METHODS: The subjects were 40 students who participated in flipped learning during neurology course in the second year. After performing flipped learning, formative assessment was conducted and the degree of satisfaction was examined. Questionnaires of satisfaction were developed to identify the perceptions of students on flipped learning. To compare the academic achievement of students, formative assessments were conducted at the end of the flipped learning class and the lecture class. The data was analyzed by frequency and paired t-test method.
RESULTS: The students showed a high level of frequency in using lecture notes (80.6%) and lecture slides (74.2%) among the pre-class learning resources. The average score (3.89) was higher for the factor of interaction and collaboration in the classroom than for the factor of improving learning (3.62). The average score of the students in the formative assessment was 4.28 points (out of 10 points) in the lecture class, while it was 5.56 points (out of 10 points) in the flipped learning class thus showing a statistically significant difference (t=-4.203, p<0.001).
CONCLUSION: It was observed from the responses of the students that flipped learning is helpful for the interaction and collaboration in the classroom. It is expect that this result will be useful as basic data for medical school to try flipped learning in the future.

Introduction

Recently, flipped learning is gaining attention as one of the various alternatives in teaching-learning methods that depart from the traditional lecture based teaching [1,2]. More and more attempts are also being made at health professional schools, including those of medicine, pharmacy, and nursing, to introduce and apply flipped learning [3-8]. In flipped learning or flipped classes, the lecture proceeds in a “flipped” manner from the traditional class where the lecturer gives a lecture to the students and then the students review and go over the lessons, understanding and internalizing what they learned in the class [9]. In flipped learning, students voluntarily and actively study the lessons pre-class, and then resolve the given problem in the course of discussion and cooperation in the class.

The latest trend in medical education is pursuing the format of the lecture hall without lectures [10], it has been moving in this direction to emphasize student-led activities and their participation in the lecture hall rather than lectures that are unilaterally given by the lecturer. Such changes are causing a swift introduction of flipped learning into medical education. Flipped learning is suitable for medical education because students can be more focused and participate more actively in classes with pre-class learning where they have to study a large volume of knowledge in a limited amount of time, and they can also secure the time for in-depth learning [11]. Moreover, many classes are offered in the problembased learning or team-based learning method at medical school, and if flipped learning is introduced, classes can be operated in a more efficient manner [11].

In medical education, flipped learning is regarded as an alternative enabling student-led classes with active participation [12], and also regarded as an efficient way to enhance student’s satisfaction level with classes and their academic achievement [13]. Many studies have reported that flipped learning increases active participation and medical students’ motivation for learning [14], and it is effective for inducing in-depth learning [15]. In Korea, however, cases where flipped learning is introduced in health profession schools are mainly offered in the field of nursing education [3-5], and have not been sufficiently studied in the field of medical education. Therefore, this study was to examine the satisfaction of medical students in flipped learning and to compare their academic achievement between the lecture class and the flipped learning class. This will provide the basic data for colleges of medicine to introduce flipped learning in the future.

Methods

1. Sample

This study was conducted with 40 medical students who took the neurology course in the second semester of the second year. The distribution of the students consisted of 24 male students (60%) and 16 female students (40%).

2. Procedure

The procedure of flipped learning is as follows. First, for pre-class activity, the lecturer provided lecture notes, lecture slides, textbooks, and videos one week ahead of the class for pre-class learning. The students were grouped into five groups, each consisting of eight students. Each group undertook pre-class learning, and then was directed to create three test items to check the details of pre-class learning; they were required to submit those test items to the professor 2 days before the class. The professor reviewed the items submitted by each group and verified their validity.

The in-class activity involved the students solving the items given by each group. Test items were provided by one group and students in other groups solved and explained the items. They cross-checked whether the answers were correct between groups, and they were directed to explain the reasons for their answers. Throughout this process, students could have in-group or inter-group discussions. The main role of the professor was to facilitate students’ full discussion on the issue and to provide additional explanations on what was not explained by the students. A formative assessment was performed at the end of the class, and a survey on satisfaction levels was conducted to elicit feedback from students on the contents of the class.

3. Instrumentation

1) Survey on perceptions of the effect of flipped learning

In this study, questionnaires were independently developed to identify the perceptions of students on the educational effects of flipped learning. The measurement tool consisted of a total of 11 questions consisting of two factors according to the results of the factor analysis that was conducted for validity verification. Two factors on improving learning (factor I, six items), and interaction and collaboration in the classroom (factor II, five items) explained 84.86% of the total variation. The reliability value (Cronbach’s α coefficient) of each factor was 0.954 for the factor of improving learning and 0.951 for the factor of interaction and collaboration in the classroom. The response style to each question used a 5-point-scale ranging from 1 (absolutely incorrect) to 5 (absolutely correct). Two further questions were added where the respondents could freely describe what they liked about flipped learning and what they thought should be improved or corrected (Table 1).

Table 1.

Factor Loadings for Flipped Learning Perception Items

Item	Factor I[a)]	Factor II[b)]	Reliability (α)
7	0.862	0.349	0.954
11	0.854	0.331
8	0.830	0.186
10	0.829	0.454
9	0.731	0.563
6	0.728	0.575
1	0.193	0.952	0.951
4	0.336	0.864
2	0.444	0.811
5	0.445	0.783
3	0.576	0.708
Percent of variance accounted for	74.594	10.266

Improving learning.

Interaction and collaboration in the classroom.

2) Student academic achievement

To compare the academic achievement of students, formative assessments were conducted at the end of the flipped learning class and the lecture class, during the neurology course. The lecture class was performed as usual, and the formative assessment was conducted at the end of the class. The contents for the two groups were different, but the contents level and the difficulty level of the test items were adjusted to a similar level for formative assessments.

4. Ethical considerations

In the survey of the satisfaction level on the effect of flipped learning, only the data of the students who were fully informed of the purpose of the survey and who provided consent in writing were used for analysis. The formative evaluation results used unique IDs with to distinguish each individual student from others so that personal information could be protected.

5. Data analysis

This study used the following method of analysis. First, the mean and standard deviation were calculated to understand the perceptions on the effect of flipped learning, and second, a paired t-test was conducted to examine the difference of student achievement between the lecture class and the flipped learning class.

Results

1. Use of learning resources in flipped learning pre-classes

The students showed a high level of frequency in using lecture notes (80.6%) and lecture slides (74.2%) among the pre-class learning resources, but the frequency of using textbooks (10.3%) and videos (16.7%) was low. They responded that lecture slides (3.84 on average) and lecture notes (3.61 on average) were most helpful for learning. The average learning time for pre-class learning was 34.3 minutes (Table 2).

Table 2.

Use of Learning Resource in Pre-class for Flipped Learning

Learning resource	Use for learning		Average learning time for pre-class learning (min)	Degree of help for learning
	Yes	No
Lecture note	25 (80.6)	6 (19.4)	35	3.61±0.88
Text book	3 (10.3)	26 (89.7)	26	3.00±0.96
Lecture slides (ppt)	23 (74.2)	8 (25.8)	40	3.84±0.93
Videos[a)]	5 (16.7)	25 (83.3)	36	3.08±1.04

Data are presented as number (%) or mean±standard deviation.

By the Korean Consortium for e-Learning in Medical Education.

2. Analysis of levels of perception on the effects of flipped learning

After undergoing flipped learning, a survey on levels of perception was conducted (Table 3). According to the results for each item, the factor of interaction and collaboration in the classroom recorded 3.89 points on average, while the factor of improving learning showed an average of 3.62 points. Overall, the average score was higher for the factor of interaction and collaboration in the classroom than for the factor of improving learning. In detail, the interaction between the lecturer and the students showed the highest score of 4.13 points on average. In addition, the respondents said that they were satisfied with the flipped learning class overall (3.74 points on average), and that flipped learning was helpful for self-directed learning (3.71 points on average) and motivation for learning (3.66 points on average).

Table 3.

Students’ Perception of the Effectiveness of Flipped Learning

Variable	Mean±standard deviation
Factor: interaction and collaboration in the classroom
1. Collaboration among group members was done properly in the solution process of the task.	4.03±0.97
2. It was helpful to collaborate with members in order to solve group tasks.	3.82±0.90
3. The process of group discussion was helpful.	3.74±0.95
4. Interaction and collaboration among members were appropriate.	3.84±1.03
5. The interaction between the students and the lecturers were appropriate.	4.13±0.84
Total	3.89±0.75
Factor: improving learning
6. Flipping learning was helpful in classroom attention	3.57±1.07
7. Flipping learning was helpful for self-directed learning.	3.71±1.02
8. Flipping learning was helpful in motivation and interest.	3.66±0.97
9. Flipping learning was helpful in academic achievement	3.49±1.09
10. I am overall satisfied with the flipping learning class.	3.74±0.95
11. Flipping learning should be applied to other classes.	3.29±1.25
Total	3.62±0.95

3. Opinions on flipped learning

The students’ freely described remarks on flipped learning class were as follows. Regarding the benefits of the flipped learning class, individuals commented, “I could be more focused on and better understand the class because I participated in the class after studying the contents in advance,” “I did not feel bored or sleepy in the class,” and “The class was interesting and funny, and it was good because I could learn by myself.” On the other hand, they pointed out what should be improved: “We could not discuss (topics) sufficiently as the class time was short,” “It was difficult to create the test items, which was the pre-class assignment,” and “It would be better to provide more diverse pre-class learning resources.”

4. Comparison of the formative assessment results of the lecture class and the flipped learning class

The formative assessment results were compared after the lecture class and the flipped learning class (Table 4). The average score of the students in the formative assessment was 4.28 points (out of 10 points) in the lecture class, while it was 5.56 points (out of 10 points) in the flipped learning class thus showing a statistically significant difference (t=-4.203, p<0.001).

Table 4.

Comparison of Formative Assessment Scores between Lecture and Flip Learning Class

Teaching method	Mean±standard deviation	t-value	p-value
Lecture	4.28±1.169	-4.203	0.000
Flipped learning	5.56±2.100

Discussion

In this study, we tried flipped learning in the neurology course in the second semester in the second year at a medical school and analyzed the results. The key results are summarized and discussed below.

The students used lecture notes or lecture slides more than videos and textbooks among the pre-class learning resources. The videos in this study were not produced for the flipped learning class but provided by the Korean Consortium for e-Learning in Medical Education. For this reason, it is estimated that the students undertook pre-class learning mainly with lecture notes or lecture slides, using videos only for reference. The resources for class are important in flipped learning because students have to complete the pre-class learning activity before participating in the class. Even if videos are not included, the educational effect can be enhanced with resources such as lecture slides, lecture notes, etc. [16]. However, as the millennium-generation students of today are familiar with digital information technology, the general trend is that videos are used in learning [17]. Moreover, the advantage of videos is that students can learn individually and repeatedly. Therefore, making videos and reflecting on the goals of the class set by the professor can be effective.

Students evaluated interaction and collaboration in the classroom as more helpful in flipped learning than improving learning. In flipped learning, students undertake pre-class learning and the class itself proceeds based on group activities rather than unilateral lecturing thus the interaction between the professor and the students is dynamic in the flipped learning class. Such a result is consistent with the results of many preceding studies conducted with the students in health professions’ schools [18]. Meanwhile, in the opinions on flipped learning presented in the survey, it was mentioned that the positive aspects of flipped learning were that students could focus well during the class, could easily and swiftly understand the classroom lesson, and found the lesson contents interesting. This made up for the weaknesses of the unilateral lecture classes and studies on nursing students showed the same results [3,5]. However, it was pointed out that the weakness of the flipped learning class was that students felt they did not have a sufficient amount of time for discussion because the class was too short. Therefore, assigning a longer time for the flipped learning class should be considered.

The formative assessment score was significantly higher after the flipped learning class than that after the lecture class. This supports the results of a preceding study that the method of flipped learning is more effective in enhancing the level of understanding of the lessons and academic achievement of students than the traditional method of lecturing [13]. Such a result was also put forward in clerkship. In the psychiatric nursing practicum course, the experimental group in which flipped learning was applied showed a higher level of academic achievement than the control group [4]. Therefore, it seems necessary to conduct research on the application of flipped learning in the practicum course for medical students in the future. Meanwhile, the test scores of the group to which flipped learning was applied were high, but the group showed the lowest satisfaction level with the class [19,20]. This study only compared the results of the formative evaluation, but a comparative study on satisfaction levels with the class should be added.

In conclusion, it was observed from the responses of the students that flipped learning is helpful for the interaction and collaboration in the classroom: it helps students to be focused in class and enables their understanding of it, it helps students to feel interested in class, and it is effective in enhancing academic achievement. However, this study was conducted only once in only one course at a medical school, so it is hard to generalize the results. It is necessary to have students experience flipped learning several times in various subjects and then study the results. As it has been reported that flipped learning positively affects affective characteristics such as motivation for learning and learning attitude [21], core competencies of university students [4], and approach to learning [15], follow-up research on medical students is required in the future. As this study compared the academic achievement of the lecture class and that of the flipped learning class within the same group, further studies could usefully distinguish and compare an experimental group with a control group.