Flipped-learning course design and evaluation through student self-assessment in a predental science class.

PURPOSE: This study explores how to design a flipped classroom for a predental science course and evaluate its course through student self-assessment in order to provide practical implications for flipped learning in an undergraduate level.
METHODS: Second- and third-year predental students in the Seoul National University School of Dentistry enrolled in Biodiversity and Global Environment, a 15-week, three-credit course based on a flipped learning model. At the end of the course, the students were asked to rate their self-directed learning, attitude toward social media, discussion skills, learning readiness, and class satisfaction. Out of the 82 predental students, 61 (74.3%) answered the survey. Pearson correlation and multivariate regression analyses were employed to examine the relationship between the self-rated measurements and the performance scores.
RESULTS: The majority of the students felt somewhat more prepared than the medium level before the class (mean score of 3.17 out of 5.00), whereas they expressed relatively low preference concerning social media use and attitude (mean score of 2.49). Thus, it was found that learning readiness was significantly associated with both discussion skills and class satisfaction. In particular, multivariate regression analysis confirmed that learning readiness had a significant influence on learning outcomes.
CONCLUSION: This study offered insights into how to design a flipped learning course in terms of predental students' preference and their learning readiness. Although learning success in a flipped classroom depends on the students' self-perceived level of preparedness, much still remains to be achieved in order to apply social media benefits in a flipped learning context.

Introduction

Higher education institutions have made rigorous efforts to find more effective pedagogical methods that can provide high-quality learning experiences. In order to maximize such learning experiences, most educators seek to personalize instruction for their students, while identifying the needs of individual learners and providing flexibility in scheduling and pacing [1]. Meanwhile, the traditional classroom lecture approach has been reviewed as being less effective for motivating students to obtain the necessary knowledge and skills [2], which has led some educators to recommend incorporating technology to “flip the classroom” [3].

The flipped classroom approach has recently been introduced, but whether it is a more practical learning approach compared with the traditional pedagogy is subject to debate [4]. In the flipped class model, traditional classwork primarily comprising instructor-driven lectures is performed before class, while homework is done during class [5]. Previous research conducted at a medical school supports the notion that the flipped classroom is more effective for achieving learning outcomes than the conventional classroom [6].

In contrast, some studies have indicated that flipped learning may be less desirable than the traditional approach [7]. For students who would be more familiar with traditional lectures, the flipped course design could work negatively in their perception of the learning experience. Since the main issue may be due to flaws in the instructional design, a flipped learning course design should address learner-centered pedagogical needs. In terms of the instructional design, it is important to examine what factors in a flipped classroom would influence the students’ learning experience and ultimately their learning outcomes.

The flipped learning design model for this study employed Eastmond’s learning theory, which focuses on three prominent aspects sequentially involving a student’s learning experience: learning readiness, online features, and learning approaches [8]. The flipped classroom can positively impact self-directed learning readiness, engage more students in the discussion, and ultimately increase their satisfaction in class. In line with Eastmond’s learning theory and the flipped classroom approach, a pilot study was meticulously designed for a science course offered to predental students at the Seoul National University School of Dentistry (SNU SOD). To examine the relationship between the students’ learning processes and their learning outcomes in a flipped classroom, their self-perceived measurements of their learning experiences were collected by administering a survey (based on a 5-point Likert scale). The measurements included: self-directed learning, attitude toward social media, discussion skills, learning readiness, and class satisfaction. Each scale was developed from a literature review and adjusted to the instructional context.

The purpose of this study was to explore how to design a flipped classroom for a predental science course, and evaluate its course through student self-assessment in order to provide practical implications for flipped learning in an undergraduate level. The implication of this study is that teachers can apply this flipped learning design model to influence their students’ engagement and academic achievement in the classroom, while building a broader understanding of its instructional value.

Subjects and methods

In this study, the predental second- and third-year students both participated in the flipped classroom model in which they were requested to view video lectures online before class, after which quizzes and group discussions were held during class. Regarding the course background, Biodiversity and Global Environment is a 15-week, three-credit compulsory course (3-hour in a row each week) for all undergraduate predental students at SNU SOD. The objectives of this course are to acquire knowledge of biodiversity and human life, to improve critical thinking skills, and to change the learners’ attitudes toward the environment. Out of the 82 predental students who signed up for the course, 61 (74.3%) answered the survey at the end of the course. The students consented to the collection of demographic and academic performance data.

This study employed five scales that assessed the students’ self-perceptions of self-directed learning, social media attitude, discussion skills, learning readiness, and class satisfaction. In order to measure the students’ self-directed learning scale, three subdomains were developed, including self-management, desire for learning, and self-control [9]. The social media attitude variable consisted of eight items, including prominent online features such as comfort interacting with online participants, barriers due to technical problems, and barriers due to access [10]. Regarding discussion skills, the features of comfort participating in discussions and actively contributing to making a logical conclusion were included [8]. To determine the predental students’ levels of learning readiness, prior learning experience, motivation, and self-discipline were adapted and included in the survey [11]. Finally, in terms of class satisfaction, the students’ willingness to refer the course to others and the likelihood that they would take another flipped learning course were self-evaluated [12].

Content validity and cross-cultural adaptation were assessed by a team of tutors who examined whether the items represented self-assessment considered essential in our flipped-learning course. As shown in Table 1, the inter-item consistency of each domain in the survey was determined using Cronbach α coefficients (values higher than 0.60). Since the α coefficient value was 0.70, the measures demonstrated acceptable reliability.

Table 1.

Descriptive Statistics for Five Measurements

Measurement	Source	Items	Mean±SD	Min	Max	Cronbach α
Self-directed learning	Fisher et al. (2001) [9]	10	3.38±0.29	2.23	4.04	0.70
Social media attitude	Mostafa (2015) [10]	8	2.49±0.58	1.25	3.62	0.72
Discussion skill	Eastmond (1994) [8]	12	3.23±0.46	2.43	4.28	0.84
Learning readiness	Kirmuzi (2015) [12]	5	3.17±0.61	2.00	4.60	0.71
Class satisfaction	Kirmuzi (2015) [12]	5	3.05±0.73	1.20	5.00	0.78

All of the measurements ranged 1 to 5.

SD: Standard deviation.

As presented in Fig. 1, the course was divided into three parts for each week. The first part was distance learning and self-study, which included watching videos and/or reading course materials. More specifically, the course instructors provided weekly reading materials and 2–4 video clips on YouTube produced by academic resources such as the National Geographic Channel and the Education Broadcasting System (EBS) in Korea. The second part consisted of a 10-minute quiz, followed by a 20-minute mini-lecture. In order to evaluate their understanding of the online contents, each quiz (consisting of ten questions regarding the video clips) served as a diagnosis assessment. In the third and final part, the students were randomly divided into five groups, after which each group held a 90-minute discussion that was evaluated by one of the instructors.

Fig. 1.

Learning Process and Activities

SNS: Social networking service.

At the conclusion of the course, the students were asked to self-assess their level of self-directed learning, attitude toward social media, discussion skills, learning readiness, and class satisfaction. Pearson correlation and multivariate regression analyses were employed to examine the relationship between the students’ self-assessment and their academic performance scores by using STATA 14 statistical software package (Stata Corp., College Station, USA). In all cases of analysis, a p-value of less than or equal to 0.05 was used to determine significance.

Results

The descriptive statistics for the survey are shown in Table 1. The majority of the students felt more self-directed and better prepared before going to class (mean score of 3.38 and 3.17, respectively, out of 5.0), whereas they expressed relatively low preference concerning social media use and attitude (mean score of 2.49). The students also responded that their class satisfaction was generally at the medium level (mean score of 3.05), indicating least satisfactory (n=1, 2%), less satisfactory (n=6, 9%), medium (n=48, 79%), satisfactory (n=4, 7%), and most satisfactory (n=2, 3%).

Overall, it was found that quizzes in a flipped classroom were significantly correlated with learning readiness (r=0.36, p=0.04) and instructor evaluations (r=0.37, p=0.03) (Table 2). In particular, Table 3 shows that the highest correlation occurred between learning readiness and discussion skills (r=0.44, p<0.01), while the second highest correlation occurred between learning readiness and class satisfaction (r=0.43, p=0.02) (Table 2). However, the correlation analysis indicated that there was a negative relationship between attitude toward social media and instructor evaluation.

Table 2.

Correlation Coefficients for Learning Outcome and Self-Perceived Measurements

	Quizzes	Instructor evaluations	Self-directed learning	Social media attitude	Discussion skills	Learning readiness	Class satisfaction
Quizzes	1.00	0.37[a)] (0.03)	0.001 (0.994)	0.00 (0.99)	0.00 (0.99)	0.36[a)] (0.04)	0.09 (0.48)
Instructor evaluations		1.00	0.24 (0.22)	-0.17 (0.38)	0.38[a)] (0.04)	0.28 (0.15)	0.06 (0.77)
Self-directed learning			1.00	0.31 (0.105)	0.36[a)] (0.04)	0.16 (0.43)	0.16 (0.43)
Social media attitude				1.00	0.07 (0.72)	-0.17 (0.39)	0.02 (0.93)
Discussion skills					1.00	0.44[a)] (0.00)	0.19 (0.31)
Learning readiness						1.00	0.43[b)] (0.02)
Class satisfaction							1.00

Indicates that the difference is significantly different from zero at the 95% level, while

indicates the difference at the 99% level.

Table 3.

Average Self-Perceived Measurement Ratings and Performance by Class Year

Measurement domain	Mean±SD		Mean difference (95% CI)
	Second-year (n=28)	Third-year (n=33)
Quizzes	7.69±1.96	8.65±0.89	-0.96[a)] (-1.72 to -0.20)
Instructor evaluations	1.82±0.48	2.20±0.32	-0.38[b)] (-0.58 to -0.17)
Self-directed learning	3.36±0.30	3.40±0.28	-0.04 (-0.19 to 0.11)
Social media attitude	2.53±0.52	2.46±0.64	0.07 (-0.24 to 0.37)
Discussion skills	3.26±0.44	3.19±0.47	0.07 (-0.17 to 0.30)
Learning readiness	3.15±0.65	3.20±0.57	-0.05 (-0.36 to 0.26)
Class satisfaction	2.95±0.71	3.15±0.73	-0.20 (-0.57 to 0.17)

Mann-Whitney U analysis was used. The quizzes score ranged 0 to 10, while the scale of instructor evaluations was from 1 to 3.

SD: Standard deviation, CI: Confidence interval.

Indicates that the difference is significantly different from zero at the 95% level, while

indicates the difference at the 99% level.

Regarding class year, the learning outcomes of the third-year students were significantly higher in terms of quizzes and instructor evaluations than those of the second-year students except for social media attitude and discussion skills (Table 3). In particular, while the third-year students perceived themselves as better prepared for learning, the second-year students considered themselves more positive in their attitude toward social media.

Finally, Table 4 presents the results of the multivariate regression analysis. The students’ self-rated scores for learning readiness and discussion skills were significant predictors of the learning outcomes in the flipped learning classroom. Contrary to what was expected, the impact of the students’ attitude toward social media on learning outcomes was both weak and negative.

Table 4.

Regression Coefficients for Multivariate Regression Analysis of Learning Outcomes on the Self-Perceived Measurements and Control Variables

	Multivariate regression model
	Quizzes			Instructor evaluations
	β	SE	p-value	β	SE	p-value
Self-directed learning	0.287	0.625	0.561	0.201	0.175	0.203
Social media attitude	0.269	0.321	0.389	-0.009	0.079	0.881
Discussion skills	0.151	0.315	0.651	0.181[a)]	0.085	0.021
Learning readiness	0.877[b)]	0.228	0.000	0.491[b)]	0.098	0.000
Class satisfaction	0.115	0.270	0.668	0.031	0.071	0.613
Prior semesters’ GPA	0.352[a)]	0.151	0.019	0.414[a)]	0.181	0.032

Multivariate regression includes gender, class year, and measurements.

SE: Standard error, GPA: Grade point average.

Indicates that the difference is significantly different from zero at the 95% level, while

indicates the difference at the 99% level.

Discussion

Most Korean undergraduate predental students are accustomed to learning in traditional lecture-based classrooms. However, the present study implemented new learning strategies for engaging students in a flipped classroom in order to maximize the effectiveness of their learning processes. The results highlighted the interconnections between the four factors in the model, thus confirming a positive relationship among self-directed learning, learning readiness, discussion skills, and class satisfaction.

The results of Pearson correlation and multivariate regression analyses showed that the factors concerning self-directed learning, discussion skills, and learning readiness were all significantly related with the students’ academic performance and class satisfaction in a flipped classroom. Such results were in line with the general pattern of noncognitive factors as a predictor of academic performance in higher education [13]. It is also critical to denote that individual students who prefer self-directed learning may be more motivated to study topics that are closely relevant for face-to-face group discussions [13].

Learning readiness deserves more attention in predicting academic performance in a flipped learning classroom. In this study, it was found that learning readiness was significantly associated with both discussion skills and class satisfaction. In other words, the more prepared the students were for their learning, the more actively they participated in the discussions, and the more likely they were to be satisfied with the class. In particular, multivariate regression analysis made it possible to confirm that learning readiness had a significant influence on learning outcomes.

Regarding the use of social media in a flipped classroom, it is important to recognize how unfamiliar students may be in using social networking services (SNSs) as a learning and teaching environment [14,15]. Hence, the educational value of social media, such as Facebook, may be under question [10]. In the present study, since some of the students expressed that they felt uncomfortable with software programs, slow Internet connections, and the lack of technological expertise, they naturally felt that the features of Facebook were less satisfactory for academic purposes. In this regard, Hew indicated that there is almost little educational value and that its application is restricted to being connected with each other [16].

In contrast, the majority of the students who benefited from the Facebook group found that their interactions primarily focused on online resources, which strengthened their motivation for learning. More specifically, the students felt that online resources helped increase their motivation for learning and gain a better understanding, since the internet-based materials enabled them to interact with their peers and receive immediate notification of updates. Based on the findings, it is possible that such use of social media could lead to the formation of a learning community in which knowledge is conveniently and easily accessed [17]. At first class, it would be beneficial to introduce how to effectively apply social media in a flipped-learning classroom in order to foster positive awareness of social media use.

Overall, this study employed a theory-based, flipped learning approach in a predental undergraduate course. The implications of the findings are that faculty members can design or adapt a flipped learning design model in order to influence their students’ engagement and academic achievement. As learning readiness interrelates with discussion skill and class satisfaction, the focus should be on motivating students to study before class as well as making a diagnosis assessment. More specifically, various learning activities, including learning resources, mini-lectures, and debates/discussion can be more tightly integrated into the core of a course design, which, in turn, can enhance the students’ learning experience. This approach can also serve as a benchmark for designing future flipped classroom models.

Although this study supports the efficacy of a flipped learning approach, its findings are not completely free from limitations. For example, the class used for this study was a relatively short (15-week), well-structured undergraduate science course for predental students using SNS-based contents. This study could not isolate the confounding factors that would influence predental students’ performance, and both pre- and post-test should have been conducted to measure pure effects of a flipped-classroom. In addition, high-achieving medical and dental students tend to be less influenced by instructional methods that future research on the efficacy of the flipped classroom should focus on a longer period of time before making any generalizations. In addition, another key component in determining a successful flipped classroom is technology that allows lectures to be pre-recorded and made available to students.

In conclusion, this study offered insights into designing a flipped learning course in terms of predental students’ learning readiness and their attitude toward social media. Well-structured learning process both at home and in class needs to be monitored to increase students’ self-directed learning at a flipped course. Above all, since success of flipped learning depends upon the students’ self-perceived level of preparedness, much still remains to be achieved in order to apply social media benefits in a flipped learning context.