Students' perceptions and satisfaction level of hybrid problem-based learning for 16 years in Kyungpook National University School of Medicine, Korea.

PURPOSE: Kyungpook National University School of Medicine has been implementing hybrid problem-based learning (PBL) since 1999. The aim of this study was to investigate the changes in the students' perceptions and satisfaction levels of hybrid PBL.
METHODS: The target period of our study was from 1999 to 2014, and target subjects were second-year medical students in Kyungpook National University School of Medicine. The survey was conducted at the end of semester. We had a focused interview with group leaders and some volunteer students.
RESULTS: As for the scores regarding students' overall satisfaction with PBL, there was significant improvement in 2005 compared to 2002, but the scores decreased and no differences between the survey years noted after 2005. The students' preference ratio for the once a week PBL sessions, tutor presence, synchronization of contents, and arrangement of PBL sessions and related lectures was 60%-80%, 50%-90%, 52%-96%, and 78%-93%, respectively.
CONCLUSION: In order to increase students' satisfaction with hybrid PBL and to improve the perception of it, firstly, it is necessary to arrange the date and the time of PBL sessions so that students can concentrate on PBL. Secondly, PBL cases should be selected and arranged to be well synchronized with the ongoing lectures. Finally, it is important to create a safe atmosphere so that students can engage actively in PBL sessions.

Introduction

Problem-based Learning (PBL) began to be implemented at some medical schools with innovative curriculum in Asia during the 1970s after Howard Barrows pioneered PBL program at the McMaster University Medical School in the late 1960s [1]. Twenty-three medical schools in Korea have adopted PBL as an experimental or a formal course since the 1990s [2]. The hybrid PBL curriculum of Kyungpook National University School of Medicine (KNUSOM) has been designed to run concurrently with a conventional curriculum over the past decade since it was adopted as a two credits pass/fail course in 1999 [3].

The available reports have involved the principle and practice of PBL [4], development and the application of PBL [5], case reports of implementation or experience of PBL [3,6,7,8,9,10,11,12], its evaluation [3,7,8,9,13], and self-directed learning readiness [14,15]. Through these studies a better understanding and sharing of their experiences of PBL could be promoted. It is still a challenge to the adoption and implementation of PBL with some variations to suit each medical school situation and needs.

This article reports our 16-year experiences of hybrid PBL regarding the level of the students’ satisfaction with PBL, the students’ preference for the number of PBL sessions per week, the arrangement of PBL sessions and lectures, and the preference for the tutor presence at PBL session. We believe that this study could help medical schools adopt and develop PBL.

Subjects and methods

1. Subjects

Our survey subjects were second year medical students who participated in the PBL curriculum of KNUSOM. The questionnaire was completed by 685 students out of 716 PBL participants (Table 1).

Table 1.

Survey Participant

	2002	2005	2009	2010	2012	2014	Total
No. of Participants	134	139	110	113	110	110	716
No. of Respondents	127	122	109	113	106	108	684
Responding rate (%)	94.8	87.8	99.0	100	96.3	98.1	95.7

Seven survey topics we categorized were students’ overall satisfaction, longitudinal or block style of PBL operations, frequency of PBL sessions, arrangement of PBL case, synchronization of contents, tutor presence, and grading method (Table 2).

Table 2.

Topic and Target Year of Survey

Survey topic	2002	2005	2009	2010	2012	2014
Student’s overall satisfaction level	○	○	○	○	○	○
Students' preference between longitudinal style vs. block style	○	○
Students' preference for frequency of PBL sessions per week	○		○	○	○	○
Arrangement of PBL session and related lecture			○	○	○
Synchronization of contents of PBL case and ongoing lecture	○		○	○
Students' preference for tutor attendance at PBL session			○	○	○	○
Students' preference for the grading system	○		○	○	○	○

PBL: Problem-based learning.

2. Methods

The questionnaire survey and focused interviews were conducted at the end of the semester. Group leaders and some volunteer students were interviewed using open-ended questions. The interview was recorded and analyzed with the approval.

The questionnaire was developed using closed-ended items modified from surveys which had been reported in previous studies [3,4,8,12]. The internal reliability of the questionnaire items was verified by Cronbach α coefficient. The items were answered using a 5-point Likert scale (1=strongly disagree to 5=strongly agree) in 2002 and 2005. Since 2009, a 7-point scale (1=strongly disagree to 7=strongly agree) was used. Different scales used in different years were synchronized into a 5-point scale by a statistics program, so as to be analyzed for inter-year variability.

The differences in students’ overall satisfaction levels between years were assessed by one-way analysis of variance after extracting the same five items from the annual PBL questionnaire (Table 3). The internal reliability of five items by Cronbach α was 0.631 (in 2002), 0.956 (in 2005), 0.763 (in 2009), 0.627 (in 2010), 0.714 (in 2012), and 0.721 (in 2014). Another frequency and descriptive analysis was also conducted. All data were analyzed with SPSS version 22.0 for Window (SPSS Inc., Chicago, USA) and p<0.05 was considered significant.

Table 3.

Student’s Overall Satisfaction Level of Problem-Based Learning

Survey item	Year	No.	Mean score	SD	SD	F	p	Scheffe
I am satisfied with the PBL	2002 (a)	127	3.68	0.916	0.081	22.506[*]	0.000	a<b, d
	2005 (b)	122	4.26	0.557	0.050			b>c, e
	2010 (c)	113	3.88	0.530	0.050			c>e
	2012 (d)	106	4.05	0.653	0.063			d>e
	2014 (e)	108	3.44	0.835	0.080
I am satisfied with my tutor	2002 (a)	127	3.31	0.870	0.077	28.305[*]	0.000	a<b, d
	2005 (b)	122	4.25	0.621	0.056			b>c, d, e
	2010 (c)	113	3.31	0.791	0.074			c<d
	2012 (d)	106	3.65	0.829	0.080
	2014 (e)	108	3.62	0.817	0.079
I am very active in PBL sessions	2002 (a)	127	3.81	0.710	0.063	9.485[*]	0.000	a<b, e
	2005 (b)	122	4.30	0.585	0.053
	2010 (c)	113	4.04	0.557	0.052
	2012 (d)	106	4.06	0.659	0.064
	2014 (e)	108	4.14	0.648	0.062
I do self-directed learning for PBL	2002 (a)	127	3.33	0.713	0.063	28.786[*]	0.000	a<b, c, d, e
	2005 (b)	122	4.23	0.614	0.056			b>c, d, e
	2010 (c)	113	3.75	0.675	0.064
	2012 (d)	106	3.90	0.661	0.064
	2014 (e)	108	3.92	0.725	0.070
Team members are cooperative and helpful for each other	2002 (a)	127	3.42	0.771	0.068	38.837[*]	0.000	a<b, c, d, e
	2005 (b)	122	4.27	0.499	0.045
	2010 (c)	113	4.31	0.568	0.053
	2012 (d)	106	4.08	0.643	0.062
	2014 (e)	108	4.06	0.681	0.066
Total of above five items	2002 (a)	127	3.51	0.509	0.045	42.452[*]	0.000	a<b, c, d, e
	2005 (b)	122	4.26	0.405	0.037			b>c, d, e
	2010 (c)	113	3.86	0.393	0.037
	2012 (d)	106	3.95	0.462	0.045
	2014 (e)	108	3.84	0.512	0.049

SD: Standard deviation, PBL: Problem-based learning.

p<0.05.

Results

1. Students’ overall satisfaction levels of PBL

The mean scores of students’ overall satisfaction levels based on a 5-point scale were 3.51 points (in 2002), 4.26 points (in 2005), 3.86 points (in 2010), 3.95 points (in 2012), and 3.84 points (in 2014). A significant improvement was noted in 2005 compared to 2002 but the score decreased significantly after 2005 (Table 3, Fig. 1).

Fig. 1.

Student’s Overall Satisfaction Level of Problem-Based Learning

PBL: Problem-based learning.

The mean scores of the first item (“I am satisfied with PBL”) in 2002, in 2005, in 2010, in 2012, in 2014 were 3.68 points, 4.26 points, 3.88 points, 4.05 points, and 3.44 points, respectively. The score of 2005 was significantly higher than that of 2002 but the score decreased significantly in 2010 and in 2014 (Table 3, Fig. 1).

The mean scores of the second item (“I am satisfied with my tutor”) in 2002, in 2005, in 2010, in 2012, in 2014 were 3.31 points, 4.25 points, 3.31 points, 3.65 points, and 3.62 points, respectively. The score of 2005 increased significantly than that of 2002, but after 2005, the score decreased significantly (Table 3, Fig. 1).

The mean scores of the third item (“I am very active in PBL sessions”) in 2002, in 2005, in 2010, in 2012, in 2014 were 3.81 points, 4.30 points, 4.04 points, 4.06 points, and 4.14 points, respectively. The score of 2005 was significantly higher than that of 2002. There were no significant differences after 2005 (Table 3, Fig. 1).

The mean scores of the fourth item (“I do self-directed learning for PBL”) in 2002, in 2005, in 2010, in 2012, in 2014 were 3.33 points, 4.23 points, 3.75 points, 3.90 points, and 3.92 points, respectively. The score of 2005 increased significantly than that of 2002, but after 2005, the score became lower significantly (Table 3, Fig. 1).

The mean scores of the fifth item (“Team members are cooperative and helpful for each other”) in 2002, in 2005, in 2010, in 2012, in 2014 were 3.42 points, 4.27 points, 4.31 points, 4.08 points, and 4.06 points, respectively. A significant improvement was noted after 2002. There was no significant difference after that (Table 3, Fig. 1).

2. Students’ preference for the style of PBL operations: longitudinal versus block

In the 2002 and 2005 survey, approximately 70% of students preferred “the longitudinal” to “the block.” Many students were prone to spent time in reviewing lecture materials and preparing for their exams during the block PBL period. Therefore the block style was replaced by the longitudinal PBL.

3. Students’ preference for the frequency of PBL sessions per week

Approximately 60% to 80 % of students preferred to meet once a week: 63.4% in 2009, 79.1% in 2010, 65.1% in 2012, and 57.0% in 2014. Only 2% to 10% students preferred to meet twice week.

4. Arrangement of PBL session and related lecture: before or after lectures

Approximately 78% to 93% of students preferred to have PBL sessions after related lectures: 78.0% in 2009, 88.7% in 2010, and 92.5% in 2012.

5. Synchronization of contents of PBL case and ongoing lecture: dependent versus independent

Approximately 52% to 96% of students showed a preference for using the PBL case well correlated with the contents of concurrent lectures: 95.9% in 2002, 52.3% in 2009, and 64.3% in 2010.

6. Students’ preference for having a tutor at PBL session: with tutor versus without tutor

Approximately 50% to 91% of students preferred the presence of a tutor in the group: 54.1% in 2009, 49.6% in 2010, 90.6% in 2012, and 80.4% in 2014. Students who preferred to have a tutor asserted that the presence of a tutor itself could create a better learning environment. On the other hand, many students claimed that they had experienced unsafe atmosphere, difficulty in adjusting to different PBL facilitation skills of tutors due to the insufficient standardization of tutor (Table 4).

Table 4.

Student’s Perception toward Tutor Attendance at Problem-Based Learning Session

Positive		Negative
•	Tutors guide discussion and prevent from going to the wrong direction.	•	Discussion is often interrupted by improper tutor intervention.
•	Tutors help students understand key points with proper advice or information.	•	Conflicts over the PBL process often caused by the frequent changes and different facilitation skills of tutors.
•	Tutors facilitate keeping track of the PBL process by proper intervention.	•	Tutors sometimes fail to guide the right direction and session becomes distracted.
•	Tutors assist students to approach the issues schematically.	•	Students often hesitate to talk in front of tutors or are intimidated by tutors.
•	Tutors help students have organized wrap-up of discussion.	•	Tutors are sometimes too aggressive in questioning and make students embarrassed and helpless.

PBL: Problem-based learning.

7. Students’ preference for the grading system: pass/fail versus grading

The tools for student assessment consisted of tutor assessment, learning report, modified essay questions (MEQs), peer assessment, responding questionnaire, drawing case mapping and schema, and attendance (Table 5). Approximately 49% to 94% of students were in favor of pass/fail system over a grading system: 48.5% in 2002, 93.6% in 2009, 93.6% in 2010, 93.6% in 2012, and 89.7% in 2014. Regarding peer assessment, students disclosed that they gave the same score to each other in most cases. Therefore, peer assessment proved not to serve as a reliable assessment tool.

Table 5.

Tools for Student Assessment in Problem-Based Learning

Year	Tutor assessment	Learning report	MEQ	Peer assessment	Responding questionnaire	Case mapping	Attendance	Others
2002	50	20	20	-	10	-	-	-
2003	-	30	50	-	10	-	-	10
2004	50	30	-	-	10	-	-	10
2005	50	40	-	-	10	-	-	-
2006	50	-	10	10	-	10	10	10
2007	50	30	-	-	10	10	-	-
2008	50	30	-	-	10	10	-	-
2009	50	-	10	10	10	10	10	-
2010	50	-	10	-	10	10	20	-
2011	40	-	10	-	10	20	20	-
2012	40	-	10	-	10	20	20	-
2013	40	-	10	-	10	20	20	-
2014	40	-	10	-	10	20	20	-

Data are presented as percentage.

MEQ: Modified essay question.

Discussion

We define the PBL run concurrently with a lecture-based traditional curriculum as hybrid PBL in this article. The analysis of the changes in the students’ perceptions and satisfaction levels of 16-year hybrid PBL showed that students preferred once-a-week PBL to twice or more PBL per week throughout the semester. This can be interpreted as a way of means for students to choose to lessen the burdens on hybrid PBL at a medical school. Kim et al. [5] also mentioned that when a medical school introduces PBL in a traditional curriculum, it is desirable to impose the least amount of burden on students.

Our results showed that most students preferred the contents of the PBL case to be synchronized with ongoing lectures, and the case to be studied after the related lectures. The possible explanation of preference is that students want to reduce the time and efforts required to study the PBL case and want to be activated the discussion during the PBL session using the knowledge they have learned in previous lectures.

In our study, the students’ opinions on the tutor presence were widely distributed (approximately 50%–96%). Students appeared to welcome a tutor who could understand the philosophy and process of the PBL. Park et al. [11] reported that students evaluated the role of tutor as somewhat negative and good facilitation skills of discussion and process of the PBL is strongly required. Hur & Kim [16] reported that the discussion was often interrupted by improper tutor intervention and the important role of tutors was to facilitate open discussion and to create a safe environment. According to Nanda & Manjunatha [17], tutors who are too strict could generate stress and pressure among students and an uncomfortable atmosphere can disrupt students’ open-minded thought. There is no need for tutor in a PBL session if the tutor is too strict. For this reason, investigators claim that tutors need more training to conduct the PBL. These studies support our results of the students’ preference for having a tutor and students’ perceptions toward tutor.

In our study, the tools for assessment included tutor and peer assessment, learning report, MEQs, responding questionnaire, drawing case mapping and schema, and attendance. Our results revealed that peer assessment proved not to serve as a reliable assessment tool. According to Kim et al. [18], peer assessment could be used as an effective assessment tool for students’ performance in PBL when peer evaluation score is not included in the grades. User-friendly peer evaluation can be used to screen for maladjusted students when students understand the purpose and feedback method of peer assessment well and the questionnaire of peer review is well constituted. Most of the students in our school preferred the pass/fail assessment to grading after 2002. The result regarding preference for grading is consistent with that of Chung et al. [12].

In summary, our results indicate that in order to increase students’ satisfaction with hybrid PBL and to improve the perception of it, firstly, it appears to be necessary to arrange the date and the time of PBL sessions so that students can concentrate on PBL. Secondly, PBL cases should be selected and arranged to be well synchronized with the concurrent lectures. Finally, it is important to create a safe atmosphere so that students can engage actively in PBL sessions. It is strongly recommended that the tutor should play a role to facilitate the group’s thinking and discussion and to guide the PBL process.