Assessing Female Pharmacy Students' Satisfaction with Active Learning Techniques at King Saud University.

BACKGROUND: Studies conducted in Saudi Arabia assess student's perception towards using the flipped classroom or audience response system (ARS) separately, none integrated the two strategies. This study aims to evaluate pharmacy students at King Saud University - college of pharmacy's perceptions of student engagement and satisfaction with audience response system (ARS), and the flipped classroom techniques.
METHODS: Two lecture topics were flipped and delivered to female students enrolled in the pharmacy practice lab course at the university. An audience response system was integrated into the flipped classroom. The satisfaction of the students was measured using an online questionnaire. They then rated their satisfaction on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree).
RESULTS: Overall, 82.1% of the students stated that the ARS made the class better; 71.64% agreed that taking a course in a flipped classroom was a useful learning experience. Online questionnaire revealed that the ARS fostered students' thinking skills and peers' interaction. Many students report that the flipped classroom made them evaluate their learning processes and pace themselves according to their speed, time, and level of interest. Students, however, stated that technical issues and time-consuming tasks were a downside to these teaching strategies.
CONCLUSION: Although the students' overall response to the use of ARS and flipped classroom techniques was positive; limited class time hindered students from achieving the full benefits reaped from these strategies. Therefore, as per our study's results, modifying the duration of the class to ensure it is not too short to be overwhelming can help instructors utilize these startegies in the best possible manner. Alternatively, a thorough and pre-emptive selection of familiar discussion topics ahead of time would also help in this regard.

Introduction

In today’s pharmacy curricula, engaging students in active learning has emerged as an essential means of equipping graduates and undergraduate students with the practical knowledge and skills required to address and resolve emerging challenges and become a competent member of a healthcare team.1 Active learning is a strategy in which instructors use various tools and techniques to help shift lectures from input-based pedagogy, in which students are simply listening to instructors, to learner-centered pedagogy, which emphasizes student participation.1,2 Among these learning strategies are the audience response system (ARS) and flipped classrooms.

In the flipped classroom, students come to the class with a pre-knowledge of the topic of discussion, through reading materials at their own pace that were uploaded by their instructor before the class.3,4 This will free up time for in-class activities and peer interaction that activates higher-order thinking and create a self-learner student. This is facilitated with the help of the instructor.3,5,6 Most studies have found that students respond positively to flipped classrooms, yet many reports insist that this model should be only implemented when students are working on basic concepts with which they are at least partially familiar.3,4 The way audience response systems work is by allowing the instructor to pose multiple-choice questions created using PowerPoint to students, students then choose the answer they think is true using an audience response keypad.8,9 After a specific time has elapsed for each question, an aggregated percentage of answers will display on the PowerPoint slide and the instructor uses these answers for further clarification and discussion.8 ARSs have been found to increase student participation significantly, as well as, motivation, and attention, especially during ungraded activities.8,10,11

In Saudi Arabia, some studies were conducted to assess various methods of active learning in medical education. Ismaile and Alhosban found that the immediate feedback and anonymity available via ARS technology-enhanced nursing students’ peer discussions at Princess Nourah bint Abdulrahman University (PNU) and improved their understanding of a lecture’s main concepts.12 A study by Almanasef et al at King Khalid University found that using a flipped classroom technique for pharmacy students increases their satisfaction and engagement with their peers and with the instructor more than a traditional class.3 Studies in Saudi Arabia have only assessed the impact of either flipped classroom techniques or ARSs separately and never integrated the two. The integration of flipped classroom techniques and ARSs have been investigated in the context of Malaysian pharmacy students enrolled in a complementary medicine course; this study found that combining these strategies provided students with a new experience which was more interesting, engaging, and valuable in terms of students’ preparation for their final exam.13 The reported impact of these techniques and the paucity of quantitative evidence regarding the overall satisfaction with these techniques among pharmacy students justifies this study.

Aim

This study aims to assess the impact of flipped classroom techniques and an ARS on pharmacy students’ perceptions of student engagement and satisfaction.

Materials and Methods

Study Design and Sample

This was a cross-sectional study conducted at King Saud University, college of pharmacy (female’s section). The study sampled only fourth-year female students enrolled in the 3-credit hour pharmacy practice lab course between March and April 2019.  The study was reviewed and approved by the institutional review board at King Saud Medical City (Project No. E-19-4022). In the course, a total of 72 female students were enrolled, with 70 female students being able to attend one or both flipped classroom sessions and the integrated ARS sessions. All-female students who did not attend the flipped classroom and ARS activities were excluded from the study.

Design

Before the Lecture

The course instructor selected two lecture topics “drugs during pregnancy and lactation” and “food-drug interaction”—to be taught using a flipped classroom technique. The reason for selecting those topics is the familiarity of the content as these topics were partially addressed in previous courses. Lecture objectives, work plans, and reading materials were uploaded to the learning management system (LMS) one week ahead.

Automated Response System (ARS)

PowerPoint slides addressing the lectures objectives with embedded interactive multiple choice questions were prepared using “iClicker classic” system (Macmillan Learning, New York, NY, USA). The system allows for LMS integration where each student can be assigned to a specific remote using the university ID. Then, the instructor can check the student’s achievement after each class by exporting an Excel sheet from LMS. The use of the ARS was integrated into the flipped classroom. During class, the instructor posed multiple choice questions both before grouping students and at the end of each lecture, and the answers were polled and displayed simultaneously. The students used the handheld remote to choose the correct answer. The system also allows for setting timer to each question, count down timer for 45 seconds was visible on each slide.

During the Lecture

Based on the lecture goals, each topic was divided into sections. Each section represents specific objective. The instructor used ARS to perform a non-graded test created with PowerPoint to evaluate the depth of students’ understanding (see Figure 1 for the flipped classroom format). Most students were expected to be unfamiliar with study material due to exams. As a result, the instructor divided students into three groups: those who are 75% familiar with studying materials as leaders; partially familiar students were assigned as leaders assistants: and those who were unfamiliar comprised the rest of the group. New groups were formed consisting of leaders, leaders' assistants, and students who did not cover the topic. This was done to maintain balance among groups.

Figure 1

The flipped classroom format. The flipped classroom begins one week before the class by supplementing students with reading materials to the LMS. Students were grouped according to their level of knowledge acquired from reading the uploaded studying materials, and the topic was divided into sections for each group to read and explain later. They were then given 25 minutes of in-class reading to allow all students to read and engage with group members, followed by 3 minutes of presenting the assigned section to the rest of the class. In the class, the instructor evaluated the students’ understanding by asking MCQs to be answered with ARSs.

All groups were given 25 minutes to read and note down the main points of their assigned section of the lecture. Then, the groups were given three minutes to explain their part to their peers, who, in turn, asked questions. The students were given the liberty to choose their method of delivery ie, PowerPoint slides or using white board and pens.

After the students had summarized all sections of the lecture, the instructor outlined the main points from the lecture that needed further attention and answered general questions by the students.

After the Lecture

At the end of each lecture, remote-based ARSs were used to evaluate students’ overall knowledge through multiple-choice questions created with PowerPoint. There were 15–20 multiple choice questions per topic.

Evaluation and Assessment

Students’ satisfaction with ARS and flipped classroom models was measured using an Online questionnaire with three sections. Section I collected demographic data (course code: PPL4/PPL5/PPL6, year: 3rd year/4th year), the number of ARS sessions attended by survey respondents (between zero and five), and the number of flipped classrooms attended by survey respondents (zero to two). Sections II and III involved a 26-item survey which was adopted from Clauson et al8 and Alamri et al.5 This survey assessed the effects of ARS and flipped classroom techniques on students’ perceptions of satisfaction and engagement and were created using Google Forms (see and for the questionnaires). Respondents were also provided with six optional, open-ended questions which asked them to gauge each active learning technique’s most and least useful features and whether they recommend continued use of these techniques for future courses. Students rated their satisfaction on a 5-point Likert scale, with responses ranging from 1 (strongly disagree) to 5 (strongly agree). Informed consent was obtained through an online form from all participating students.

Statistical Analysis

We employed descriptive statistics with mean and standard deviations to measure students’ satisfaction and engagement regarding the aforementioned active learning techniques. An independent sample t-test was used to compare the satisfaction and engagement of students who attended more than three ARS sessions to those who attended less than three sessions, and students who attended only one flipped classroom to students who participated at both flipped classrooms. All statistical analysis was done using IBM SPSS version 20.0 for Windows. Thematic coding was used to evaluate the responses obtained from students who answered open-ended questions in the Online questionnaire.

Results

Students’ Demographic Data

Out of 72 students, 67 completed the survey – a response rate of 93%. All respondents were fourth-year students taking Pharmacy Practice Lab 5 (PPL5). Fifty students (75%) attended 4 or 5 ARS sessions and 17 (25.4%) attended 3 ARS sessions or less, 53 (79.1%) attended both flipped classrooms, 12 (18%) attended only 1, and 2 students (3%) were unable to attend either class.

Students’ Satisfaction with ARS

As shown in Table 1, students reported that they felt more encouraged to participate when using the ARS than they normally did in class (76% (Mean 3.9). Students also reported that the ARS made the class better and that they considered their attendance worthwhile (82.1% (Mean 3.99), and 75% (Mean 3.91)).

Table 1

Students’ Responses Regarding Their Satisfaction with the ARS After Completing a Pharmacy Practice Lab Course (N = 67)

Survey Itema	Strongly DisagreeNo. (%)	DisagreeNo. (%)	NeutralNo. (%)	AgreeNo. (%)	Strongly AgreeNo. (%)	Mean (SD)
“Use of the ARS encouraged me to participate in class.”	1 (1.49)	1 (1.49)	8 (11.94)	31 (46.27)	26 (38.81)	4.19 (0.821)
“Use of the ARS encouraged others to participate in class.”	0	1 (1.49)	12 (17.91)	32 (47.76)	22 (32.84)	4.12 (0.749)
“Use of the ARS was distracting.”	7 (10.45)	32 (47.76)	18 (26.87)	8 (11.94)	2 (2.99)	2.49 (0.943)
“Because of the ARS, I participated more in this class than I normally do.”	0	8 (11.94)	8 (11.94)	34 (50.75)	17 (25.37)	3.90 (0.923)
“I valued that the ARS allowed people to respond anonymously.”	0	5 (7.46)	22 (32.84)	32 (47.76)	8 (11.94)	3.64 (0.792)
“Use of the ARS slowed down the class too much.”	5 (7.46)	28 (41.79)	22 (32.84)	9 (13.43)	3 (4.48)	2.66 (0.962)
“The professor depended too much on the ARS.”	2 (2.99)	22 (32.84)	29 (43.28)	11 (16.42)	3 (4.48)	2.87 (0.886)
“Use of the ARS helped make some topics clearer.”	2 (2.99)	5 (7.46)	15 (22.39)	33 (49.25)	12 (17.91)	3.72 (0.950)
“I would rather use the ARS to participate in class discussions than use the microphone.”	2 (2.99)	11 (16.42)	15 (22.39)	22 (32.84)	17 (25.37)	3.61 (1.128)
“Overall, I think the ARS made the class better.”	2 (2.99)	3 (4.48)	7 (10.45)	37 (55.22)	18 (26.87)	3.99 (0.913)
“Attending class was worthwhile.”	0	3 (4.48)	14 (20.90)	36 (53.73)	14 (20.90)	3.91 (0.773)

Note: a5-point Likert scale (1= strongly disagree, 2= disagree 3= neutral, 4= agree, 5= strongly agree).

Students’ Satisfaction with the Flipped Classroom

In total, 72% of the students agreed that taking a course in a flipped classroom was a useful learning experience (Mean 3.79). Higher mean scores were reported for the survey items, “the instructor for this course met my learning expectations,” and “discussions in the flipped classroom assisted me in understanding other points of view.” Lower mean scores were reported for the survey items “I invested a great deal of effort researching on the Internet to increase my participation” and “I was stimulated to complete additional readings or research on topics discussed in class.” (Table 2).

Table 2

Students’ Responses Regarding Their Satisfaction with the Flipped Classroom Technique After Completing a Pharmacy Practice Lab Course (N = 67)

Survey Itema	Strongly DisagreeNo. (%)	DisagreeNo. (%)	NeutralNo. (%)	AgreeNo. (%)	Strongly AgreeNo. (%)	Mean (SD)
“I was able to learn fromthe course materials inthe flipped classroom.”	2 (2.99)	4 (5.97)	22 (32.84)	33 (49.25)	6 (8.96)	3.55(0.858)
“I was stimulated to complete additional readings or research on topics discussed in class.”	2 (2.99)	9 (13.43)	23 (34.33)	28 (41.79)	5 (7.46)	3.37(0.918)
“Discussions in the flipped classroom assisted me in understanding other points of view.”	1 (1.49)	5 (7.46)	18 (26.87)	36 (53.73)	7 (10.45)	3.64(0.829)
“As a result of my experience with this course, I would like to take another flipped classroom course in the future.”	1 (1.49)	5 (7.46)	21 (31.34)	31 (46.27)	9 (13.43)	3.6(0.868)
“Taking a course in a flipped classroom was a useful learning experience.”	2 (2.99)	3 (4.48)	14 (20.90)	36 (53.73)	12 (17.91)	3.79(0.897)
“The diversity of topics and resources in this course prompted me to participate effectively in discussions.”	2 (2.99)	6 (8.96)	22 (32.84)	25 (37.31)	12 (17.91)	3.58(0.987)
“I invested a great deal of effort researching on the Internet to increase my participation.”	2 (2.99)	10 (14.93)	28 (41.79)	19 (28.36)	8 (11.94)	3.31(0.972)
“My level of learning that occurred in this course was of the highest quality.”	1 (1.49)	8 (11.94)	31 (46.27)	21 (31.34)	6 (8.96)	3.34(0.863)
“The learning activities and assignments of this course met my learning expectations.”	1 (1.49)	4 (5.97)	27 (40.30)	30 (44.78)	5 (7.46)	3.51(0.786)
“The instructor for this course met my learning expectations.”	0	1 (1.49)	20 (29.85)	33 (49.25)	13 (19.40)	3.87(0.736)
“This flipped classroom course met my learning expectations.”	1 (1.49)	5 (7.46)	21 (31.34)	35 (52.24)	5 (7.46)	3.57(0.802)
“The flipped classroom helped me learn more than I would have if we had completed traditional lessons.”	4 (5.97)	8 (11.94)	17 (25.37)	30 (44.78)	8 (11.94)	3.45(1.049)
“I found it easy to pace myself successfully through this course.”	3 (4.48)	3 (4.48)	18 (26.87)	36 (53.73)	7 (10.45)	3.61(0.904)
“The flipped classroom has improved my learning outcomes.”	2 (2.99)	4 (5.97)	24 (35.82)	32 (47.76)	5 (7.46)	3.51(0.842)

Note: a5-point Likert scale (1= strongly disagree, 2= disagree 3= neutral, 4= agree, 5= strongly agree).

Frequency of Attendance and Student Satisfaction

To test the hypothesis that the perceptions of students who attended more than three ARS sessions or both of the flipped classroom courses would differ from the perceptions of students who attended three or fewer ARS sessions or only one flipped classroom in a statistically significant way, an independent sample t-test was performed. There were no statistically significant differences between students who attended more ARS or flipped classroom sessions and those who attended less for all survey items, assuming equal variances. Increased attendance did not affect students’ satisfaction with the techniques and they could conceptualize the added benefits of integrating these active learning strategies into class.

Students’ Overall Feedback

Students were asked open-ended questions at the end of the online questionnaire to elicit their overall impressions of their experience with the ARS and flipped classroom techniques. Overall, students reported that they had positive impressions and experiences. The major themes related to the flipped classes were; familiarity, timing and group formation. While the major themes for the ARS were; speed, technical issues and positive learning environment. Students indicated that the ARS allowed all students to participate in a fun and competitive environment and that the ARS helped with communication apprehension, as students’ answers were anonymous and displayed as percentages. One student stated that the ARS, “makes me examine the knowledge that I have in a competitive way.” Another commented that via the ARS, “everyone was able to participate in class, and everyone was able to choose the answer they wanted without judging them if they [chose a wrong answer].” Students indicated their appreciation of the flipped classroom technique by saying that this technique enhanced peer teaching, made students evaluate their own learning process, and allowed them to pace themselves according to their own speed, time, and level of interest. Regarding the flipped classroom technique, one student said “students can choose what they want from the materials and what the new information will be to make the lecture more informative.” Another reported that the flipped classroom technique “improves [students’] ability to read and summarize [material] in a very efficient way,” and that through this technique, “everyone can get the main idea” of a lecture. Some students described the challenges that they faced. For instance, many students reported that they faced technical issues when using ARS. One student stated their apprehension that, if they forgot their ARS remote, “then my attendance will be useless.” Another commented that the ARS remotes “sometimes become hanged. Moreover, students reported that when someone answered a wrong answer, but majority selected the right one, discussion often neglected leaving those who answered wrongly feeling left out. As one student said “Learning outcomes were measured by taking the score. It was not helpful to know the reason when we got a wrong answer.” For the flipped classroom, distraction was a major drawback for many students. One said that “there was Too many students talking at the same time, and I couldn’t focus”. Another said, “I didn’t like the way that we discussed and shared information during the 25 minutes of assigned reading of the flipped classroom”. Suggesting that Forming groups ahead and ensuring most of them read most of the Topic content might improve teamwork, learning from peers, and reduce distraction.

When the ARS was used in combination with a flipped classroom, some students who did not read the pre-class materials found it difficult to take full advantage of the ARS’s capabilities. Regarding this problem, one student said,

[t]he topics covered needed to be read, we couldn’t have the full benefit or understand the whole topic by ARS only, but at least we had a few points summarizing the topic. It would be great if there was more time to discuss more about it after the session.

Another common refrain from students was that they felt pressured for time; they felt that they were given insufficient time both during the class (eg, while reading and summarizing topics) and before the class (given the demands of the pre-class reading materials, other courses, and midterms) to make the most of these techniques. As one student put it:

I did not learn properly about other groups’ topics because I was busy searching for my topic – and then the groups represented their topic by reading it, which was not a helpful [way] of understanding [their topic]. I’d prefer [a] regular class with presentations and visual aids.

Another commented that “the use of ARS has taken a lot of time, and it may not be possible to cover all aspects of the topic in the same way as a traditional class” .Therefore, we suggest that when these two techniques are used together; instructors should choose topics that are familiar to students, devote more time to discussing wrong answers that were received from students using ARSs, as well as, not to overwhelm students by uploading a fair amount of study materials.

Discussion

The results of this study suggest that the ARS and class-flipping techniques increase students’ feelings of engagement and satisfaction during class. Students’ responses indicated that they favored the ARS and flipped classroom techniques for enhancing peer discussions, interactions, instant instructor feedback, peer comparison of knowledge acquired by most students, and helped them to engage others’ thinking paths in a fun and competitive way. The anonymity of the ARS enabled students to engage without the instructor addressing a single student, which in turn helped students who feel anxious when pressured to communicate with their peers and the instructor to engage in the lecture.

This finding broadly aligned with Hussain et al systematic review that assess the impact of ARS on pharmacy student’s perception. In this systematic review, 11 studies were included, results demonstrate that using ARS increased classroom participation, peer discussion, and overall learning experiences.14

Our findings are consistent with other national studies that were conducted among dentistry and nursing students.12,15 Both studies found the ARS system to be an effective tool to enhance student attention, engagement, and overall attitude. Kaleem et al who did not find any change in students’ academic achievements but stated their belief that the fun environment fostered by the ARS could eventually result in higher student achievement.15 The current study found that the flipped classroom enhanced peer teaching and students’ satisfaction more than a traditional class. These findings are consistent with the Almanasef et al study that found an improvement in satisfaction of pharmacy students, enhanced learning due to in-class activities, and better interaction among peers and instructor during a pharmacoepidemiology course delivered at King Khalid University using the flipped classroom.3 In our study, some students highlighted that topics selection is an important factor for successful implementation of a flipped classroom. Few studies highlighted that topics and courses involving a flipped classroom approach should be chosen carefully and selectively in order for students to reap the full rewards of the techniques.6,16 McCabe et al found a significant reduction in the knowledge and confidence of second-year students who were enrolled directly into experiential self-care course and had to learn the didactic content online, via a flipped classroom technique, compared to students who took a traditional lecture followed by an experiential course.16 Khanova et al also report that some pharmacotherapy concepts need to be fully explained prior to flipped class.6 These findings indicate that the flipped class technique has its limits and may not be effective for all topics in pharmacy education.

Although students in our study reported positive attitudes towards these strategies, the strategies themselves are poorly implemented in healthcare colleges. Alruthia et al highlighted challenges which obstruct instructors’ use of active learning strategies, including lack of technical support for instructors, time constraints, a lack of administrative support, a lack of appreciation, and interest by instructors in utilizing these and other active learning strategies.17 This indicates that implementing such teaching techniques requires sufficient collaboration between the instructors and the educational institutions by providing training courses and a reliable technical support service.

This feasibility study is important to identify barriers of implementing new teaching technologies before embarking on larger studies. Several issues related to flipped class technique including timing within the semester, topics, method of grouping were uncovered during this study. Other issues related to ARS including the practical set up and ARS remote functionalities were also revealed.

There are several limitations to this study. For instance, we had a small sample size, therefore, results from t-test need to be interpreted with caution. In addition, we could not assess male students’ perceptions of these strategies due to practical issues associated with campuses distance. This research also only assessed the satisfaction of students with these techniques between March and April 2019 and not the whole semester, different results could have been obtained if this study was conducted for a longer period. Finally, there was a lack of a control group and, therefore, a lack of an effective comparison.

Future research should assess the impact of the techniques mentioned above on students’ academic performance. The impact of flipped classes on student learning has been previously studied in medical education.18 This meta-analysis included 28 studies, found that flipping classes were more effective compared with traditional teaching in terms of academic performance. The analysis also revealed general preference to flipped classes over traditional teaching. Similar results were also reported for the impact of ARS on academic performance.19

Given the practicality issues with the classic ARS, newer alternatives such as “Poll everywhere®” have been successfully used in medical education.13,20,21 Poll everywhere ® and similar polling software’s are affordable, more convenient in terms of set up, students can use their own mobiles instead of remotes. Other advantages poll everywhere® is flexibility in terms of location, instructors can integrate active learning remotely using such technology. However, one of the drawbacks of poll everywhere® is distraction, students might browse non educational websites or social media. Instructors can limit distractions by setting a timer for each question.

We recommend that instructors either adjust class time to suit these techniques to optimize the benefits of integrating ARSs into the flipped classroom, or carefully choose relatively familiar topics for discussion whenever they plan to use these two techniques together.

Conclusion

Students responded positively to the implementation of the ARS and flipped classroom techniques and indicated their willingness to enroll in courses which utilize these strategies in the future. This study also provided us with an insight into these strategies’ potential for enhancing students’ learning experiences with emphasis that when these techniques are to be combined, students are at least partially familiar with the topic selected.