Enhancing students' active learning and self-efficacy using mobile technology in medical English classes.

PURPOSE: This study investigated student experiences of mobile technology in a medical English course using a free mobile classroom application, Socrative and its impact on student learning to help them achieve the competency expected in the course.
METHODS: Questionnaires were administered of year 2 medical students enrolled in a medical English course between 2016 and 2017 to investigate their experience in using Socrative. Learning outcomes were assessed by comparing students' academic performance with that of the 2015 cohort, who had taken the same course without using mobile technology, and also by measuring changes in their self-efficacy in English over time during the course.
RESULTS: This study found students' positive perceptions of and overall satisfaction with their experience in using mobile technology in the medical English course. There were no differences in the academic performance between the study cohort and the comparison group. The ratio of students who were in the remediation program after the mid-term examination was 11%, whereas the remediation ratio of the 2015 cohort was 21%. Students' self-efficacy in English improved significantly in the writing and reading domains (p<0.05).
CONCLUSION: This study found several benefits of using Socrative in the medical English course. Socrative fostered the classroom interaction by encouraging input from every student and sharing it instantly with the whole class. There was also learning improvement with the use of Socrative. Furthermore, it has the potential to enhance assessment by offering more various forms of assessment than that of conventional audience response systems.

Introduction

Acquiring competencies in medical English is fundamental for tomorrow’s doctors in the increasingly globalized healthcare environment. As the aim of the medical English course is to have students achieve competencies in medical English, the course needs to be designed with the competency-based education framework. Competency-based education puts an emphasis on student-centered learning that promotes active learning to help students achieve the learning outcomes [1]. This paper describes how mobile technology is integrated into a medical English course to make it more interactive and engaging. Furthermore, this study investigated student experiences of using mobile technology and its impact on student learning by evaluating the learning outcomes. It is important to make the medical English course more interactive and engaging for several reasons. First, one of the core principles of competency-based education is to move toward student-centered learning [1]. To that end, understanding who the students are and what motivates them to learn is germane to effective course design [2], and in doing so, setting objectives and providing feedback helps to create an environment for learning [3]. This warrants analysis of students’ needs to identify appropriate learning goals by actively interacting with them. Second, there is an increasing emphasis on student-centered learning in foreign language teaching and moving away from didactic lectures in higher education settings and also in teaching medical English [4]. Therefore, an instructional strategy was warranted to enhance classroom interactions in teaching and learning medical English. Third, active learning needs to be promoted to help students achieve the learning outcomes. This was particular an issue as there were challenges in promoting active participation of students in the classroom discussion in the Asian context due in large part to their cultural backgrounds. It has been known there are distinctive learning styles across cultures [5] and research suggests the classroom climate in Asian cultures tend to be teacher-centered [6], which pertains to the traditions that emphasize hierarchy and a tendency to avoid uncertainty [7-10].

Studies have shown interactive technologies, such as audience responses systems, when used effectively, are effective for promoting classroom interactions and student engagement [11-15]. With the advent of mobile technologies, there are increasing opportunities for students to engage in learning using interactive technologies. Many students and teachers are using mobile technologies within medical education, and research shows medical students have positive attitudes towards using such technologies for learning [16].

Research suggests the use of mobile technology has the potential to enhance medical education [17]. In particular, mobile audience response systems have benefits of utilizing new question types not available in conventional ones in addition to the convenient access and have the potential to improve learner interaction [18]. Still, there has been little evidence on how medical schools are using such technologies across the curriculum [19], nor there is solid evidence base to support their effective use [20]. Furthermore, research is scant on the use of such technologies for the teaching and learning of medical English.

This study explored the impact of using mobile technology on teaching and learning in a medical English course. To that end, this study assessed students’ perceptions of using the mobile classroom application in the medical English course. Moreover, student learning outcomes was assessed by comparing their academic performance with that of those who had taken the same course when mobile technology was not used, and also by measuring changes in the students’ self-efficacy in English over time during the course.

Methods

1. Participants and setting

The medical English course under investigation was offered for year 2 students in the 3-year accelerated degree program for graduate entry to medicine at Dongguk University School of Medicine. This was a two-credit course and was offered over a semester for the duration of 16 weeks. This course was intended for students to achieve competencies in reading English in topics related to healthcare, medical science, and healthcare professions and in writing in English. To achieve these goals, students learned medical English by reading news reports and magazine articles in English on relevant topics, which were selected by the instructor. Those resources were targeted for the general audience, not for medical experts, which was suited for the students enrolled in this course considering their level of knowledge in medical English.

Students were assessed by written tests through the mid-term and final examinations together with some assignments that they had to write essays in English. Students were given formative assessments frequently throughout the semester, and those who performed poorly in the mid-term assessment were put into the remediation program to ensure that every student achieve the competency expected in this course. Those in the remediation program took a retest after a period of self-study.

The course was redesigned in 2016 by integrating three components to enhance student engagement in learning as an effort to move toward competency-based education. First, the students were introduced to the concept of competency-based education and discussed it on the first day of class to help them understand it and motivate them to achieve the learning goals. Second, the instructor received feedback from the students regarding the course objectives to reflect their varying needs and expectations in the course as their English language proficiency prior to taking this course varied due to their diverse backgrounds and learning experiences. Third, technology needs to support formative assessment to enhance student learning, which is a fundamental component of competency-based education.

To achieve those goals, mobile technology was integrated into the teaching and learning in this medical English course. In particular, the integration of mobile technology was intended to promote active learning and classroom interactions and to enhance learning. Among several mobile classroom applications currently available, Socrative (http://www.socrative.com) was chosen for this course. Socrative is a free, cloud-based system and it is one of the mostly widely adopted in higher education settings [21]. Previous studies have shown that using Socrative promotes active learning in college students and medical students by supporting activities that facilitate instant feedback and promote collaborative learning and critical thinking [21-23]. Moreover, El Shaban [24] argues Socrative is an effective tool for the instructors to encourage classroom interactions for the students in a class setting such as English courses with the opportunity to gather students’ feedback anonymously.

On the first day of the class, students were introduced to Socrative and they downloaded and installed it in their mobile phones. Afterwards, instant polling was used in this application to survey the students’ experiences in learning English prior to this course in multiple choice questions and their expectations on this course in open-ended questions for free text responses. After the students answered all the questions, the instructor shared the results with the class anonymously by showing the report generated by the mobile classroom application.

Moreover, quizzes were provided weekly during the class using the quizzing tool in Socrative throughout the semester as formative assessment to reinforce students’ understanding of course materials. The quizzes were available in multiple-choice, true or false, and short answer formats and the instructor had control over how they were delivered. In this study, quizzes were offered weekly for formative assessment to reinforce student acquisition of vocabulary and grammar skills covered in the course materials. Vocabulary skills were assessed using short answer questions, and multiple-choice questions were used to assess students’ grammar skills. The instructor could choose to show the test items one by one or at once. The instructor also could turn on or off immediate feedback on student answers depending on whether to allow students to check the correct answer immediately when they submit the answer for each question or to have them receive the feedback after they finish answering all questions. In this course, students were given test items one by one so that the instructor could give feedback for each item. The instructor was able to view student progress and scores during an assessment in teacher mode and shared it with the class by showing it up on the screen, where student scores were displayed anonymously.

2. Instruments and procedures

A questionnaire was developed and implemented of the students enrolled in the medical English course during the years 2016 and 2017. This 28-items questionnaire consisted of three sections: (1) respondent demographics, (2) student experiences with mobile technology, (3) and their perceptions of self-efficacy in English. This questionnaire also included one openended question on their expectations of this course.

The section on student perceptions of mobile technology included six items and were adapted from the questionnaire for evaluating student perceptions of the mobile audience response system developed by Wash [22]. Students responded to the items using a 5-point Likert scale, where 1=‘strongly disagree’ and 5=‘strongly agree.’ The English in the original instrument was translated by the author. To evaluate the clarity of the items, those items were pilot tested with year 3 students in 2016, who had taken the ‘Medical English’ course prior to this study cohort in 2015. Three students volunteered to participate in the pilot test. They were asked to think aloud while they completed the questionnaire and to make comments on any words or phrases not clear or unfamiliar to them. The students commented there were no items unclear for them. These items showed reliable internal consistency with the Cronbach’s α coefficient being 0.89.

This questionnaire also included 20 items on self-efficacy in English. Self-efficacy in English is a person’s belief that he or she can listen, speak, read, and write in English successfully in a context where English is used as a medium for communication [25]. It is known that students’ self-efficacy beliefs influence their motivation and academic performance [26,27], and self-efficacy in English is a key indicator of successful acquisition of the proficiency in English as a foreign language [25]. Therefore, this study assessed changes in student perceptions of self-efficacy in English as a measure of their acquisition of English proficiency.

The items on self-efficacy in English in this study were adapted from the self-efficacy in English questionnaire developed by Cubillos and Ilvento [28]. This instrument was tested with US college students and was found valid and reliable [28]. The Korean version of this instrument was used for this study, which was translated and validated by Moon [29]. The items on self-efficacy in English consisted of four domains of the English language proficiency—i.e., listening (five items), speaking (five items), reading (five items), and writing (five items). Students responded to the items using a 5-point Likert scale, where 1=‘strongly disagree’ and 5=‘strongly agree.’ A medical faculty who had taught medical English for medical students reviewed the questionnaire to check for its content validity for this study. The Cronbach’s α coefficients for the four domains of self-efficacy in English in this study ranged from 0.87 to 0.93, which demonstrated reliable internal consistency.

Students’ self-efficacy in English was assessed in a pre- and post-test format to measure changes in their self-efficacy perceptions during the course. The pre-test on students’ self-efficacy in English was implemented in the first week of the semester and the post-test was conducted in the final week of the semester together with the items on their perceptions of using mobile technology. The questionnaires were delivered online using Socrative.

Student learning outcomes were assessed to investigate the impact of using mobile technology on student learning. Students’ total assessment scores that contributed to their course grades was compared with that of those in the year 2015 cohort, who had taken the same course prior to this study cohort and mobile technology was not used. Furthermore, the ratio of students who were in the remediation program after the mid-term examination was examined to evaluate their academic performance.

An ethical review was conducted and informed consent was exempted by the institutional review board of Dongguk University, Gyeongju, Korea (IRB approval no., DGU IRB 2016004-01). Students were informed that participation was voluntary and their responses would remain anonymous. Consent was implied with completion of the survey. To protect student anonymity and ensure voluntary participation, each student completed the questionnaire privately and anonymously.

3. Data analysis

The data was saved at the mobile classroom application’s website and the results of descriptive analysis of the data were also obtained from the website. An independent t-test was conducted of the student perceptions of self-efficacy in English between the students who had taken the medical English course in years 2016 and 2017 for baseline comparison. Changes in students’ self-efficacy in English were analyzed using a paired t-test. A Kruskal-Wallis test was conducted of students’ test scores across three years to compare their academic performance between when mobile technology was used and when it was not used. IBM SPSS ver. 23.0 for Windows (IBM Corp., Armonk, USA) was used for the inferential statistics and the significance level was 0.05 for the statistical analysis.

Results

1. Student demographics

A total of 46 students returned the questionnaires, yielding a 100% response rate; 21 of them took the medical English course in the year 2016, and 25 took it in 2017. Twelve (26.1%) of the students were female; 34 (73.9%) were male. Their ages ranged between 19 and 22 years (mean age, 20.45±0.91 years). The students’ self-efficacy in English in the first week of the course was compared between those who had taken in this course in the year 2016 and in the year 2017. There were no significant differences in the students’ self-efficacy in English between the two groups (t=0.19, p=0.85).

Student performance data also came from the students who had taken this course in the year 2015 (n=14); five of them (35.7%) were female and nine (64.3%) were male, and they were between 19 and 21 years in age (mean age, 19.9±0.59 years).

2. Student perceptions of mobile technology use

Table 1 shows the results of the student survey. A majority of students found that the instant polling using the mobile classroom application helped them understand the course objectives and communicate their expectations on this course effectively; they also agreed with the statement that formative assessment using this application enhanced their learning. Moreover, a majority of students felt such learning activities helped them get more motivated and engage in learning. In addition, a vast majority of the students were generally satisfied with the use of mobile classroom application in this course.

Table 1.

Results of Student Survey on Their Experience with Mobile Technology (n=46)

Items	Responses[a)]
1. Instant polling using Socrative helped me understand the course objectives.	4.15±0.76
2. Instant polling using Socrative helped me communicate myexpectations on this course with the class.	4.23±0.72
3. The use of Socrative motivated me to learn in this course.	3.88±0.91
4. The quizzes using Socrative highlighted gaps in my knowledge.	4.12±0.81
5. The use of Socrative in this course helped me engage in learning in this course.	3.96±0.83
6. I was generally satisfied with the use of Socrative in this course.	4.17±0.58

Data are presented as mean±standard deviation.

Indicates that 1 is ‘strongly disagree’ and 5 is ‘strongly agree.’

The results of the class poll on student expectations of the course showed they varied in their needs for developing their English proficiency. Approximately one-third of the students (n=15) disagreed with the statement that it was important for them to develop competencies in the English language, while a half of them (n=22) agreed with that statement. Still, there were several students who stated in free-text responses to the class poll that they intended to take an English language proficiency test in the near future and hoped the medical English course helped them better prepare for it.

3. Student learning improvement with mobile technology

Table 2 illustrates student performance in this course over the three consecutive years. There were no significant differences in students’ course grades among the three cohorts of students (H (2)=0.28, p=0.87). Moreover, students’ remediation rates were compared to evaluate student performance. The ratio of students who were in the remediation program after the mid-term examination (i.e., taking a retest after a period of self-study) has decreased to 10.9% (5/46) from 21.4% (3/14) of the comparison group who had taken the course when mobile technology was not used. All of the students in remediation performed successfully in the retest.

Table 2.

Student Performance for the Consecutive Years (n=60)[a)]

	Control group	Experimental group (those who used mobile technology)
	2015 Cohort (n=14)	2016 Cohort (n=21)	2017 Cohort (n=25)
Total pointsa)	84.4±11.0	82.7±11.0	84.4±9.91
Remediation rates	21.4 (3/14)	9.5 (2/21)	12.0 (3/25)

Data are presented as mean±standard deviation or % (number of remediation/total number).

Out of possible 100 points.

Table 3 shows changes in students’ self-efficacy in English over time during the course. Students’ self-efficacy in reading and writing in English increased significantly (p<0.05), whereas their self-efficacy in listening and speaking in English did not change.

Table 3.

Changes in Student’s Self-Efficacy in Medical English (n=46)[a)]

Domain	Pre-test	Post-test	t-test	p-value
Listening	2.97±0.82	3.02±1.07	0.69	0.49
Speaking	2.57±0.70	2.95±0.82	0.48	0.72
Reading	3.16±0.54	3.44±0.61	1.13	<0.05
Writing	3.32±0.68	3.56±0.46	1.25	<0.05

Data are presented as mean±standard deviation.

Indicates that 1 is ‘strongly disagree’ and 5 is ‘strongly agree.’

Discussion

This study found students’ positive perceptions of and overall satisfaction with their experience in using mobile technology in the medical English course. This finding is consistent with that of conventional audience response systems [11,30] and also with previous studies of the use of the mobile classroom application to promote active learning in higher education settings [21,23].

Furthermore, this study found several benefits of using mobile classroom application by utilizing features not available in conventional audience response systems. In particular, free text responses, which were available in the instant polling tool in the mobile classroom application, gave richer information about students’ motivation and expectations on their learning. In previous years, the course orientation session tended to be didactic with little classroom interaction. This made it difficult for the instructor to communicate with the students on their expectations and needs on this course, which was necessary for identifying appropriate learning goals. The use of mobile classroom application fostered the classroom interaction by encouraging input from every student in the class and sharing it instantly with the whole class.

In addition to instant polling, formative assessment using the quizzing tool in the mobile classroom application was helpful for the instructor to gauge students’ level of understanding of the course material and also for giving students formative feedback. Moreover, although it was not tried in this course, multimedia—i.e., image files—can be incorporated in the assessment items when creating quizzes in the mobile classroom application. Research indicates use of multimedia in test items would help make the assessment more authentic than conventional paper-based tests [31,32]. Using mobile classroom applications has the potential to enhance assessment by utilizing more various formats than those of conventional audience response systems.

This study also shows positive learning outcomes in terms of the improvement in students’ self-efficacy in English. Still, students’ self-efficacy in English increased in the two domains of learning a language (i.e., reading and writing) and did not show significant improvement in the listening and speaking domains. It can be speculated this was due, in part, to the course objectives that did not focus on listening and speaking skills. Moreover, student learning improved compared to that of the previous year when mobile technology was not used in terms of the ratio of students who did not perform successfully in the mid-term assessment and therefore needed to be remediated. It is speculated that students’ active classroom engagement and in particular frequent feedback through formative assessment using the mobile classroom application was helpful for enhancing student learning especially for low-performing students.

The study limitations need to be acknowledged. First, this study was conducted using a small sample size due to the small number of student intakes in program under investigation. This may affect the generalizability of the study. Yet, it can be argued that the sample was fairly representative of demographics of pre-med students in Korean medical schools. Still, future study with a larger sample is warranted to enhance the generalizability of the study. Second, although this study found a significant improvement in students’ self-efficacy in English, the findings do not provide direct evidence as to whether such an improvement is the results of the use of mobile technology. According to social cognitive theory, there are four main sources of information that influence students’ self-efficacy: enactive mastery experiences, vicarious experiences, social persuasions, and physiological and psychological states [33]. Further study is warranted to understand how the use of mobile technology influences students’ self-efficacy in English.

Although the mobile technology was adopted with the intention to enhance classroom interactions, no comparison was made in student engagement between when mobile technology was used and when it was not used as this was not an experimental study. Future study is recommended of comparing students’ classroom engagement in two different interventions in order to understand the impact of the use of mobile technology on student engagement. Furthermore, this study used the mobile technology mostly in individual learning settings, such as individual quizzes, and did not focus on using it in small-group settings. Therefore, this study is limited in suggesting how the mobile technology can be used in small-group learning settings. For instance, the group assessment tool in the mobile classroom application has the potential to support team-based learning. Future study is warranted to investigate how the mobile technology can be used effectively to promote student learning in small-group settings so that we can enhance our knowledge base on the effective use of mobile technology in various educational contexts.

This study illustrates how mobile technology can be integrated into the teaching and learning medical English to enhance classroom interactions. It is our experience that the use of mobile classroom application fostered the classroom interaction by encouraging input from every student in the class and sharing it instantly with the whole class. In addition, using the mobile classroom application has the potential to enhance assessment by using more various forms of assessments than that of conventional audience response systems.