Learning English with electronic textbooks on mobile devices: Impacts on university students' vocabulary development.

The current study examined the use of electronic textbooks designed as mobile applications for learning vocabulary in English among Iranian university students. To this end, 95 university students in an experimental (N = 50) and a control group (N = 45) participated in the study. An explanatory sequential mixed methods design was employed and over an academic semester, the participants used either traditional materials or mobile-based electronic textbooks for learning 600 words in English. To assess the outcomes from different learning conditions, receptive knowledge of the target vocabulary items was tested in three junctures of time (i.e. pre-, post-, and delayed post-test). Additionally, open-ended questionnaires and interviews were used to collect qualitative data from the experimental group to further investigate their perceptions of using mobile-based electronic textbooks for vocabulary learning. The findings revealed a significant main effect for time and both groups significantly improved their vocabulary knowledge from pre-test to post-test. Moreover, a significant main effect was found for using electronic textbooks on mobile devices, and the experimental group outperformed the control group on the post- and delayed post-tests. The qualitative findings revealed three perceived benefits, namely episodic learning, easy access to materials, and enhanced enjoyment for mobile assisted vocabulary learning through electronic textbooks. The perceived challenges were related to health concerns, distractions associated with mobile environments, and external pressure resulting from excessive mobile use among the participants. In general, the findings of the study shed light on the potential offered by mobile-based textbooks for learning English vocabulary, with implications for teachers and materials developers in language teaching programs.

Introduction

In recent years, the use of electronic textbooks in educational contexts is gaining increased attention (Almekhlafi, 2021; Korat & Shneor, 2019; Martin-Beltrán et al., 2017; Raynaudo & Peralta, 2019; Rockinson- Szapkiw et al., 2013; Shepperd et al., 2008; Sung & Wu, 2017). With the integration of various multimedia features and reading support cues, electronic textbooks provide learners with new affordances that traditional books may not be able to cater (Carla et al., 2018; Liu & Leveridge, 2017). These developments are transforming the nature of literacy from a set of static skills to more complex, fluid, and multifaceted competencies that involve interacting with multimodal texts on digital screens. The enhanced affordances of electronic textbooks include the possibility of extending learning to anywhere and anytime, the audio narration of texts, immediate feedback, and multimedia annotation, to name just a few (Dore et al., 2018). It has been argued that such features render electronic textbooks as promising tools for language learning and teaching (Zhang et al., 2020). Nevertheless, much of the existing research in this area investigated the use of electronic textbooks on personal computers (PCs) or tablet devices, and although modern smartphones are produced with larger screens and high processing capabilities, research focusing on their affordances for the use of eBooks remains largely limited. As many students are attending language classes with mobile or tablet devices on which they do most of their reading and writing (Park et al., 2020; Reiber-Kuijpers et al., 2021; Zhang, 2015), there is a growing need to investigate their associated behavior in these new learning environments.

The current study was primarily motivated by pedagogical considerations and concerns. With the start of the COVID-19 pandemic, all educational activities in Iranian universities including general English (GE) courses moved to online platforms. Given the difficulties for implementing fully communicative and interactive courses in online environments (poor internet connection, lack of appropriate software and infrastructures, etc.), a decision has been made to focus on developing essential vocabulary of English in GE courses offered in the study context. The importance of vocabulary knowledge as an essential aspect to all uses of language has been widely acknowledged in the second language acquisition literature (e.g., Nation, 2001, 2013; Schmitt et al., 2017; Webb & Nation, 2017). Yet, in most English as a Foreign Language (EFL) contexts, many learners fall short of acquiring the 3000 essential English vocabulary, which is crucial for general uses of the language (Schmitt & Schmitt, 2014; Webb & Chang, 2012), and vocabulary instruction receives insufficient attention in language teaching programs (Webb & Nation, 2017). Given the usefulness and importance of the 3000 essential English words, English language teachers are recommended to prioritize them in their instructional practices (Read, 2004). In this regard, the use of electronic textbooks with various multimedia features in smartphone can help EFL learners in learning vocabulary, and the growing body of knowledge in eBook-based language education is providing ample evidence for the positive contributions of electronic textbooks to vocabulary knowledge development (Zhang et al., 2020).

Furthermore, in order to help language learners achieve satisfactory levels in vocabulary knowledge, there is a need for explicit and direct attention to vocabulary in language teaching courses (Vilkaitė-Lozdienė & Schmitt, 2019). Previous research indicated that using mobile devices and associated technologies has been generally effective in teaching and learning vocabulary (Lin & Lin, 2019; Mahdi, 2017). Although mobile learning has a great potential to be implemented in a variety of learning contexts, the focus of much research has been on self-directed and out-of-the-classroom language learning, and affordances of Mobile Assisted Language Learning (MALL) for formal learning environments and curriculum integration remained less explored (Chen et al., 2020; Chwo et al., 2018; Hwang & Fu, 2019). Accordingly, there is a need for empirical research to examine the effectiveness of electronic textbooks used on mobile devices to address the vocabulary learning needs of EFL learners in formal instructional programs. The present study is significant since the growing functionality of mobile devices has provoked increased attention toward implementing MALL around the world (García Botero et al., 2018; Godwin-Jones, 2017; Hou & Aryadoust, 2021; Lin & Lin, 2019; Oliveira et al., 2021; Pratama, 2021), and the findings can inform language teaching programs in designing more effective vocabulary learning materials, and guide teachers in using digital technologies to improve instructional processes.

Literature review

This section provides a general overview of the related studies and situates the current research within the broad context of eBook-based language education by highlighting some gaps in this line of inquiry. According to a recent systematic review, the last decade has witnessed increased empirical attention to the role of electronic textbooks in second language education (Zhang et al., 2020). This growing body of knowledge investigated the use of electronic textbooks on personal computers (PCs), tablets, and other portable devices such as PDAs and e-readers. Research has also examined the contributions of various features provided by electronic textbooks such as multimedia, gamification, personalization of learning, and automatic feedback given to the learners while interacting with digital texts. Moreover, the studies compared learning outcomes attained from electronic textbooks and traditional print-based materials for different age groups and probed into the participants’ psychological states and behaviors.

A number of studies in the literature explored different aspects of reading electronic textbooks on computer screens by university students, however, these studies reported conflicting results. Chou (2012) studied the screen reading behaviors of five graduate students from different disciplines over a period of one month. Collecting data through interviews and observations, the researcher showed that the participants’ eBook reading behaviors were impacted by their proficiency in English, reading purpose, opportunities to read electronic texts, and language learning strategies. More specifically, the study found that graduate students had less preference for intensive and detailed reading of texts on computer screens, and believed that this environment negatively impacted their use of reading strategies. Moreover, in a semester-long study, Chou (2016) investigated 20 EFL university students’ perceptions of reading electronic textbooks for the purpose of responding to the literature. The study found that although university students regarded electronic textbooks as valuable resources, most of them considered eBook reading as a negative experience. However, the study also revealed that the university students’ attitudes towards electronic textbooks improved over time, pointing to the conclusion that giving more time and reading opportunities might develop effective eBook reading habits and strategies. In contrast to these findings, some studies reported positive learning outcomes and attitudes among university students for eBook-based language learning. For example, Huang (2013) investigated first-year university students’ perceptions of an e-Book reading program in Taiwan. The participants were 67 EFL learners at the intermediate level of proficiency, and during a yearlong investigation, they were asked to read one e-Book every week. The study used questionnaires and interviews for data collection. The findings of the study showed that university students had positive attitudes towards their reading experience and regarded electronic textbooks as eco-friendlier than printed books. The participants also believed that the use of electronic textbooks enhanced their motivation for reading, and cultivated more appropriate reading habits among them. Nevertheless, the study also reported some challenges in relation to reading electronic textbooks that included eyestrain and difficulty in handling long texts.

With respect to the use of electronic textbooks on tablet devices, the findings from previous studies mostly provided evidence for their increased effectiveness when compared to personal computers or print-based materials (Carla et al., 2018; Dore et al., 2018; Liang & Huang, 2014; Martin-Beltrán et al., 2017; Richter & Courage, 2017). For example, Daniel and Woody (2013) studied 298 university students’ use and performance in reading electronic textbooks and printed books at home and under laboratory conditions. The study reported that in terms of students’ performance, both material types produced similar results. However, the reading time on tablet devices was significantly higher for reading at home, which was also associated with increasing multitasking in e-Book reading such as checking emails and chatting on social media. Additionally, Lin (2014) investigated the use of tablet devices and personal computers by two 10th grade intact classes in a ten-week extensive reading program. In this study, one class received their reading materials on tablets, and the other class read the same contents on computer screens. The reading program dedicated one classroom time every week for online reading, and the participants were allowed to continue their reading outside the classroom. The findings of the study provided evidence for the effectiveness of tablet devices in helping students to perform better on online tasks and reading achievements. The tablet users also perceived the reading program more positively. In a study with different results, Martin-Beltrán et al. (2017) compared the engagement with digital texts on tablet devices and printed materials among school children. The findings indicated that when using print-based materials, students were more likely to engage in meaningful talk and tended to use text features more to support their comprehension.

Some studies, in particular, investigated the use of electronic textbooks on tablet devices in formal instructional settings. In a large-scale study with 538 participants, Rockinson- Szapkiw et al. (2013) investigated the relationship between textbook format and university students’ grades and learning scores. The findings revealed that reading on e-readers and tablet devices was beneficial, and those students that used electronic textbooks attained more effective learning and psychomotor development when compared to those who used traditional and print-based materials. In a recent study, Lin (2017) investigated the use of tablet devices for reading textbooks among four intact classes of 11th graders in a high school in Taiwan. In this study, one group of the students read and accessed their textbooks on tablet devices only, while another group used printed books. The results of the study showed no significant differences among the two groups in terms of various achievements; nonetheless, those using tablet devices reported positive experiences in using electronic textbooks and perceived them as useful and easily available. Furthermore, in a case study of four international students in the USA, Park et al. (2020) examined the reading of digital texts on tablet devices over a period of eight months. The researchers collected data from different sources including observations, verbal reports, interviews, and field notes. The study revealed that the participants employed a variety of reading strategies and had both positive and negative experiences in using tablets for reading. The ability to interact with others while reading, the availability of authentic materials and topics, and the possibility to read electronic textbooks at any time and any place were identified among the perceived benefits. Moreover, small screen size, distraction, internet disconnection, and technological malfunctions were the perceived negative experiences.

Moreover, research also examined the impacts of different enhanced features in electronic textbooks, which included the integration of multimedia, giving feedback, gamification, and personalization of the learning content (Zhang et al., 2020). For example, Wu (2016) investigated the contribution of a feedback and content personalization mechanism integrated into an English-reading eBook system to analyze the students’ reading profile in terms of article difficulty, relevance of the content, and the learner's language ability. The findings indicated that the developed guidance mechanism provided the participants with appropriate recommendations for further reading materials and hence contributed positively to their improved reading comprehension. Given the relevance and the implications of the cognitive theory of multimedia learning (Mayer, 2014) for eBook-based language learning, the majority of the studies within this strand focused on the impacts of different multimedia content (visual, audio, dynamic pictures, etc.) (Korat, 2010; Korat et al., 2014). Very broadly, the cognitive theory of multimedia learning asserts that learning is much more effective when the new content is presented with a combination of pictures (visuals) and words, rather than words alone (Mayer, 2014). Recently, Liu et al. (2019) used eye-tracking technology to study the online reading behavior of 52 advanced learners of English. In this study, the participants read electronic textbooks with six support features at the micro and macro level that included glosses, vocabulary focus, footnotes, illustrations, infographics, and photos. The results of the study showed that when participants were reading the electronic textbooks for learning vocabulary, vocabulary focus (explaining the meaning of new words) and glosses received significant fixations. However, in reading for text comprehension, the participants focused more on illustrations and visual aids.

In sum, the review of the studies investigating the integration of electronic textbooks into language education highlighted the potential and various affordances of electronic texts delivered on PC or tablet devices for language learning. Nevertheless, there are some gaps in the literature demanding further empirical research. Firstly, as mentioned earlier, research on the use of electronic textbooks on smaller platforms such as smartphone devices remained limited. It has been claimed that the rise of smartphones has initiated a new era in human-machine relationships, which has a huge potential to fundamentally transform educational practices (Godwin-Jones, 2017). The accumulated body of knowledge in MALL also indicates that the implementation of various mobile technologies significantly contributed to the students’ learning outcomes and improved language learning (Abu Saaleek, 2014; Burston, 2013, 2014; Chen et al., 2020; Chwo et al., 2018; Godwin-Jones, 2011; Golonka et al., 2014; Hwang & Fu, 2019; Stockwell, 2013; Sung et al., 2015; Wong & Looi, 2011). In this regard, as mobile devices provide learners with various affordances and opportunities for new forms and cultures of learning, it is of significant importance to investigate these emergent learning environments, as such undertakings help us to better understand the learners’ associated behaviors and performances (García Botero et al., 2019; Ko, 2017). Secondly, the majority of the studies discussed above were conducted in short-term periods, and employed pre- and post-test designs to investigate the learning outcomes of eBook-based language learning. As a result, the long-term impacts or delayed effects of the interventions remained less explored, and this shortcoming has made it difficult to understand the true impacts of using various devices in the development of second language knowledge among the participants. Thirdly, only a small number of studies employed qualitative or mixed methods (Zhang et al., 2020), and investigation of the learners’ perceptions and experiences in using electronic textbooks for language learning received far less attention. Moreover, research investigating the use of electronic textbooks by university students in less developed countries is scarce, and there is a need for balancing the literature by giving more voice to those language learners in less represented contexts. Considering the above-mentioned gaps in the literature, the current study aimed to investigate the use of electronic textbooks designed as mobile applications for learning English vocabulary by Iranian university students, and addressed the following research questions:

Which material type (paper based or electronic textbooks on mobile devices) is more effective in enhancing the participants’ knowledge of vocabulary in English over time?

What are the perceived benefits and challenges of using electronic textbooks on mobile devices for vocabulary learning among the university students?

Methodology

Participants

The participants of the study were 95 (36 males and 59 females), first-year university students, majoring in Psychology, Accounting, Urban Geography, and Architecture. The mean age of the participants was 19, and they were selected based on their availability to the researchers from two universities in Iran. The participants were divided into an experimental (N = 50) and a control (N = 45) group. Despite studying English for some years (at least 6 years in the state-run schools), the majority of the participants had limited command of English vocabulary (see the Results section) which is also the case in some other EFL learning contexts (Webb & Chang, 2012). All participants in the experimental group owned Android or iPhone smartphone devices with appropriate screen sizes (above 4.5 inches) for installing and using the mobile application intended for the current study. The study adhered to ethical considerations by obtaining informed consent from the participants, and ensuring the confidentiality of obtained data through various means.

Materials and instruments

The widely used textbook ‘4000 Essential English Words 1’ (Nation, 2009) was employed as the main material for learning English vocabulary in the GE course. This textbook is the first volume in a six-book series that aim to teach essential and high-frequency words in English (both spoken and written). Each textbook contains 30 units, and 20 essential words are introduced in every unit. Participants in the control group used the printed textbook (with associated CDs for audio files), while the experimental group used the equivalent mobile application version of the book that is accessible via Google Play or iPhone App stores (Kelyn Le Studio, 2020). Despite being different in format, the content presented to the participants in the printed books or e-Books including the vocabulary items in each unit and associated exercises were the same and resembled each other exactly. However, the e-Book version of the material had integrated audio files where the participants had the option to use the narration of the text while reading, while the printed book users had to use audio CDs downloadable to MP3 players or mobile devices. Moreover, the e-Book version provided the users with instant feedback on learning activities (indicating whether the selected responses were true or false), but the traditional books had the answer key at the end of the book.

In order to measure the participants’ receptive vocabulary knowledge prior to, and after the treatment, the new vocabulary levels test (NVLT) (McLean & Kramer, 2015) was used. The NVLT is one of the validated and standard tests of English vocabulary for measuring the receptive knowledge of English words in 1000 to 5000 frequency levels (Nation, 2012) that employs multiple choice question types. More specifically, the NVLT is organized around five levels, each containing 24 questions. The scores were calculated by adding the number of correct responses in each level (the maximum score for this test is 120). Accordingly, the NVLT provides a pedagogically useful instrument for assessing the vocabulary knowledge of EFL learners, as it is possible to discern improvements in different frequency levels. In addition to the NVLT, three vocabulary knowledge tests (VKT) were also administered. These tests were developed for the purpose of assessing the receptive knowledge of 600 target words introduced in the course material. In each test, following procedures employed by earlier studies (Wu, 2015a, b), 100 items were randomly selected from target words, and participants’ knowledge of those items was tested using a multiple choice test format. The scores in this test represented the number of correct responses out of 100. The validity of the locally developed tests was examined in relation to the NVLT by administering the tests to a similar group of EFL learners (N = 30), and the measures demonstrated acceptable concurrent validity (Frey, 2018).

Additionally, for the qualitative phase of the study, first, an open-ended questionnaire (Dörnyei & Taguchi, 2009) was used to investigate mobile application users’ attitudes towards mobile-assisted vocabulary learning in terms of perceived benefits and challenges. The questions addressed different aspects of MALL discussed in the literature, including perceived benefits and challenges (Dashtestani, 2012, 2016; Dashtestani & Hojatpanah, 2021), learner attitudes and perceptions (Hsu, 2013; Metruk, 2021; Pengnate, 2018), and mobile technology acceptance for language learning (Hoi, 2020; Hoi & Mu, 2021; Hsu & Lin, 2022). The link for the open-ended questionnaire was sent to the participants in the experimental group using a social media platform (i.e., WhatsApp). Second, in order to obtain further insights regarding the participants’ experiences in using electronic textbooks on mobile devices for vocabulary learning, chat-based interviews (Lannutti, 2017) were conducted with 15 participants in the experimental group. The interviews were guided by the same questions in the questionnaire, but probing techniques were used to obtain more detailed responses from the participants (Given, 2008).

Data collection and procedures

At the start of the academic semester, the NVLT was administered to the university students in the four intact classes. After assigning the participants to either experimental or control groups, they were provided with the course materials, and the first VKT was administered. During the semester, students in both groups received online instruction on a weekly basis (70 min, 15 weeks) by two English language teachers (M.A. holders in TEFL), and two units (containing 40 words) were covered every week. In order to control for different teaching approaches, each teacher had one class assigned to the experimental learning condition and one class to the control group. Moreover, one of the researchers held two online training sessions related to vocabulary teaching techniques and strategies for the teachers and worked closely with them during the research period to ensure the implementation of the same instructional procedures for all participants. Teachers used an online teaching platform (SKYROOM: https://www.skyroom.online/) to conduct online classes, and allocated 30 min in every session for reviewing previously learned vocabulary items, and also dealing with students’ vocabulary learning problems. Moreover, a locally popular social media application (i.e. WhatsApp) was used to send weekly study reminders and notifications regarding the upcoming tests. At the end of the academic semester, the participants’ vocabulary knowledge was tested using NVLT, and the second VKT was developed based on course material. In order to investigate students’ attitudes towards mobile-assisted vocabulary learning, those in the experimental group were given an open-ended questionnaire, and chat-based interviews were conducted with 15 participants. Finally, to assess the long-term impacts of the interventions, participants’ vocabulary knowledge was tested in a delayed post-test three months later using the NVLT and the third VKT.

Data analysis

The quantitative data were analyzed for both descriptive and inferential statistics using IBM SPSS software version 23. To this end, descriptive statistics including mean values and standard deviations were calculated for scores on pre-, post-, and delayed post-tests. In order to see if the observed differences were statistically significant, a mixed between-within subjects analysis of variance (Pallant, 2016) was used. The between-subjects variable was group (experimental or control), and the within-subject variable was time with three levels (pre-, post-, and delayed tests). As for the qualitative data, the participants’ responses obtained via the open-ended questionnaire and interviews were analyzed through thematic analysis using both deductive and inductive techniques (Braun & Clarke, 2006; Fereday & Muir-Cochrane, 2006). In this regard, to have a comprehensive and balanced view of the data (Xu & Zammit, 2020), in the first step of analysis, pre-defined codes such as ‘attitudes’ and ‘perceived ease of use’ were used to label different parts of the data. This deductive coding was followed by a process of inductive coding to identify other significant aspects in the participants’ responses (such as ‘distributed learning’) which were related to the study and were not labeled in the first step. Finally, the evolving themes as specific patterns being shared across the data that capture crucial information in relation to the research questions were identified and grouped under two main categories as benefits and challenges. In order to ensure the credibility of the findings, the following steps were taken. First, as suggested in the literature related to qualitative research (Campbell et al., 2013; O’Connor & Joffe, 2020), the consistency of the coding process was checked by re-analyzing around 25% of the data two months after the first analysis (i.e. interviews conducted with four participants selected randomly out of 15). The results of intra-rater reliability calculated by Cohen’s Kappa indicated substantial agreement between the two coding sessions (Cohen’s Kappa = 0.70). Additionally, two colleagues familiar with qualitative data analysis methods were asked to review the data, coding processes, and findings. Disagreements were resolved through discussions, recoding some parts of the data, and renaming some themes.

Findings

Quantitative results

The experimental and control groups’ performances in pre-, post-, and delayed post-tests on the two measures of the New Vocabulary Levels Test (NVLT) and Vocabulary Knowledge Test (VKT) were analyzed. The results revealed that the groups’ performances were similar in the pre-test for both measures. The mean value for the scores on the NVLT for the control group was 57.73 (SD = 3.38), and for the experimental group was 57.06 (SD = 3.58). Both groups also had similar scores on VKT, and the mean values for this measure were 49.33 (SD = 3.42) and 50.08 (SD = 3.26) for the control and experimental groups respectively. As for the post-tests which were administered at the end of the semester, the mean values for the obtained scores on NVLT were 62.40 (SD = 4.07) for the control group and 65.58 (SD = 4.28) for the experimental group indicating better performance. Participants in the experimental group also scored higher compared to those in the control group on the second VKT, and the mean value for their scores was 72.18 (SD = 4.28), which was considerably higher than the scores obtained by the control group (M = 66.69, SD = 3.90). The same pattern was found for the scores on delayed post-tests, and the experimental group showed better performance than the control group on NVLT (M = 67.98, SD = 4.02, control group: M = 61.18, SD = 3.99) and VKT (M = 80.46, SD = 4.68, control group: M = 67.02, SD = 4.00).

Figure 1 is a visual representation of the mean scores for the NVLT on pre-, post-, and delayed post-test. From pre-test to post-test, both groups improved their scores, however, those participants in the experimental group benefited more from the intervention. Nonetheless, from post-test to delayed post-test, the obtained scores on NVLT had different patterns. In this regard, while those in the experimental group scored higher in delayed post-test compared to both pre- and post-test, those in the control group attained higher scores in delayed post-test only in comparison to pre-test, and not post-test.

Fig. 1

Visual representation of the mean scores for NVLT on pre-, post-, and delayed post-test

Figure 2 is a visual representation of the mean scores for VKT on pre-, post-, and delayed post-test. Similar to obtained scores on NVLT, both groups improved their scores considerably from pre-test to post-test, although those in the experimental group gained better results. However, from post-test to delayed post-test, an improvement in scores was found for the experimental group, but not for the control group.

Fig. 2

Visual representation of the mean scores for VKT on pre-, post-, and delayed post-test

In order to find out if the observed differences among the groups were statistically significant, a mixed between-within subjects analysis of variance was performed. Before conducting the test, the assumptions of homogeneity of variances (Levene’s Test of Equality of Error Variances) and equality of covariance matrices (Box’s Test of Equality of Covariance Matrices) were checked and no violation was observed. Table 1 shows the results of multivariate tests. The results showed a significant interaction between the two variable of group and time, Wilks’ Lambda = 0.32, F (4, 90) = 48.48, p = 0.000, ηp2 = 0.683. This interaction effect indicates that the impact of one variable is influenced by the level of the second variable, and the general conclusions on the main effects of the variables should be interpreted carefully. The findings also revealed a significant effect for time (Wilks’ Lambda = 0.036, F (4, 90) = 604, p = 0.000, ηp2 = 0.964) and for group in general (Wilks’ Lambda = 0.252, F (2, 92) = 136.46, p = 0.000, ηp2 = 0.748). As a result, there was a significant difference among the two groups on the one hand, and also a significant change in NVLT and VKT scores across the time periods (i.e. pre-, post-, and delayed post-test). The findings also suggest a very large effect size for the observed differences (Cohen, 1988).

Table 1

Multivariate Testsa

Effect			Value	F	Hypothesis df	Error df	Sig	ηp2
Between Subjects	Intercept	Pillai's Trace	0.999	84,758.290b	2.000	92.000	0.000	0.999
		Wilks' Lambda	0.001	84,758.290b	2.000	92.000	0.000	0.999
		Hotelling's Trace	1842.572	84,758.290b	2.000	92.000	0.000	0.999
		Roy's Largest Root	1842.572	84,758.290b	2.000	92.000	0.000	0.999
	Group	Pillai's Trace	0.748	136.464b	2.000	92.000	0.000	0.748
		Wilks' Lambda	0.252	136.464b	2.000	92.000	0.000	0.748
		Hotelling's Trace	2.967	136.464b	2.000	92.000	0.000	0.748
		Roy's Largest Root	2.967	136.464b	2.000	92.000	0.000	0.748
Within Subjects	TIME	Pillai's Trace	0.964	604.007b	4.000	90.000	0.000	0.964
		Wilks' Lambda	0.036	604.007b	4.000	90.000	0.000	0.964
		Hotelling's Trace	26.845	604.007b	4.000	90.000	0.000	0.964
		Roy's Largest Root	26.845	604.007b	4.000	90.000	0.000	0.964
	TIME * Group	Pillai's Trace	0.683	48.480b	4.000	90.000	0.000	0.683
		Wilks' Lambda	0.317	48.480b	4.000	90.000	0.000	0.683
		Hotelling's Trace	2.155	48.480b	4.000	90.000	0.000	0.683
		Roy's Largest Root	2.155	48.480b	4.000	90.000	0.000	0.683

aDesign: Intercept + Group.

Within Subjects Design: TIME.

bExact statistic.

Table 2 shows the results for the tests of between-subjects effects, in relation to the two measures of NVLT and VKT. The main effect for the NVLT (F (1, 93) = 49.687, p = 0.000, ηp2 = 0.348), and VKT (F (1, 93) = 213.069, p = 0.000, ηp2 = 0.696) were both significant, which means statistically significant differences in the performances of the experimental and control groups, resulted from using electronic textbooks on mobile devices or traditional materials. The effect size of the differences was also very large (Cohen, 1988).

Table 2

Tests of Between-Subjects Effects

Transformed Variable:	Average
Source	Measure	Type III Sum of Squares	df	Mean Square	F	Sig	ηp2
Intercept	NVLT	1,092,100.669	1	1,092,100.669	79,318.516	0.000	0.999
	VKT	1,174,848.999	1	1,174,848.999	81,904.947	0.000	0.999
Group	NVLT	684.122	1	684.122	49.687	0.000	0.348
	VKT	3056.270	1	3056.270	213.069	0.000	0.696
Error	NVLT	1280.475	93	13.769
	VKT	1333.997	93	14.344

In order to further investigate the observed differences in more detail, a series of pairwise comparisons were conducted (Tables 3 and 4). Table 3 represents the results of pairwise comparisons for the scores on NVLT and VKT. Using Bonferroni’s adjustment for multiple comparisons in SPSS software, it was found that the differences observed in groups’ performances in the pre-test were not statistically significant on both NVLT and VKT measures (p = 0.350 and p = 0.279 respectively), which indicates that the groups were similar with respect to their vocabulary knowledge prior to the treatments. Nonetheless, comparisons for the same measures on post- and delayed post-test indicated significant differences among the control and experimental groups (p = 0.000).

Table 3

pairwise comparisons of scores on NVLT and VKT among the groups

Dependent Variable	(I) Group	(J) Group	Mean Difference (I-J)	Std. Error	Sig.b	95% Confidence Interval for Differenceb
						Lower Bound	Upper Bound
NVLT1	Control	Experimental	0.673	0.716	0.350	-0.749	2.095
VKT1	Control	Experimental	-0.747	0.686	0.279	-2.110	0.616
NVLT2	Control	Experimental	-3.180*	0.859	0.000	-4.887	-1.473
VKT2	Control	Experimental	-5.491*	0.842	0.000	-7.164	-3.818
NVLT3	Control	Experimental	-6.802*	0.823	0.000	-8.436	-5.168
VKT3	Control	Experimental	-13.438*	0.898	0.000	-15.222	-11.654

Based on estimated marginal means

*The mean difference is significant at the .05 level.

bAdjustment for multiple comparisons: Bonferroni.

Table 4

pairwise comparisons of obtained scores across time.

Group	Measure	(I) TIME	(J) TIME	Mean Difference (I-J)	Std. Error	Sig.b	95% Confidence Interval for Differenceb
							Lower Bound	Upper Bound
Control	NVLT	1	2	-4.667*	0.774	0.000	-6.593	-2.740
			3	-3.444*	0.780	0.000	-5.386	-1.503
		2	1	4.667*	0.774	0.000	2.740	6.593
			3	1.222	0.870	0.502	-0.944	3.388
		3	1	3.444*	0.780	0.000	1.503	5.386
			2	-1.222	0.870	0.502	-3.388	0.944
	VKT	1	2	-17.356*	0.815	0.000	-19.383	-15.328
			3	-17.689*	0.753	0.000	-19.562	-15.816
		2	1	17.356*	0.815	0.000	15.328	19.383
			3	-0.333	0.756	1.000	-2.215	1.549
		3	1	17.689*	0.753	0.000	15.816	19.562
			2	0.333	0.756	1.000	-1.549	2.215
Experimental	NVLT	1	2	-8.520*	0.856	0.000	-10.642	-6.398
			3	-10.920*	0.839	0.000	-13.000	-8.840
		2	1	8.520*	0.856	0.000	6.398	10.642
			3	-2.400*	0.780	0.010	-4.335	-0.465
		3	1	10.920*	0.839	0.000	8.840	13.000
			2	2.400*	0.780	0.010	0.465	4.335
	VKT	1	2	-22.100*	0.829	0.000	-24.154	-20.046
			3	-30.380*	0.827	0.000	-32.430	-28.330
		2	1	22.100*	0.829	0.000	20.046	24.154
			3	-8.280*	0.949	0.000	-10.632	-5.928
		3	1	30.380*	0.827	0.000	28.330	32.430
			2	8.280*	0.949	0.000	5.928	10.632

Based on estimated marginal means

*The mean difference is significant at the .05 level.

bAdjustment for multiple comparisons: Bonferroni.

The results of the multiple pairwise comparisons for obtained scores on NVLT and VKT are shown in Table 4. With respect to the NVLT, the participants in the control group (using the traditional materials) attained significantly better results on both post- and delayed post-test compared to pre-test, nonetheless, the changes in their scores from post-test to delayed post-test were not statistically significant. A similar pattern was found for the control group’s performance on VKT. For the experimental group that used electronic textbooks on mobile devices, the analysis revealed that they obtained significantly higher scores on both measures across time. In this regard, they attained better results on NVLT and VKT on post-test, and also significantly improved their scores on these measures on delayed post-test.

Qualitative results

Perceived benefits of using mobile applications for vocabulary learning

The analysis of the obtained data via open-ended questionnaires and chat-based interviews revealed three major benefits of mobile assisted vocabulary learning as perceived by the participants in the experimental group. In this regard, one emergent theme from data was episodic learning affordance provided by mobile devices. From the participants’ perspective, mobile devices enabled them to use small amounts of time to undertake episodic learning experiences that were distributed over days. From the participants’ views, this contributed significantly to improvements in their vocabulary knowledge:

I think mobile devices are really great in using small times for learning something new. I am using my phone to study English vocabulary on public transport. The best thing is that I don’t have to carry around books, and using these times for learning vocabulary seems to be the best thing I can do in such places. (P#12, interview)

I am using my phone to learn English vocabulary when I want to do something different while studying other subjects. Sometimes this really helps, and gives me more energy. After studying math for two hours, a 10- or 15-minute vocabulary study is a great experience. Generally, I don’t spend specific times on English course, but use these [short] times and I am doing great I think. (P#3, interview)

Another major theme in the participants’ responses was related to various affordances provided by mobile devices for having easy access to different learning materials, stored in one place (i.e. smartphone devices). In this respect, the participants highlighted the benefits of electronic textbooks employed alongside other applications such as electronic and online dictionaries, note-taking applications, and media players for their learning of English vocabulary. One notable aspect of the participants’ responses within this theme was related to easy access to audio files associated with the mobile application that made it possible for them to listen to the course content as a strategy for reviewing new vocabulary items. The following excerpts demonstrate these aspects:

One of the best things in using mobile applications for learning vocabulary is that all things needed are in your pocket. I no longer need to save audio files in my phone or computer [because the app contains audio files in it]. It is also possible to copy words from the app and look them up in a dictionary which is also right there! (P#27, Questionnaire data)

The mobile application we used in this course was great in many respects. First, the app was complete in that audio files were in the app, and there was no need to play them using a media player. Second, it is possible to take screenshots from the words that need more attention for later reviews. (P#31, interview)

I am using the internet to find out more about some interesting words, or to find synonyms and [L1] translations. I also take notes on my study habits to see how much time I am spending on learning English vocabulary. (P#42, interview)

Another theme in the participants’ responses was related to various motivational aspects of mobile assisted vocabulary learning. A number of participants reported increased use of mobile devices for language learning as a result of more enjoyment of learning, and having more hope and a positive outlook towards the future:

I think the focus on learning vocabulary with mobile phones is a good way for language learning. I want to improve my English vocabulary, and my plan is to finish the series [4000 essential words] in a year. I have all the other 4000 applications already installed on my phone. (P#19, interview)

I have been to a lot of language classes in the past, both in schools and private institutes. But I think that learning English with mobile on my own, and at times that I decide myself is more fun and effective. (P#22, questionnaire)

Perceived challenges of using mobile applications for vocabulary learning

The analysis of qualitative data also revealed three major challenges faced by the participants in using electronic textbooks as mobile applications for vocabulary learning. In this regard, a group of participants mentioned that using mobile applications as the course and study material created some wrong impressions among their friends or family members:

When I was using the app at home for studying course material and learning English vocabulary, I had to explain it all the time to my parents that I’m studying. These days when you are using your phone, everybody thinks that you are in web, or chatting with your friends. My parents seem to be a little sensitive to mobile use, as they believe this negatively influences my studies. (P#16, questionnaire)

I remember using the app for studying while we were in a family gathering. I think everyone was thinking that I was chatting with my friends! When I realized this, I stopped using the phone, and decided to study in another time. You know, some might consider this as disrespect to the guests. (P#27, interview)

The distraction caused by social media was also a challenge in using mobile phones and associated applications for language learning. A number of participants in the experimental group made some references to this issue, and highlighted that this type of distraction caused them to engage in entirely different activities while studying:

We are all using social media these days. When studying with my mobile phone, I had to respond to messages sent to me by my friends, and sometimes this was a real distraction for me. Sometimes, we spent a lot of time chatting, and some other times I completely forgot what I was doing and what I was studying! You can’t also concentrate well while messages keep coming every second! (P#32, questionnaire)

Finally, some participants considered health concerns among the serious challenges for mobile-assisted vocabulary learning, and mentioned the small screen size of devices, eyestrain, and possible negative impacts of using mobile devices on their bodies as the main sources of their worries:

The use of mobile devices sometimes leads to headaches, especially when using them in moving cars. I had several such bad experiences. Besides, holding the phone with your hand and trying to carefully read what is on the screen is not without challenges, and sometimes I felt slight pain in my hands and my neck. (P#19, questionnaire)

I believe one should be extra careful when using mobile devices for long times. We are already using them a lot in our lives, and using them to study may add to their negative side effects, such as eyestrain, and inappropriate sitting habits which may lead to chronic pains. (P#43, questionnaire)

Discussion

The first research question was concerned with the relative effectiveness of printed books and electronic textbooks as mobile applications for vocabulary development among university students. The findings indicated that both groups improved their vocabulary knowledge from the pre-test to the post-test, nonetheless, the experimental group that used electronic textbooks on their smartphone devices outperformed the control group. Furthermore, a significant main effect for time was found for using electronic textbooks as mobile applications in vocabulary learning. This is in line with previous studies that reported similar results for the relative effectiveness of electronic textbooks compared to print-based materials (Lin, 2014, 2017; Richter & Courage, 2017; Rockinson- Szapkiw et al., 2013). However, the findings of the current study are incongruent with some earlier studies (Chou, 2012, 2016; Martin-Beltrán et al., 2017). As both types of materials were found to be effective in developing vocabulary knowledge, it seems that the delivery of learning content on mobile devices with well-designed applications is a viable approach for mobile-assisted language learning. Given the growing popularity of mobile devices among language learners (Godwin-Jones, 2011, 2017), language teachers and materials designers should make more use of such devices to provide learners with increased opportunities for language learning in general and vocabulary learning in particular.

The findings also indicated that from the pre-test to the post-test, developments in vocabulary knowledge followed a similar pattern across the groups (albeit with different degrees); nonetheless, from the post-test to the delayed post-test, this development followed different trajectories. As for VKT which was a test of vocabulary items delivered to the participants via different materials, there was no significant change in the control groups scores from post-test to delayed post-test (Fig. 2 and Table 4). However, the scores obtained by this group on NVLT (a standard test of English vocabulary knowledge) decreased by 1.22 points on average during the same time. Although this level of loss in vocabulary knowledge was not statistically significant (Table 4), nevertheless, it points to a declining trend that might yield significant differences if tested in future times. In contrast, those participants in the experimental group improved their scores on both measures over time, and the observed differences were significant (Table 4). There might be several reasons for these findings. First, the changes in NVLT scores are a reflection of vocabulary gains attained by the participants. Hence, as the experimental group scored higher on VKT (learned more target words), they also performed better on NVLT. Second, these findings might be best interpreted in light of the so-called ‘inherent motivational effects’ of new technologies for language learning (Stockwell, 2013). This perspective holds that the introduction of new technologies for language learning results in enhanced motivation and hence better performance among learners. For the participants in the experimental group, the use of electronic textbooks on mobile devices for vocabulary learning created a more engaging environment that contributed to their overall vocabulary improvement.

The second research question explored the perceived benefits and challenges associated with using electronic textbooks for mobile-assisted vocabulary learning from the participants’ perspective. As for perceived benefits, the analysis of qualitative data revealed three main benefits, namely (1) episodic learning, (2) easy access to learning materials, and (3) positive emotional loading (fostering motivation). The issues of access and motivation are well established in MALL literature (Chwo et al., 2018). The findings from the current study point to additional factors including episodic and enhanced enjoyment resulting from the use of electronic textbooks on mobile devices. This is partly in line with the findings reported by Park et al. (2020) as international students in the US also reported enjoyment in reading electronic textbooks, and highlighted the affordance offered by tablet devices to extend learning beyond the classroom. Moreover, the findings from the qualitative dimension of the study also helped better understanding of the quantitative results. The participants in the experimental group reported using a mobile application for vocabulary learning in small times during the study period, which is in line with any time and anyplace learning promise of mobile learning in general. This helped them to go beyond the time and physical constraints of traditional learning, and the portability of mobile phones further facilitated this process. As for enhanced enjoyment of learning, recent postulations of language learning motivation highlighted the crucial role of emotions in the learning process (Dewaele & Li, 2018; Henry et al., 2015; MacIntyre & Vincze, 2017; Oxford, 2015; Richards, 2020). This positive emotional loading of learning with mobile devices made the participants in the experimental group spend more time and effort on learning English vocabulary, which also resulted in their improved performance over time.

Finally, with respect to the challenges that the participants faced in mobile assisted vocabulary learning, distraction, health concerns, and misunderstandings of different uses of mobile devices were mentioned as the major drawbacks of using mobile applications for language learning. As highlighted by Park et al. (2020), distraction seems to be a persistent challenge in using electronic textbooks on mobile and tablet devices. However, it should be emphasized that in contrast to the findings reported by Park et al. (2020), small screen size and technological malfunctions are not identified as the challenges for the participants of this study. This might have resulted from the use of modern smartphones, and electronic books developed in the form of mobile applications in this study. Reading PDF files might be difficult on small screens, nonetheless, when designed as mobile applications, the same content might be adjusted to screen size, and provide learners with much-improved interface for content delivery. In this regard, although mobile devices bring extensive opportunities for learning and teaching in new ways that promise transforming education, educators and material designers should be aware of the downside of such practices. One major issue is related to health concerns raised by the participants. Moreover, increased addiction to mobile devices (Chen, 2020; Gong et al., 2020) among the younger generation and parental concerns about excessive use of such devices make it necessary for teachers and parents to create new cultures of learning, and in this process, it is of prime importance to raise the awareness of the involved parties on benefits and drawbacks of using modern mobile devices in education. This brings more responsibility to the teachers that implement mobile assisted learning, as they need to discuss these concerns with students, and be aware of the possible negative impacts of mobile use.

Conclusion

The study aimed to compare the associated impacts of using traditional materials and electronic textbooks developed in the form of mobile applications as the main course material in formal education on English vocabulary development among university students. Perceived benefits and challenges of using mobile devices and electronic textbooks for vocabulary learning were also investigated. The findings revealed that the use of mobile devices as a delivery medium for target vocabulary items was effective in both short- and long-term periods. There were also various benefits of electronic textbooks for mobile-assisted vocabulary learning as identified by the participants, including episodic learning affordance, easy access to all learning materials in one place, and enhanced motivation and enjoyment in the learning process. On the other hand, some perceived challenges were health concerns, distraction caused by other mobile applications such as social media, and a lack of mobile learning culture among the students and their parents. These findings promise implications for using electronic textbooks in MALL in general and mobile-assisted vocabulary learning in particular. First, given the effectiveness of mobile applications in delivering the learning content, there is a need to produce high-quality and user-friendly materials in order to utilize anytime and anywhere learning affordances of mobile devices. More specifically, such considerations are crucial given the small screen size of smartphone devices. As the findings indicated, electronic textbooks developed as mobile applications are used conveniently by university students. Second, given the motivational benefits of using new technologies in education which results in increased engagement of the students with the learning materials, educators and teachers can make use of mobile technologies to complement traditional materials. With the global impacts of the COVID-19 pandemic on formal education, such materials can help educators in delivering high quality content in remote teaching. Third, as learning foreign language vocabulary pose a major challenge for EFL students, it seems that the exploitation of mobile devices and associated application have huge potentials to help the students and teachers to set vocabulary learning goals, and use new forms of learning to bridge the vocabulary gap in the language learning process. The use of mobile technologies provides learners with increased exposure to target vocabulary items, that facilitates their learning, retention, and recognition.

The current study had some limitations that should be acknowledged. First, as a common practice in classroom-based research, intact classes were employed to investigate the use of electronic textbooks which posed some limitations on the obtained outcomes and the generalizability of the findings. Second, although special care was taken to control for some intervening variables (such as teacher and instructional differences), controlling some other variables including extra exposure to the English language through different media and other venues was not possible. This concern is serious for both groups given the duration (8 months) of the study, and with the experimental group in particular, as by being more users of smartphone devices, they had by nature much easier access to other online resources and materials for learning English vocabulary. In this regard, even though the study employed various tests and standard procedures, such important factors should be taken into consideration when interpreting the findings. A further limitation arises from the focus of the study on just one aspect of vocabulary knowledge in the measurements of learning outcomes. As learning a word also entails some other important dimensions in addition to the form-meaning connection, such as collocations, grammatical function, fluency, and associations (Nation, 2013, 2019), there is a need to investigate the impacts of mobile assisted vocabulary learning using more comprehensive measures of vocabulary knowledge (Read, 2019). Bearing in mind these limitations, the study provided empirical evidence for the effectiveness of electronic textbooks used on mobile devices in enhancing vocabulary learning among EFL students, and the findings promise important implications for vocabulary learning. Nonetheless, more research is needed in this area to shed light on other dimensions of the implementation of various mobile technologies on students’ vocabulary developments across age groups, proficiency levels, and different learning contexts.