A synthesis of systematic review research on emerging learning environments and technologies.

In this introduction to the special issue on systematic reviews on emerging learning environments and technologies, we introduce best practices for conducting systematic reviews and meta-analysis and discuss the need for a systematic review on emerging learning environments and technologies. We synthesize research on seven primary areas of emerging learning environments and technologies that include: (1) social media, (2) massive open online courses, (3) special education technology, (4) mobile learning, (5) game-based learning and gamification, (6) adaptive learning, and (7) learning analytics and introduce the thirteen articles that were included in this special issue. This article also provides implications for the readers on using and conducting systematic reviews. © Association for Educational Communications and Technology 2020.

Introduction

In recent years, there has been an increase in the number of systematic reviews and meta-analyses employed to synthesize research from primary studies. Systematic reviews aim to examine secondary data by retrieving, synthesizing, and assessing existing knowledge on a subject in a logical, transparent, and analytical manner. Systematic reviews are not literature reviews or narrative reviews but follow a specific methodology to locate, select, evaluate, analyze, synthesize, and report evidence that helps to draw conclusions (Denyer and Tranfield 2009).

There are guidelines on how to conduct systematic reviews. The U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 (2017). The What Works Clearinghouse (WWC) reviews and profiles high quality research in education to assist educators to make evidence-based decisions. The WWC team recommends a five-step systematic review process that includes: (a) developing the review protocol, (b) identifying relevant literature, (c) screening studies, (d) reviewing articles, and (e) reporting findings. Alexander (2020) offers methodological guidance for framing a systematic review (e.g., situating the review theoretically, establishing what is already known and still needs to be known, articulating an answerable question, etc.), procedural steps and challenges (e.g., setting one’s search parameters, specifying search terms and time frames, appropriately delimiting the search, etc.), consolidating and summarizing challenges (e.g., recording the basic details, charting other appropriate characteristics, beginning to consider other ways of grouping the data that are potentially relevant, etc.), and interpreting and communicating findings (e.g., recognizing outcomes that are meaningful and relevant, finding ways to capture significant patterns and trends, clearly communicating the outcomes of the systematic review, etc.). All these components of the systematic review are intended to assist researchers to explore educational research more deeply, critically, rigorously, clearly, openly, and meaningfully.

Within the context of systematic reviews, a meta-analysis is “a set of statistical techniques for synthesizing the results of multiple studies. Such techniques are used when the guiding research question focuses on a quantitative summary of study results” (Pigott and Polanin 2020, p. 24). Pigott and Polanin (2020) recommend the following best practices while conducting a meta-analysis, (a) developing a research question for a meta-analysis; (b) searching all eligible studies; (c) unbiased screening of abstracts and full-text; (d) coding important moderators of effect size variability; and (e) computing and reporting all effect sizes. Some meta-analyses focus on estimating a treatment effect in a set of experimental studies, whereas others focus on estimating the magnitude and direction of an association between two variables. Though time-consuming, meta-analyses add value for future researchers, practitioners, and policymakers. Figure 1 shows a visual of reviews. As is clear from that figure, while all meta-analyses are systematic reviews, not all systematic reviews are meta-analyses.

Fig. 1

Visual depiction of research reviews

Systematic reviews and meta-analysis have benefits such as answering critical questions, formulating appropriate search parameters, and using explicit methods of data extraction and synthesis. They also employ sources that otherwise might be considered inconclusive and small. And they provide an opportunity to address topics of mixed-findings (Ioannidis and Lau 1999).

Some of the limitations of systematic review and meta-analyses include that they can be prone to bias due to analyzing only published data as well as the bias that results from screening and coding of those available studies. Another prominent limitation is that such reviews and analyses only provide information pertaining to what is already published and available, and, therefore, such analyses cannot improve the quality of primary research. Additionally, systematic reviews suffer from questions about their inherent validity. To begin addressing validity, researchers might attempt to include detailed inclusion and exclusion criteria, incorporate unpublished studies in addition to published studies, be transparent in screening and coding process using a protocol, and discuss the quality of studies included (Schlosser 2007; Thompson et al. 2012). Finally, researchers might seek to make comparisons that are appropriate and relevant so as not to compare apples and oranges.

Need for systematic reviews on emerging learning environments and technologies

There has been an increase in the use of learning technologies such as MOOCs, social media, virtual and augmented reality, collaborative technologies, open educational resources, synchronous online technologies, adaptive technologies, and mobile technologies (including wearables), to name a few (Bonk 2009, 2016). These technologies are referred to as emerging technologies, a term that indicates that their status and use in educational contexts is still fairly fluid (Veletsianos 2010). Early research in an area typically focuses on what Borko (2004) refers to as “existence proofs,” or one-off studies of individual implementations.

It takes time for a more systematic, mature body of research to emerge and for noticeable research gaps to start to be filled in. This special issue brings together a collection of systematic review articles, each focusing on a different aspect of emerging learning technologies. As is apparent, there is a need for a strategic approach to review research on the use of these emerging learning environments and technologies. In this special issue, seven primary focus areas of emerging learning environments and technologies are included: (1) social media, (2) massive open online courses, (3) special education technology, (4) mobile learning, (5) game-based learning and gamification, (6) adaptive learning, and (7) learning analytics. While there are several other emerging learning environments and technologies, only these topics were included as these were based on author submissions to this special issue. We summarize the reviews of research based on these key areas below.

Social media

While the origins of social media can be traced back to the late 1990s (Boyd and Ellison 2007), it took a decade until the popular social networking sites that we use today to emerge. According to a couple of recent Pew Internet surveys (Anderson and Jiang 2018; Perrin and Anderson 2019; Smith and Anderson 2018), these social networking sites included Facebook, Twitter, Instagram, Pinterest, Whatsapp, LinkedIn, and Snapchat, which serve a broad range of social and communication functions. Among educational researchers, the reception to social media has been mixed.

While these popular social networking sites have been hailed for their educational promise (Greenhow and Askari 2017, Greenhow et al. 2019a, b; Paskevicius et al. 2018), they have been critiqued for the potential discomfort (Dennen and Burner 2017) and disruption (Flanigan and Babchuk 2015) that they may cause in educational settings. In short, the value of taking these more generally used tools and coopting them for academic purposes is uncertain, although both formal and informal learning clearly have been facilitated through social media use (Greenhow and Lewin 2016).

Existing reviews of social media research in the education context explore an array of applications. First, there are reviews that focus on educators and leaders, along with how they use social media. Ranieri (2019) directly examined social media’s role in teacher professional development, finding that knowledge exchange and network development are key uses. In contrast, Macià and García (2016) took an indirect approach, conducting a review on professional development communities and networks which are frequently situated in or facilitated through social media platforms. Leonardi and Vaast’s (2017) review does not focus specifically on educators but touches on the affordances of the social media.

Next are reviews that focus on the student experience. In moving the field of social media forward, Greenhow and Askari (2017) assessed the state of social media research in education by using Roblyer’s (2005) framework of studies in educational technology. They found that while implementation-focused studies along with studies exploring uses of social media by teachers and students were common, the major gap is studies that connect social media to concrete measures of learning. This finding resonates with an earlier review study that noted enthusiasm for social media use, but insufficient empirical support for claims that such technology can be an effective learning tool (Tess 2013). Reflecting on these findings, Greenhow et al. (2019a, b) recommend research that focuses on practices, outcomes, and learning across contexts.

Finally, reviews of research on social media conducted outside of education are valuable to consider as well, given the broad-reaching impact of social media across an individual’s life. For example, Bolton et al. (2013) reviewed the research on the millennial generation’s use of social media and PrakashYadav and Rai (2017) reviewed the research on GenZ. Both consider how different generations may have unique approaches to social networking and how to use tools in different ways. In contrast, Leonardi and Vaast’s (2017) review did not focus specifically on educators or students but touches on the affordances of the social media. Most recently, Orben’s (2020) review targeted teenage digital media use and overall wellbeing. This particular study revealed that many review articles on social media aggregate and discuss all platforms together, which is a shortcoming because such an approach overlooks the nuanced differences among the tools.

As the research progresses and evolves over time, more generally focused review studies can help education researchers more thoroughly consider the role of tools or platforms in addition to purpose. To that end, reviews like Chugh and Ruhi (2018) explored Facebook research and Tang and Hew (2017) summarized Twitter research. The former found that Facebook brings both advantages such as increased engagement and disadvantages such as privacy concerns to the classroom, whereas the latter found that Twitter was most frequently used as a means of broadcasting messages to a class.

In this special issue, two articles provide systematic reviews of social media research from different angles. First, Dennen, Choi, and Word look broadly at research on social media, education, and teens, including studies from related fields. They find points of overlap between the research in and out of education, with little interdisciplinary work being conducted between education and other fields. This review highlights opportunities for future researchers to collaborate widely to understand how social media not only fits narrowly within the scope of academic learning, but more holistically across the school setting. Luo, Freeman, and Stefaniak take a different perspective, investigating studies of social media use for professional development among higher education faculty. Their findings show promise for this use of social media, but also pose various challenges to overcome before social media-based professional development becomes a widespread, successful practice among this population.

Massive open online courses (MOOCs)

People around the globe were aware of the dire situation in education brought about by the COVID-19 pandemic. What many failed to ascertain is that a form of online education that had been widely criticized during the past decade, massive open online courses (MOOCs), was finding increased traction during the COVID-19 outbreak. The MOOC enrollment numbers in the early months of the pandemic were staggering. An article in the New York Times in late May 2020 by Steven Lohr (2020), “Remember the MOOCs?: After Near-Death, They’re Booming,” noted that the top MOOC provider, Coursera, added more than 10 million learners from mid-March to mid-May 2020. A report from MOOC provider, Udemy, found a similar pattern and attributed the surge in MOOC enrollments to people remotely working and not traveling to events for training. As a result, countless millions were needing to learn online (Udemy 2020).

Perhaps such numbers should not be surprising given that online degrees and MOOCs were flourishing prior to the pandemic (Schroeder 2019b); it was becoming increasingly possible to earn academic degrees and certificates through MOOCs, including master’s degrees and other credentials (Hollands and Kazi 2019; Pickard 2019; Shah 2019a, b). As that was occurring, there was a distinctive shift in MOOC offerings toward adult learners in the workplace (Rayome 2017; Schaffhauser 2018). For instance, online programs were becoming modular (IBL News 2018) and stackable where learners could go from taking a few free online courses via a MOOC to enrolling in certificate programs that can later count toward a master’s degree (DeVaney and Rascoff 2019).

Although MOOCs are a relatively new phenomenon, many research studies have been carried out since inception in 2008 (Downes 2008). In addition, numerous systematic reviews of MOOC research have been conducted (e.g., Kennedy 2014; Liyanagunawardena et al. 2013). The MOOC research reviews to date span a range of issues such as MOOC success factors, quality, MOOC research in different countries, and MOOC pedagogical approaches. Some MOOC research reviews relate to MOOC scholarship (Ebben and Murphy 2014) such as methodological approaches employed and the publication outlets (Deng and Benckendorff 2017; Veletsianos and Sheperdson 2016; Zhu et al. 2018), whereas other reviews focus on general trends, topics, and themes (Bozkurt et al. 2017; Liyanagunawardena et al. 2013; Zhu et al. 2018). Still other reviews have investigated the geographic distribution of MOOC publications (Veletsianos and Sheperdson 2016; Zhu et al. 2018), most cited articles (Veletsianos and Sheperdson 2015), or MOOC researcher disciplinary affiliations (Veletsianos and Sheperdson 2015). A couple of recent MOOC research reviews have explored self-regulation (Lee et al. 2019) and MOOC engagement and dropouts (Joksimović et al. 2018).

Early MOOC advocates argued that this technology offered greater democratization of education. Recently, there have been concerns made that MOOCs have done little to help the disadvantaged or less educated. In response, there are now systematic reviews of MOOC research related to problems and issues in the Global South (King et al. 2018; Zhang et al. 2020), social mobility among the underprivileged (van de Oudeweetering and Agirdag 2018), and the accessibility of MOOCs (Sanchez-Gordon and Luján-Mora 2018).

Zhu, Sari, and Lee, in this issue, provide a comprehensive look at the extant MOOC research. In the present study, they significantly build on their earlier research of 146 empirical studies published between October 2014 to November 2016 (Zhu et al. 2018). In fact, of the many systematic literature reviews of MOOCs to date, this study is the most comprehensive thus far in terms of both the number of years of research in the analyses and the number of studies. Previously, the largest undertaking of MOOC research was by Bozkurt et al. (2017) who captured and analyzed 362 studies that met their criterion spanning 2008 to 2015. More impressively, Zhu et al. (this issue) included 541 empirical published journal articles from 2009 to 2019. As a result, this study is rich with data and figures, both current and historical.

Zhu and her team appropriately divide their data into two phases: Phase 1 from 2009 to 2016 and Phase II from 2017 to 2019. What distinguishes the two phases is that, after 2016, MOOCs increasingly became attached to a degree program or college credit or led to a certificate (Pickard 2019; Shah 2018, 2019b). In effect, starting in 2017, MOOCs became monetized. With this framework for dividing MOOC research, Zhu, Sari, and Lee provide a better sense of MOOC research trends, topics of interest and focus, data collection and analysis methods, and extent of collaboration within the MOOC research field.

Special education technology

Increasingly society is concerned with meeting the needs of diverse learners across educational settings and sectors. Over the past few decades, the field of special education has developed and researched technologies to support those with learning disabilities across the various content areas including reading, writing, math, and science (Hasselbring and Bausch 2017). As the field of learning technology expanded, naturally, there was increasing attention into learning with technology in the field of special education. This has resulted in systematic reviews focusing on studying the use of various learning technologies including online technologies with learners with cognitive disabilities (Cinquin et al. 2019), support tools for those with autism spectrum disorders (Aresti-Bartolome and Garcia-Zapirain 2014), and mobile technologies to assist individuals with developmental disabilities (Kagohara et al. 2013).

Like with other learning contexts, more common technologies are often repurposed in special education settings. This repurposing has become increasingly common with the widespread use and social acceptability of smartphones and tablets (Campigotto et al. 2013). Part of the motivation behind this repurposing of existing technologies and the use of mobile applications (app) to support learners in special education contexts comes from their affordability (McEwen 2014). It also is derived, in part, from the lessened risk associated with trying an app that often costs less than five dollars in comparison to the many dedicated cognitive support tools that have abandonment rates that exceed 35% (Dawe 2006). The most recent explorations into the uses of new technology for special needs children have considered how to support socio-collaborative learning approaches (Baykal et al. 2020) and the creation of robots who can support inclusive play (Metatla et al. 2020).

This exploration of the use of robots is unsurprising given the current general focus on artificial intelligence (AI) within society. While the use of AI to support learning dates back to the 1970s and 1980s, its adoption in schools has been less pronounced and its use for supporting special education contexts is less established. However, the types of AI-based approaches that are used to support learners in other contexts have been shown to differentially benefit students with disabilities (Woolf et al. 2010).

In a review of the research on special education in this special issue, Olakanmi, Olakanmi, Akcayir, Ishola, and Demmans Epp reviewed 126 publications from 2014 to 2018 focused on various cognitive and developmental disabilities; across these works, there were some 283 different disabilities represented across the studies in this review, making generalities especially difficult. Most addressed the K-12 student population and targeted gaming. The primary goal across the studies that they reviewed was enhancing cognitive outcomes. Most studies took place in academic disciplines such as the natural sciences, reading, and writing prompting the Olakanmi et al. to suggest that more attention should be paid to special education technology in work and social settings as well as life in general.

Mobile technologies

Mobile phones have found their way into the hands of the vast majority of adults in the United States (96%; Pew Research Center 2019) and around the world, though not equally across regions and countries (Taylor and Silver 2019). With one million new mobile learners being added each day, the exact number of worldwide mobile users is difficult to pin down; however, as of late January 2019, there were 5.11 billion unique mobile users out of 7.676 billion people on the planet or about 67% (Kemp 2019). This rapid growth has paved the way for exploring mobile learning as a meaningful way of supporting both formal and informal learning experiences. The existing research on mobile learning is extensive and broad, much like mobile phone ownership; nevertheless, mobile learning is not yet a widespread global phenomenon. A review of research conducted in Africa shows that issues like infrastructure, age of technology, pedagogical ability, and attitudes can make mobile learning prohibitive (Kaliisa and Picard 2017). Although sub-Saharan Africa is the region with the greatest disparity, the shift to remote learning during the COVID-19 pandemic demonstrated that inequality of access for mobile learning extends throughout the world community (United Nations 2020; Vogels et al. 2020).

Although mobile learning—which includes learning with tablets and wearables, in addition to phones—may not yet be a viable solution for all learners, prior reviews have shown that the research in this area has matured over time in terms of research design and focus (Krull and Duart 2017; Lai 2019). For PK-12 learners, most research has looked at mobile learning in science-related disciplines with elementary school learners using behaviorist approaches (Crompton et al. 2017). The mobile learning literature base in science has, in turn, been reviewed, with recommendations made for expanding research into areas such as mobile collaborative learning (Zydney and Warner 2016) and exploring the affordances of mobile learning in the science classroom; specifically, the ability to learn across spaces and contexts (Bano et al. 2018). In higher education settings, mobile learning research shifts to focus more heavily on language instruction and student achievement (Crompton and Burke 2018). Similar to Crompton et al.’s (2017) PK-12 review, Pimmer et al. (2016) found that behaviorist approaches are also common in higher education.

In this special issue, we have four reviews that address mobile learning, each extending and reinforcing earlier reviews. First, Eutsler, Mitchell, Stamm, and Kogut examine 13 years of research on mobile technology use with early learners to develop literacy skills. From the 61 studies that Eutsler et al. reviewed, they found that tablets were the most common devices used with young learners, with a diverse array of mobile learning implementation. Additionally, their results revealed mixed results in terms of effectiveness, especially where reading comprehension is concerned.

Also in this issue, Chen, Chen, Jia, and An focus on mobile-assisted language learning (MALL) research, conducting a meta-analysis across 11 years of research to examine effectiveness when compared to other language learning methods. Their findings support MALL as an effective approach to language learning, especially when it takes advantage of the situated and collaborative affordances of mobile devices. This study echoes the suggestions and pleas from earlier reviews of mobile learning (Crompton et al. 2017; Pimmer et al. 2016) that call for researchers to increase their focus on collaborative and constructivist uses of mobile technologies to support learning. It also demonstrates that the skills that collectively support language learning are not all equally taught via mobile learning, which is akin to the mixed results found by Eutsler et al.

The third review of mobile learning in this special issue, by Gao, Li, and Sun, examines game-based learning in the mobile context. By looking at 30 studies of STEM-oriented game-based learning, they were able to identify trends in this field. The majority of the studies that they examined were conducted with younger children, reinforcing Crompton et al.’s (2017) findings. They identified a major limitation in the existing studies; notably, the brevity of mobile game-based learning interventions. Like other reviews in this area, Gao et al. noted a dearth of studies focused on collaboration.

Whereas the other three reviews in this special issue focus on mobile learning through smartphones or tablets, the final mobile learning review investigates the research on wearable technology. Havard and Podsiad included 12 studies in their meta-analysis. They discussed the challenge of defining the scope of this area given the diverse nature of wearable technologies that are proliferating (e.g., smartwatches, head-mounted devices, fitness trackers, eyewear, wrist bands, and smart garments). Their findings showed generally positive results for learners who use wearable technologies, especially for cognitive outcomes. Although there were relatively few studies in this area, this area seems ripe for growth as wearable technologies show up in both formal and informal educational settings.

Collectively these reviews of mobile technology for learning show how mobile learning remains an emerging and fast-changing technology that has potential for broad use across populations, disciplines, and contexts. Although the research base is maturing, the technology that supports mobile learning continues to develop at an even faster pace. Similarly, the field’s understanding of when and how mobile learning applications might best be designed and implemented remains a topic of investigation (Krull and Duart 2017; Lai 2019).

Game-based learning and gamification

While game-based learning existed even in the 1970s (Coleman 1971), the development of the Internet and personal computing resulted in increased interest and instructional practices based on game-based learning. Importantly, de Freitas (2006) defines games for learning as “applications using the characteristics of video and computer games to create engaging and immersive learning experiences for delivering specified learning goals, outcomes and experiences” (p. 9). Using games for learning has several advantages including, to motivate and engage learners, for skill rehearsal and practice, for role-playing different jobs or professions, problem-solving, communication, collaboration, and critical thinking. Researchers have differentiated game-based learning from gamification, which is defined as the use of “use of game design elements in non-game contexts” (Deterding et al. 2011, p. 9).

Meta-analyses have been conducted in the last two decades to study the effectiveness of digital game-based learning. In an early meta-analysis on games and simulations, Vogel et al. (2006) found that using games and simulations resulted in higher cognitive outcomes and attitudinal outcomes compared to traditional face to face instruction. More recently, Clark et al. (2016) conducted a meta-analysis on digital games and learning for K-16 students and found among 57 studies that were included in that particular meta-analysis, digital games significantly enhanced student learning.

Lamb et al. (2018) conducted a meta-analysis on examining serious educational games, serious games, and simulations. They defined serious games as games “to train a broad series of tasks using real-life examples” (p. 158), and that serious educational games are similar to serious games but “incorporate specific a priori pedagogical approaches to not only train tasks but teach content as well” (p. 158). Among the 46 studies they examined, Lamb et al. (2018) found higher cognitive gains and positive affects toward learning when participants used serious educational games, serious games, and simulations. However, when comparing traditional instruction to instruction using serious educational games, serious games, and simulations, they did not find any statistically significant differences.

Some researchers in this field have focused on learning theory foundations of game-based learning (Wu et al. 2012). Others have examined digital game-based learning in a specific subject such as English as a foreign language (Kao 2014), mathematics (Byun and Joung 2018; Tokac et al. 2019), and vocabulary learning (Chen et al. 2018).

Included in this special issue is a meta-analysis by Chen, Shih, and Law, who analyzed 25 digital-game based learning (DGBL) studies. Their meta-analysis found that competition in DGBL studies had a significant positive effect on learning. Interestingly, they found that competition in DGBL had a significant positive effect in the domain subjects of language, math, and science, but not in the subject of social science or other subjects. Their study also discovered a significant effect on all educational levels—elementary, secondary, and college—though the effect of competition in college students was weaker than with K-12 students. In addition, their study revealed that competition in DGBL was effective in role-playing games, puzzles, simulations, and strategy types of games but not in action games. Individual and peer games had both significant effects, and overall the findings were significant for both cognitive and non-cognitive outcomes. Their findings add to the research synthesis on DGBL to inform how DGBL elements have an effect on student learning.

In addition to game-based learning, there has also been an increased interest in gamification. Alomari et al. (2019) systematically reviewed 40 studies to examine the relationship between gamification techniques (e.g., points, badges, leaderboards, levels, rewards, progress bar, challenges, feedback, and avatars) and student learning. From their review, points and badges were the most commonly used gamification techniques. One of their recommendations was to inform the students about the gamification techniques before they participate in gamified learning.

In another review, Antonaci et al. (2019) studied the effects of gamification in MOOC-based online learning environments. Their review included 61 studies focusing on six areas, performance, motivation, engagement, attitude towards gamification, collaboration, and social awareness. They categorized 24 game elements and described the effects of the various game elements in the different focus areas. Badges, leaderboards, and points were the most used game elements in the studies that they reviewed.

In a systematic mapping study, Dicheva et al. (2015) examined gamification in education. Among the 34 studies that Dicheva et al. reviewed, they discovered that visible status and social engagement were the gamification design principles that were highlighted in most studies. Similar to the Antonaci study, badges, leaderboard, and points were used in most studies. They also found that most gamification occurred in blended learning courses.

In an investigation into gamification in higher education, Subhas and Cudney (2018) conducted a review of 19 studies focusing on gamification elements. In terms of discipline, the largest percent of studies were conducted in the computing discipline, whereas in terms of location, the largest percentage of studies occurred in Spain. Their findings were consistent with previous studies that badges, leaderboards, and points were the game elements that were most studied. Subhas and Cudney (2018) also explored student performance, engagement, attitudes, and motivation.

A few meta-analyses have also been conducted on gamification. Sailer and Homner (2019) conducted a meta-analysis to study the effects of gamification on cognitive, motivational, and behavioral outcomes. They discovered a significant but small effect of gamification for the three outcomes. That same year, Baptista and Oliveria (2019) conducted a meta-analysis to take a look at gamification and serious games in which they analyzed 54 studies and found that attitude, enjoyment, and usefulness commonly predict intent to using gamification.

The present special issue of ETR&D includes a meta-analysis by Huang et al. focusing on gamification. Their meta-analysis included 30 independent studies where they compared gamification to non-gamification conditions on student learning outcomes. Huang et al. discovered that the gamification condition had a small to medium effect in comparison to the non-gamification condition. This finding is similar to the meta-analysis findings from Sailer and Homner (2019) noted above. Huang et al. also examined a number of gamification elements. Ironically, they discovered that studies not using leaderboards had a higher statistically significant effect than the ones using it. They concluded that while using leaderboards is considered a form of motivation, it also might have adverse influences on students progressing towards their educational outcomes. The other gamification elements such as badges/awards, points/experience, and advancement/levels had a similar effect and resulted in increased effect size with or without those features. However, responsive feedback, collaboration, use of quests/missions, or modules were found to have a larger effect size in their meta-analysis. While the authors recommend the use of gamification in well-designed instructional content, they caution that it cannot replace poor instructional design and inadequate learning materials.

Overall, the studies on DGBL and gamification show that DGBL and gamification elements have effects on student educational outcomes. At the same time, these studies also reveal that it is critical for the instructional content to be well-designed.

Adaptive learning and technologies

Adaptive Learning is defined as a process that “dynamically adjusts the way the instructional content is presented to students based on their comprehension of the material as revealed in their responses to embedded assessments or learner preferences such as visual presentation of materials” (Lowendahl et al. 2016, p. 7). In effect, it provides a learning experience that adjusts for individual differences. As such, it is linked to popular trends to personalize or customize learning (Schroeder 2019a).

There have been several systematic reviews on adaptive learning since 2008. One of the earliest studies was by Verdú et al. (2008). Verdú et al. (2008) conducted a meta-analysis of adaptive learning research to address the question, can students improve their knowledge when the system adapts to their profile and performance? Though they did not calculate an overall effect size for the 15 studies they analyzed, they tabulated individual effect sizes which were mostly positive in terms of improvement in academic achievement when comparing adaptive learning with a control group.

Three years later, Vandewaetere et al. (2011) examined learner characteristics in computer-based adaptive learning environments. Vandewaetere et al. (2011) concentrated their efforts on how instruction will be adapted. They examined both theoretical and empirical studies and categorized 42 studies based on the source of adaptation focusing on learner and learner-environment interaction. Through the lens of what is adapted, they analyzed 29 studies and categorized the source of adaptation as content, presentation, and instruction. Finally, in their review, they also identified pathways of adaptation and categorized 25 studies as rule-based, probability-based, or other adaptive pathways.

The following year, Akbulut and Cardak (2012) conducted a content analysis with 70 studies that addressed learning styles in Web-based adaptive educational hypermedia (AEH) systems. The findings of their review indicated that the learning style-based AEH system had positive impacts on students’ performance, learning process, and satisfaction levels. However, most of the studies emphasized the adaptive framework or model of AEH as a system; scant studies evaluated the effectiveness of AEH on learning outcomes. Akbulut and Cardak suggested that future studies should focus on the adaptivity of the learning system as well as the adaptability from the learners’ perspective.

To determine what type of characteristics should be considered for a learner model, in 2015, Nakic et al. (2015) used an evidence-based approach and reviewed 98 journal papers and conference proceedings that specifically focused on the evaluated adaptive learning systems. Nakic et al. identified 17 individual differences as sources of adaptation for adaptive learning, including age, gender, processing speed, cognitive abilities, personality, experiences, learning styles, motivations, and preferences. The findings of this study contribute to the improvement of user interaction and learning performance in an adaptive learning environment.

Next, Kumar et al. (2017) conducted a document analysis with 78 studies and reviews focused on learning style in adaptive intelligent tutoring systems (AITS). The findings of this study further confirmed that most of studies addressed a framework or architecture of AITS (e.g., Felder-Silverman learning style model). Studies also centered on the learning outcome or learner satisfaction of using AITS. Applications of learning styles in AITS covered the learning materials, learner characteristics, learner knowledge, learning media content and resource format, recommended tutoring materials or pedagogy, intelligent games, learner evaluation and practice, or other mixed characteristics. Rule-based and Bayesian network algorithms were the two most adopted learning style classification techniques. Kumar et al. suggested that future studies should focus on the implementation of mixed learning style models in AITS as well as the application of algorithms to detect these mixed learning styles to improve the adaptivity of the system.

In the most recent review, Normadhi et al. (2019) reviewed 78 studies to explore the most commonly used learners’ personal traits (e.g., learning styles, cognitive styles, knowledge level, etc.), dimensions of personal traits (e.g., visual, aural, and kinesthetic), identification techniques (e.g., questionnaire, computer-based detection, etc.) for these attributes in an adaptive learning environment. The findings revealed four categories of personal traits: cognitive, affective, behavioral, or psychomotor which were appropriately mixed with the six levels of remembering, understanding, applying, analyzing, evaluating, and creating. The personal trait identification techniques included computer-based detection using algorithms or programs (such as machine learning), traditional questionnaires, and both. Normadhi et al. also pointed out emerging issues related to the identification of personal traits in adaptive learning environments. Such traits include as the lack of validation of the obtained data, the concern of learning motivation and cognitive load, and the limitations of existing adaptive technology in this regard. Normadhi et al. suggested that future studies should focus on the empirical evaluation related to the effectiveness of the personal traits and its identification techniques as wells as the strengths and weaknesses of personal traits being integrated into the learning object and materials in adaptive learning environments.

Several of the systematic reviews of adaptive learning above concentrated on the learner characteristics aspect of adaptive learning and not on the instructional aspects. This special issue includes a study by Martin, Chen, Moore, and Westine who conducted a systematic review on adaptive learning focusing on both a learner model and an instructional model that builds on the previous review studies described above. They analyzed 61 empirical studies from the years 2009 to 2018 and studied adaptive strategies and technologies. The authors examined adaptive sources based on learner models and adaptive targets based on content and instructional models. Learning style was the most observed learner characteristic, while adaptive feedback and adaptive navigation were the most investigated adaptive targets. Though there are several meta-analyses focusing on intelligent tutoring systems (ITS), there is a need for meta-analysis in the area of adaptive learning which is considered as the next generation of ITS (Essa 2016).

Also included in this special issue a bibliometric analysis and a meta-analysis review by Liu, Moon, Kim, and Dai who reviewed educational games with adaptivity. They analyzed 62 studies for the qualitative thematic analysis and a subset of 12 studies that used experimental designs for the meta-analysis to study the effects of adaptivity in educational games. Their meta-analysis found that adaptive learning condition did not result in a substantial overall effect compared to a non-adaptive condition. However, the target effect size was positive when focusing on learning but not for game performance.

Learning analytics

The Society for Learning Analytics for Research (SOLAR) defines learning analytics as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (2011, p.1). More recently, Ifenthaler (2015) elaborates and defines learning analytics as “the use, assessment, elicitation and analysis of static and dynamic information about learners and learning environments, for the near real-time modelling, prediction and optimization of learning processes, and learning environments, as well as for educational decision-making” (p. 447).

Identified as an emerging trend due to the availability of data, a number of research studies have focused on learning analytics. As a result, systematic reviews have been conducted using primary research studies on learning analytics. We found at least seven systematic reviews on learning analytics. One of the earliest systematic reviews was by Papamitsiou and Economides (2014) who reviewed learning analytics and educational data mining from 40 empirical studies. Their focus was to identify research objectives and methods researchers use to achieve these objectives. Two years later, Avella et al. (2016) reviewed 44 studies to study learning analytics methods, benefits, and challenges in higher education.

More recently, Mangaroska and Giannakos (2019) conducted a systematic review on learning analytics for learning design by examining 43 articles. They reviewed these studies for their research design, topic of study, educational context, learning scenario, pedagogical practices, learning platform, technology tools, and methodological techniques. Around that same time, Vieria et al. (2018) reviewed 52 papers in the field of learning analytics by systematically focusing on visual learning analytics of educational data to identify approaches, audiences, purposes, contexts, and data sources. They also examined how the existing literature integrates educational theories and visualization principles into visual learning analytics strategies.

In addition, there have been a couple of important reviews specifically focusing on learning analytics dashboards. For instance, Bodily et al. (2018) systematically reviewed learning analytics dashboards and open learner models (OLM) attempting to bridge these two areas. They sought to study the types of data collected in OLM systems and different modeling methods used, research themes, the nature of OLSM system evaluations, and the similarities and differences between OLM and learning analytics dashboards. Matcha et al. (2020) also reviewed learning analytics dashboards from a self-regulated learning perspective. Their focus was to review the support of learning analytics dashboards in self-regulated learning, the types of information that was offered as feedback, and the quality of study designs that reports empirical evaluations.

Another recent systematic review of 11 publications related to learning analytics focused on the efficacy of learning analytics interventions in higher education in terms of student retention and/or academic success (Larrabee Sønderlund et al. 2019). Most of the studies they examined showed a post-intervention increase in grades or activities. Building on this particular systematic review, is the final systematic review in this special issue from Ifenthaler and Yau. These researchers inspected 46 empirical articles to address whether and to what extent learning analytics is successful in facilitating study success in higher education. Importantly, they conceptualized positive factors of study success as course completion and student retention, whereas they identified negative factors as student-at risk of dropping out, loss of academic success, attrition, and other more abstract factors. They categorized the factors contributing to study success broadly as predictors and visualization. Predictive algorithms and visualizations were used in a number of studies that they reviewed. In their review, Ifenthaler and Yau confirmed that a combination of learners’ background information, behavior data from digital platforms, formative and summative assessment data, and information collected through surveys could be applied in the field of learning analytics.

Conclusion

We hope that this introduction has excited potential readers about the contents of this issue. Our goal was to bring together systematic reviews in one issue, so that together the reviews could show both breadth and depth in the range of research being conducted on emerging learning environments and technologies. As Reeves and Lin point out in the closing article of this special issue, systematic reviews have a relatively recent history and have become increasingly popular with the explosion of research reports related to educational and instructional technology since the dawn of this new millennium. With waves of technological progress occurring during the past few decades, as detailed by Bonk and Wiley in the preface to this issue, such reviews of research are increasingly necessary.

It is nearly impossible to stay abreast of research and development on any learning technology trend, let alone the assorted dozens of such trends at this time (Bonk 2016, 2020). Fortunately, the 44 authors who contributed to the systematic review articles found in this special issue have spent painstaking months searching, mining, filtering, accumulating, analyzing, cataloging, documenting, and summarizing what they found. They should be commended for their efforts.

Perhaps you are already sensing a potential role that you can play in one or more emerging learning technologies in the form of a research gap that is apparent or a possible project that you might commence with a research partner. Or perhaps you have simply found someone to add to your network. It is conceivable that one or more research projects will spring to life from reading and discussing the findings of the next 13 articles of this issue.

As Bonk and Wiley highlight at the end of the preface and Reeves and Lin caution several times in the conclusion of this particular journal issue, it is vital to study complex and pervasive educational problems instead of the latest technology system or device. Accordingly, we intend for this issue to be valuable to graduate students and those new to the field of instructional design and educational technology who simply want to know the state of the research in areas that are attracting much professional interest. At the same time, instructors might find timely research findings to add to a graduate class or to reference in a technical report or book chapter. Administrators might find data for an upcoming strategic plan or campus technology committee.

As is stated in the preface, this special issue only serves up a portion of the emerging learning technologies research scene in 2020. In effect, this issue offers but a glimpse of the state of research in field; albeit, we hope, an eye-opening glimpse.

To be fair, technically speaking, emerging learning technologies is not a field; it is a reflection of conditions in society and life itself. As such, there is immediate intrigue in researching aspects of it. Where else is one able to spend a morning reading about the use of social media in school-based learning, the afternoon focusing on wearable technology, and the evening learning about the use of mobile technology to learn a language? Importantly, the articles you are about to read contain key insights into what we know in terms of the existing research in these fields and more, so as to help form policy guidelines and other practical implications. These articles will also tell us what we do not know, thereby perhaps provoking much contemplation and a few trial studies.

Enjoy your journey through this volume. It is only through systematically reviewing and synthesizing the literature on emerging technologies for learning that we can truly make informed decisions as practitioners and know what needs to be seriously studied next as researchers. Whatever you do, whether researcher, student, educator, trainer, policymaker, instructional designer, educational technologist, technology vendor, consultant, or parent, find your niche within the wide spectrum of learning environments and technologies that permeate each our lives. Think about what problems or issues you are addressing and potentially solving. For only then will the learners of the world have a chance to benefit in truly transformative ways.

Articles in the systematic review special issue

Bonk, C.J., & Wiley, D. A. (2020). Preface: Reflections on the waves of emerging learning technologies. Educational Technology Research and Development.

Martin, F., Dennen, V. P., & Bonk, C. J. (2020). A synthesis of systematic review research on emerging learning environments and technologies. Educational Technology Research and Development.

Dennen, V., Choi, H., & Word. (2020). Social media and the school context: A scoping review of research in education and related fields. Educational Technology Research and Development.

Luo, T., Freeman, C., & Stefaniak, J. (2020). “Like, comment, and share”: Professional development through social media in higher education: A systematic review. Educational Technology Research and Development.

Zhu, M., Sari, A.R., & Lee, M. M. (2020). A comprehensive systematic review of MOOC research: Research techniques, topics, and trends from 2009 to 2019. Educational Technology Research and Development.

Oluvabunmi, O., Akcayir, G., Oluvabukola, M.I., & Demmens Epp, C. (2020). Using technology in special education: Current practices and trends. Educational Technology Research and Development.

Eutsler, L., Chrystine, M., Stamm, B., & Kogut, A. (2020). The impact of mobile technologies on preschool and elementary children's literacy achievement: A systematic review spanning 2007–2019. Educational Technology Research and Development.

Chen, Z., Chen, W., Jia, J., & An, H. (2020). The effects of using mobile devices on language learning: A meta-analysis. Educational Technology Research and Development.

Gao, F., Li, L., & Sun, Y.Y. (2020). A systematic review of mobile game-based learning in STEM education. Educational Technology Research and Development.

Havard, B., & Podsiad, M. (2020). A meta-analysis of wearables research in educational Settings published 2016–2019. Educational Technology Research and Development.

Chen, C-H., Shih, C.-C., & Law, V. (2020). The effects of competition in digital game-based learning (DGBL): A meta-analysis. Educational Technology Research and Development.

Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., & Li, J. (2020). The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development.

Martin, F., Chen, Y., Moore, R.L., & Westine, C. D. (2020). Systematic review of adaptive learning research designs, context, strategies, and technologies from 2009 to 2018. Educational Technology Research and Development.

Liu, Z., Moon, J., Kim, B., & Dai, C.-P. (2020). Integrating adaptivity integrating adaptivity in educational games: A combined bibliometric analysis and meta-analysis review. Educational Technology Research and Development.

Ifenthaler, D., & Yau, J.Y-K. (2020). Utilising learning analytics for study success in higher education: A systematic review. Educational Technology Research and Development.

Reeves, T., & Lin, L. (2020). The research we have is not the research we need. Educational Technology Research and Development.