Identification and evaluation of technology trends in K-12 education from 2011 to 2021.

Educational technologies have captured the attention of researchers, policy makers, and parents. Each year, considerable effort and money are invested into new technologies, hoping to find the next effective learning tool. However, technology changes rapidly and little attention is paid to the changes after they occur. This paper provides an overall picture of the changing trends in educational technology by analyzing the Horizon Reports' predictions of the most influential educational technologies from 2011 to 2021, identifying larger trends across these yearly predictions, and by using bibliometric analysis to evaluate the accuracy of the identified trends. The results suggest that mobile and analytics technologies trended consistently across the period, there was a trend towards maker technologies and games in the early part of the decade, and emerging technologies (e.g., VR, AI) are predicted to trend in the future. Overall, the specific technologies focused on by the HRs' predictions and by educational researchers' publications seem to coincide with the availability of consumer grade technologies, suggesting that the marketplace and technology industry is driving trends (cf., pedagogy or theory).

Introduction

Due to the perception that new technologies can facilitate and improve learning, there has been a longstanding societal push from policy makers and parents to adopt technology into education (Artym et al., 2016; Nevski & Siibak, 2016; Reiser & Ely, 1997; Skinner, 1954, 1958, 1968; Vanderlinde et al., 2010). Based on the hope that technology will improve teaching and learning, schools are investing in information and communication technology (Machin et al., 2006); teachers are implementing technology into classrooms (Hutchison & Woodward, 2014); and parents are ensuring students have internet access at home (83.9% in Canada, 73.4% in the US, OECD, 2018). However, the specific technology that schools, teachers, and parents are expected to adopt changes rapidly and the inclusion of new technologies can change how learning occurs in classrooms. Therefore, it is critical to understand the changing trends in educational technology and how these changes affect the role of technology in classrooms.

Technologies have many affordances in education. The interactivity of Web 2.0 was supposed to enhance student’s comprehension and interest of online information (Karvounidis et al., 2018); social networking may develop writing and collaboration skills (Voivonta, & Avraamidou, 2018); mobile devices enable anytime, anyplace learning; augmented reality increases student’s learning attitudes and learning efficiency (Teng et al., 2018); and digital games increase engagement and hence improve academic achievement (Kiili et al., 2014; Outhwaite et al., 2017). From these few examples, it is clear that the mass adoption of any one technology could shift the focus in a classroom. Adopting social networking into education would emphasize collaboration as being central to learning in today’s classrooms. In contrast, adopting digital games may place more of an emphasis on whether or not students are engaged as a means to increase their understanding. These different foci could affect how teachers evaluate the effectiveness of their instruction and the types of activities they have students complete (e.g., online group work vs individual game progression). Thus, a better understanding of previous and current trends in educational technology use helps paint a picture of the present and future classroom.

Literature review

Technology trend reports

A few reports provide technological or pedagogical predictions that could be used to paint the aforementioned picture. These include Innovative Pedagogy (produced by Institution of Educational Technology), the Institute for Prospective Technological Studies Reports (made by Joint Research Centre of the European Commission), and the Horizon Reports (produced by New Media Consortium). Innovative Pedagogy is a series of reports starting from 2012 that cover some technology uses in education (e.g. learning with robots) but mainly focus on new forms of pedagogy (e.g., such as learning through wonder, student-led analytics, and intergroup empathy). The Institute for Prospective Technological Studies Reports also contain technology predictions, but the report’s main goal is to facilitate policy making from an economic perspective rather than an educational perspective. In contrast, the Horizon Report Project predicts and explores technology developments that may potentially impact education. The Horizon Reports (HRs) have continuously been issued since 2002 and have typically reached more than 500,000 downloads per year across 195 countries. Thus, the HRs are a unique source of information on technology trends in classrooms.

More specifically, the HRs are a global ongoing research report exploring technology trends and developments that are likely to have an impact on formal education. Each year, an advisory board consisting of a broad spectrum of experts in education, technology, and other fields engage in a comprehensive review and analysis of educational technologies based on current research and educational practice. The board finalizes six technologies they believe will influence K-12 teaching and learning across three periods: near-term (the year of the report), mid-term (2–3 years), and far-term (4–5 years). The HRs are argued to provide a link between current societal interest, research, educational practice, and future educational community’s mainstream technology practice. It should be noted that the potential technologies chosen in each report are selected based on the Delphi Technique. The Delphi technique is a deliberation method which involves collaborative decision making among advisory board members who ultimately come up with the final six high ranking technologies each year (Harold et al., 2011). Importantly, the selection of technologies in the HRs were due to their supposed popularity in research and educational practice rather than specific evidence-based benefits on teaching and learning. The goal is to identify technologies that are likely to influence education not necessarily to identify technologies that are the best learning tools, as determined by research or theory.

As mentioned earlier, technology changes rapidly and researchers, policy makers, educators, and parents should be aware of greater trends. Proper awareness allows policy makes and the public to spend their educational capital more wisely by avoiding the adoption of devices that are unlikely to persist (e.g., Mobile VR; Robertson, 2019). Awareness also allows researchers to better understand those technologies that are likely to impact a broad range of classrooms rather than those adopted by the few techno-enthusiasts. There is some previous work that has tried to identify technology trends in education, but the predictions in these works are now outdated and were never validated. Ely did a content analysis of journals, dissertations, conferences, and documents from ERIC and other sources and highlighted eight technology trends from 1988 to 1995, which included televisions, desktop computing, and early networking (1996). In a more recent report, Bonk (2009) stated that web-based technology was changing education by generating new forms of learning and listed ten trends: e-books, blended e-learning, open sources, learning objects, e-collaboration, mobile learning, and personalized learning. Importantly, these studies only described technologies that the authors thought were most likely to be adopted in the near future but there was no examination of whether the predictions came to pass.

Martin et al.’s (2011) work is the first and the only study to identify trends in educational technology in K-12 education and also evaluate the accuracy of the predicted trends. Marin et al. (2011) provided an evaluation of the most important technology trends in K-12 education across 2004 to 2010 by comparing the technology adoption rates predicted by the Horizon Reports with published articles in Google Scholar using bibliometric analysis. Specifically, they collated the six technologies predicted by each yearly report, clustered these individual predictions into larger trends, and then looked at whether the trends were supported by an increased level of scholarly discourse (i.e., publications). They originally conducted a market survey to see if technology purchasing rates correlated with the predictions but found that the buying power of the education sector was insufficient to affect greater societal buying trends. Alternatively, Martin et al. (2018) attempted to use Relative Search Volume (i.e., the number of times a term is searched in google) as a metric of societal impact in addition to bibliometric analysis but found this approach to be uninformative when they narrowed down the search to education related results. Outside Martin’s work, several studies have argued in support of the use of bibliometrics analysis as a means to evaluate the effect of emerging technologies (Daim et al., 2006; Han & Shin, 2014; Huang et al., 2014; Morom et al., 2018; Stelzer et al., 2015; Yeo et al., 2015). While bibliometrics does not directly reflect the use of technologies in society, it does provide insights into which technologies researchers believe are affecting society and the analysis can help guide future studies using more direct measures.

The goal was to see if the predicted technologies actually influenced education. The study concluded that the social web and mobile devices held the most influence on education and predicted that video games would have a bigger impact after 2010. Since Martin et al. (2011) original work, they used a similar methodology and evaluated the technology trends in higher education from 2010 to 2015 (Martin et al., 2018). No recent study has identified and evaluated the educational technology trends in K-12 after 2010 using the HRs. Further, Marten et al.’s work somewhat ignored an assumption underlying the HRs about the connection between society and education, that broader societal trends in research and practice determine the educational community’s mainstream technology usage. To address this gap, we use the same methodology to provide an updated overview of educational technology trends in K-12 education from 2011 to 2021 by collating the yearly predictions from the 2011 to 2017 HRs, identifying larger trends across these yearly predictions, and using bibliometric analysis to evaluate the accuracy of the identified trends.

The following research questions guided the work:

Research questions

What are the educational technology trends across 2011 to 2021, as predicted by the HRs?

Are the HRs’ predictions supported by a bibliometric analysis of published educational technology articles from 2011 to 2018?

Method

Data sources

Martin et al.’s (2011) work suggests that the HRs can be used as a basis for analyzing the influence of technology on education. Therefore, we chose the Horizon Reports on primary and secondary education from 2011 to 2017 to be used as the sole source of technology predictions for this analysis. To test the accuracy of Horizon Report’s prediction, Google Scholar was chosen as the bibliometric database due to it providing metadata of scholarly literature across disciplines and it connecting repositories of articles stored worldwide. Further, Google Scholar is considered to provide a broader coverage of publications as compared to Scopus and Web of Science (Bergman, 2012; Harzing, 2010).

Procedures

This paper adopted Martin et al.’s (2011) methodology but with the latest HRs from 2011 to 2017. The methodology involved the following stages:

Seven Horizon Reports were gathered from 2011 to 2017 and the six technologies predicted in each report were recorded according to their time frame (near, mid, far).

Based on the records, a visual representation of the HRs’ predictions was made. These visualisations use different colors to differentiate the technologies from each report and provide a clear picture of all the technologies predicted across the seven reports.

Similar technologies across all the reports were grouped into clusters and visual representations (same method as mentioned in step 2) were created for each cluster. These clusters are used in the subsequent bibliometric analysis.

Using the newly created clusters, the evolution of educational technologies across 2011–2017 were analyzed and discussed for each group.

The fifth stage involved the bibliometric analysis. Bibliometric analysis was used since the number of publications on a given educational technology are an index of its importance (Morries, 2002; Norton, 2001). The search process in Google Scholar involved the following steps.

Keyword selection. Technology related keywords were generated based on the clusters identified in stage 3. The technology specific keywords were derived from the technologies predicted by the HRs. Taking ‘mobile technology’ for example, the technologies identified in this cluster by the HRs were mobile, tablet, App, Bring Your Own Devices, and wearable technology. These specific technologies and their derivatives were entered as keywords in sequential searches (e.g., the technology ‘App’ included searches using the keywords Application, App, Apps) along with keywords representing schools (e.g., classroom, school). To limit redundancy, keywords used in a previous search were entered as an exclusion criteria in subsequent searches (e.g., search 1 = ‘App’, search 2 = ‘Applications’, -‘App’). Thus, the keywords used in the search were based on the specific technologies mentioned by the reports.

Year of publication. The number of publications for each keyword in each year from 2011 to 2018 was obtained, as well as the total number of occurrences across all years.

Title search. To limit the search to education publications, the words “learning” or “education” have to appear in the title together with the cluster keywords.

Result confirmation. Each individual search was conducted three times, across separate days and computers, to ensure that Google Scholar was returning consistent metrics.

Result weighting. The total number of education related publications available in Google Scholar changes every year. To account for this, a weighting factor (WF) for each year was applied. The WF was calculated using the number of papers published each year divided by the mean of papers published in education from 2010 to 2018. The equation for the WF is shown below.

the mean of papers published in education from 2010 to 2018,  = the number of papers published in year i, i = the year from 2010 to 2018, , N = total number of years.

To assess the accuracy of the HRs predictions, the trends predicted by HRs were compared to their weighted impact from step 5-d. Note* technologies predicted to trend in a given year would likely have a delayed impact on publications (i.e., predicted 2011, publications increase 2012).

Results

Visual representation of predicted technologies

A visual representation of the technology predictions from step 1 and 2 can been seen in Fig. 1. The vertical line depicts the year of prediction (year the HR was released), and the horizontal line depicts the year(s) in which technologies were predicted to have an impact.

Fig. 1

Technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

Technology clusters and trends

Following the approach by Martin et al. (2011), a visual analysis of Fig. 1 and a thematic analysis of the specific predictions made in the HRs were used to identify seven clusters of technology predictions and named them according to their common theme: mobile technology, maker technology, analytics technology, games, simulation technology, artificial intelligence (AI), and other technologies. The thematic analysis involved reading the HRs explanation for each prediction (e.g., mobile, Apps, BYOD) and identifying commonalities based on the type of technology involved (i.e., portable, personal computing devices). The themed clusters accounted for 30 of the total 42 predictions (71%) made by the HRs from 2011 to 2017. The ‘other’ cluster contains predictions that did not clearly cluster around a specific technology (e.g., cloud computing, open content, internet of things, natural user interface, digital badges, online learning, personal learning environments). In the following section, the seven clusters will be expanded upon by identifying how these technologies are proposed to affect education according to both the HRs (i.e., prediction justifications) and recent reviews of educational technology research. Understanding why each prediction was made by the HRs will also aid in the later evaluation of their prediction accuracy.

Mobile technology

This cluster (see Fig. 2) included every technology and practice related to mobile learning technologies in the HRs’ predictions, such as mobile devices, Apps, tablet computing, bring your own device (BYOD), and wearable technology.

Fig. 2

Mobile technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

The 2011 HR forecasted the importance of mobile on teaching and learning signifying a shift in how students and educators connect to the internet, from computers to mobile devices. Especially when tablets began to join the family of mobile technology, enabling the immediate and easy access to thousands of Apps all at once. The seamless access to the third-party applications is proposed to open the door to multiple resources for education (McEwen & Dubé, 2017). The 2012 HR also predicted mobile devices and Apps to be influential on education since mobile devices were reported to be one of the most common ways for youth to access educational software (Hirsh-Pasek et al., 2015). Meanwhile, teachers started to use Apps in their classrooms as supplementary tools to engage students with complex learning content (e.g., Zhang et al., 2015). In the same year’s report, tablets were separately predicted and emphasized to have impact on education, due to their larger screens and a richer range of gestures that may provide a more hands-on learning experience (Dubé & McEwen, 2015, 2017). The 2013 HR gave attention to mobile learning again, as it was widely adopted in school’s one-to-one learning initiatives and educational Apps became the second most downloaded category in the Apple App Store (Shuler, 2012). Similarly, the 2014 and 2015 HRs forecasted a new form of mobile learning—Bring Your Own Device (BYOD). BYOD is argued to facilitate student-centered learning and provided a more seamless learning experience between learning at home with the device and learning in the classroom with the same tool (e.g., McLean, 2016).

The 2014, 2015 and 2016 HR all predicted that wearable technology (e.g., smart watches, fitness bands) would be increasingly adopted in daily-life and education. However, the application of wearable technology to education was still emerging and was predicted to produce an impact on learning in the far-term. During this time, researchers were similarly predicting that wearables would be increasingly adopted into education, with a focus on their use as collaborative fieldwork tools in STEM subjects (e.g., taking pictures with Google Glass while collecting field samples in a biology course, see Sapargalivev, 2015 for a review of uses).

To sum up, in the HRs, mobile technology was continuously predicted to have an impact on learning. From 2011 to 2015, mobiles, tablets, Apps, and BYOD were predicted to have impact in near term (one year or less). From year 2014 to 2016, wearable technology was forecasted to have a far-term impact (4 to 5 years). All six reports (2011–2016) emphasized the importance of mobile technology from 2011 to 2021 and predicted a change in focus from mobile, Apps, tablets, and BYOD to wearable technology. The shift in focus may reflect the waning impact of currently pervasive mobile devices and tablets and the increasing impact of emerging wearable technologies, which were relatively new at a societal level.

Maker technology

The Maker movement and associated technologies (see Fig. 3) aim to promote authentic learning through hands-on design and construction (Loy, 2014). The specific maker technologies predicted by the HRs consist of 3D printing, robotics, and makerspaces.

Fig. 3

Maker technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

3D printing was forecasted to be influential on teaching and learning in both the 2013 and 2015 HRs; Due to the high cost and teacher training needed for inclusion in classrooms, 3D printing was only predicted to have an impact on education in the mid- and far-term. These critiques of 3D printing in education are echoed by both researchers and educators (e.g., Eisenberg, 2013; Turner et al., 2017), and have since been somewhat mitigated by the development of free, child-friendly 3D printing software and tutorials (e.g., tinkercad.com).

The robotics industry witnessed a significant growth in this decade (Ford, 2015; Ross, 2016) and were predicted in both the 2016 and 2017’s HRs to have an impact on education in the mid-to-far-term. Robotics were generally predicted to have a positive influence on the development of children’s twenty-first century skills. Contemporaneous reviews of research on robotics in education (Toh et al., 2016) suggests that students building and reasoning about robotics is said to contribute to problem-solving, collaboration, overall school achievement, STEM skills, and language ability (due to coding), and produce more participation from both students and parents in school activities through after-school workshops.

Makerspaces are a created workshop environment for learners to collective practice hand-on construction with technologies and to share resources and knowledge (Fourie & Meyer, 2015). Makerspaces appear in the 2015, 2016, and 2017 HRs all with predictions of near-term impacts, as makerspaces gained considerable attention worldwide. The makerspace movement can be considered as central to both the 3D printing and robotics movement, but is sometimes deemed technology agnostic (i.e., can build with Legos or robots). The original movement was focused on sharing knowledge and resources in a joint workspace whereas the later educational makerspace movement aims to promote learning through building (e.g., constructionism, Papert, 1980) and to promote the 4Cs of twenty-first century skills (i.e., critical thinking, collaboration, creativity, and communication, Fourie & Meyer, 2015).

In total, four HRs emphasized the impacts of maker technology on learning from 2015 to 2018. The predicted impact of maker technology gradually moved from long-term predictions, to mid-term, and then near-term, which suggests an adoption of this technology into education across the period. Of note, this pattern of prediction coincides with the increased availability and quality of consumer-grade maker technologies across this period (Li et al., 2017).

Analytics technology

Analytics technology or learning analytics (see Fig. 4) uses individualized data to provide adaptive instruction and assessment tailored to each student’s needs (Yu & Jo, 2014). Learning analytics is argued to improve existing assessment practices by providing continuous, formative assessment that can be used to both identify a learner’s strengths and weaknesses and subsequently adapt instruction (Johnson et al., 2011).

Fig. 4

Analytics technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

In total, five HRs predicted that analytics technology would influence education from 2014 to 2019. Though continuously predicted to be important, the impact was always predicted in the future and never moves to the near-term. This might occur because the technology was first applied in higher education, primarily on at-risk students (Johnson et al., 2011), and the application to elementary education was deemed more difficult. The implementation in K-12 settings has been stymied due to the inherently qualitative nature of elementary assessment that is not amenable to the big data approach needed for learning analytics (cf., university grading systems, Zhang et al., 2018). By the 2017 HR, the authors noted that the delayed impact of learning analytics on K-12 education could partially be caused by the enterprise market driving investment in analytics technologies and causing the development of technologies that meet the needs of enterprise and not education. The noted exception being the development of learning dashboards that track and visualize student performance. The growing interest in learning analytics coincides with the larger societal interest in ‘big-data’ and its uses across business and public policy (e.g., Kim, 2017; McGregor et al., 2013).

Games

Gaming technologies (see Fig. 5) focus on how digital games can be used to facilitate learning, such as game-based learning and gamification. Game-based learning involves the creation of educational experiences in which content knowledge or procedures are imbedded into the mechanics of the game such that playing the game and learning occur simultaneously (Dubé & Keenan, 2016). Gamification involves incorporating reward and leveling systems from video games into traditional academic tasks (e.g., complete math problems and receive a star, for reviews of games in education see Landers, 2014; Plass et al., 2020; Young et al., 2012).

Fig. 5

Game technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

In the 2011 HR, game-based learning was predicted to affect education due to schools integrating online games into classrooms. Online learning games provide free access to educational software that previously required download and installation. The 2012 HR again focused attention on games citing that serious games helped students engage with learning content (Boyle, 2016); role-playing games offered students the opportunity to see the world from a different perspective (Annetta et al., 2009); online social games developed student’s communication and collaboration skills (Paraskeva et al., 2010); and game-designing classes fostered learners to creatively construct knowledge (Games, 2010). The 2014 HR specifically highlighted the importance of gamification and discussed how game-like elements could be applied to daily learning and produce a more engaging and motivating classroom experience.

From 2004 to 2017, the HRs made more predication about gaming than any other educational technology. Gaming appeared in six of the seven HRs from 2004 to 2010 (except 2009, Martin et al., 2011) and appeared in three of the seven reports from 2011 to 2017. As such, games were predicted to have an impact almost every year from 2006 to 2014. Across all of the reports, most of the predictions were mid-term impacts. These repeated predictions suggest a sustained interest in games but with an impact that was perpetually two to three years away. Overall, games show promise as learning tools and have captured steady attention from the HRs, but the continuous mid-term predictions suggest that it is taking more time to implement games in the classroom than earlier HRs foresaw.

Simulation technologies

Simulation technologies (see Fig. 6) provide an immersive and interactive learning environment for learners by placing them in virtual reality (VR) or by blending virtual data or visualizations into the real world using augmented reality (AR).

Fig. 6

Simulation technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

In the 2012 HR, AR was forecasted to have a long-term effect on education by 2016 with mention of how advancements in both the Apple iOS and Android operating systems were allowing for augmented reality applications to be developed for mobile. The report also mentions how augmented reality will move beyond mobile with the announcement of Google’s ‘Project Glass’, an AR system that provided a heads-up display in the user’s line-of-sight. In the 2013 HR, virtual and remote laboratories (e.g., virtual frog dissection) were predicted to have a long-term effect on secondary education by 2017. In the 2016 HR, VR was forecasted to permeate the mainstream of K-12 education in the mid-term citing the recent successful application of VR to other areas (e.g., entertainment) and the availability of affordable mobile VR (e.g., Google Cardboard). The 2016 HR highlighted the potential benefits of simulation in education, specifically the ability for lower income schools to create virtual science labs and go on virtual fieldtrips. In the 2017 HR, mobile VR was again cited as contributing to interest in the technology along with a financial prediction by Goldman Sachs stating that the VR industry would ‘reach 15 million learners by 2025’ (Freeman et al., 2017, p. 46). However, the HR did note that the impact of VR would be in mid-to-far-term due to time required to develop educational software for the mobile VR market. The 2017 HR report highlighted the potential to foster other soft-skills like collaboration, language development, and empathy. The two most recent HRs predicted virtual reality to have an impact across 2017 to 2019 and this prediction coincided with the development of more affordable consumer grade VR systems (e.g., Oculus Rift, Playstation VR).

The HRs’ focus on the potential of VR to simulate learning environments and support soft-skill development was supported by early reviews of VR education research (see Hew & Cheung, 2010). The potential of consumer-grade mobile VR systems to foster educational use has also been cited by recent researchers (see reviews by Jensen & Konradsen, 2018; Kavanagh et al., 2017). These reviews concluded that the head-mounted displays used in these consumer grade systems can engage students (e.g., Loup et al., 2016), improve spatial reasoning (e.g., Rasheed et al., 2015), and train emotional responses to adverse situations (Anderson et al., 2013); but HMDs could also distract from learning due to the frequent occurrence of both motion sickness (e.g., Madrigal et al., 2016) and technological problems using the devices in an educational setting (e.g., space requirements).

Artificial intelligence

AI technologies (see Fig. 7) support live adaptive learning with tailored content (cf., analytics-based lesson planning based on historical records).

Fig. 7

Artificial intelligence predicted to impact education according to the Horizon Reports from 2011 to 2017

The 2016 HR predicted that AI would have an impact on education in the far-term. The report cited an influential milestone in the AI field that occurred in March of 2016, when Google’s AI program AlphaGo defeated the world Go champion. Following this event, both the 2016 and 2017 HRs predicted AI to be influential in the Far-Term. The 2016 HR identified the existence of imbedded AI that students already use but are not aware of (e.g., Digital assistants like Siri, Google Search) as current influences of AI on education and the existence of Chatbots that interact with learners to facilitate second-language acquisition (e.g., Duolingo). The 2017 HRs placed greater emphasis on the potential of AI to perform ‘administrative’ tasks like grading as to allow teachers more time for individualized instruction. Roll and Wylie’s (2016) review of AI education research proposes that AI is developing along two co-existing tracks in education. One track is enhancing current practices (e.g., cognitive tutoring systems combining AI technology and curriculum) whereas the other track is redefining educational practice (e.g., AI driven formative assessment via consistent feedback). Both the HRs and researchers argue for these potentials but acknowledge that these changes are not likely to occur anytime soon.

Other technologies

Several individual technologies that did not meaningfully cluster together were predicted by the horizon report, many of the them repeatedly (see Fig. 8). Technologies or practices that received multiple predictions include cloud computing (e.g., Google Classrooms) with near-term predictions in the 2011, 2013, and 2014 HRs; open content (e.g., Kahn Academy) with mid-term predictions in the 2011 and 2013 HRs; internet of things (e.g., Smart Televisions) with far-term predictions in the 2014 and 2017 HRs; and personal learning environments (i.e., technologies and practices that enable and foster self-directed learning) with a far-term prediction in the 2011 HR and a mid-term prediction in the 2012 HR. Technologies or practices that received a single prediction include far-term predictions for natural user interfaces in the 2012 HR and digital badges in the 2015 HR and a near-term prediction for online learning in the 2016 HR.

Fig. 8

Other technologies predicted to impact education according to the Horizon Reports from 2011 to 2017

Bibliometric analysis

In the previous section, a brief discussion of the HRs predictions alongside educational researchers’ interests in these different technologies shows some alignment between the HRs and the educational technology field at large. To further evaluate the accuracy of the HRs predictions, a bibliometric analysis was conducted based on step 5. Table 1 shows the total number of educational publications available in Google Scholar from 2011 to 2018 along with their weighting factor (WFi), as calculated using the same equation as Martin et al. (2011) and explained in step 5. Table 2 shows the total number of publications and the weighted number of publications available for each of the analyzed years in each technology cluster.

Table 1

The number of educational papers available in Google Scholar from 2011 to 2018 and their corresponding weighting factor

Year	Number of papers available	Weighting factor (WFi)
2011	188,000	0.825664894
2012	194,000	0.800128866
2013	188,000	0.825664894
2014	176,000	0.881960227
2015	155,000	1.001451613
2016	142,000	1.093133803
2017	106,000	1.464386792
2018	92,800	1.672683190

Table 2

The raw and weighted number of educational papers available in Google Scholar from 2011 to 2018

Year	Mobile		Maker		Analytics		Simulation		Games		AI		Others
	R	W	R	W	R	W	R	W	R	W	R	W	R	W
2011	1549	1279	246	203	101	83	303	250	1428	1179	103	85	1651	1363
2012	1827	1462	241	193	203	162	296	237	1553	1243	98	78	1802	1442
2013	2158	1782	263	217	210	173	343	283	1599	1320	85	70	1883	1555
2014	2055	1812	311	274	351	310	346	305	1652	1457	63	56	1954	1723
2015	2289	2292	291	291	377	378	356	357	1778	1718	97	97	1905	1908
2016	2055	2246	382	418	516	564	541	591	1742	1904	135	148	1987	2172
2017	2131	3121	441	646	551	807	673	986	1713	2508	256	375	1891	2769
2018	2058	3442	496	830	564	943	828	1385	1800	3011	488	816	2032	3399

As in Martin et al. (2011), the results from Table 2 are graphically represented in Fig. 9 and depict the publishing evolution for each technology cluster. Figures 10, 11, 12, 13, 14, and 15 show the weighted number of publications for each individual cluster and provide detailed information on the contribution of each specific technology within the cluster (e.g., contribution of tablets to the total number of mobile publications). The following section will discuss the accuracy of the HRs predictions for each technology cluster.

Fig. 9

The weighted number of publications in Google Scholar by technology cluster from 2011 to 2018

Fig. 10

The weighted number of publications in Google Scholar for the mobile technology cluster from 2011 to 2018

Fig. 11

The weighted number of publications in Google Scholar for the maker technology cluster from 2011 to 2018

Fig. 12

The weighted number of publications in Google Scholar for the analytics technology cluster from 2011 to 2018

Fig. 13

The weighted number of publications in Google Scholar for the games cluster from 2011 to 2018

Fig. 14

The weighted number of publications in Google Scholar for the simulation technology cluster from 2011 to 2018

Fig. 15

The weighted number of publications in Google Scholar for other technology cluster from 2011 to 2018

The HRs from 2011 to 2017 contained a total of 42 predictions (6 per report, 7 reports) and mobile technologies accounted for the largest percentage of overall predictions (i.e., 9 predictions or 21%). The bibliometric results revealed that compared to other themed clusters, mobile technology had the biggest impact on educational research from 2011 to 2018 (see Fig. 10). Both the raw and weighted number of mobile technology publications in Google Scholar increased steadily across 2011 to 2018. with a 269% increase in the weighted number of publications across the period. This pattern matches the HRs’ short-term and long-term predictions for mobile technologies, especially when considering the specific technologies within the cluster. Figure 11 shows the proportion of publications for each technology in the mobile cluster. Mobile technology was the most published topic in this group, followed by tablets, Apps, wearables, and BYOD. The number of articles on APPS shows an increased focus on this aspect of mobile technology that is not predicted by the HRs. However, the increased number of articles on wearables in 2017 and 2018 corresponds well with the HRs’ predictions.

Maker technologies accounted for the second largest percentage of overall predictions (i.e., 7 or 16%) but had the fifth highest level of publications (see Fig. 8). Both the raw and weighted number of publications increased across 2011 to 2018, with a 408% increase in the number of weighted publications. Within the maker technology cluster (see Fig. 11), robotics had the highest number of publications followed by 3D printing and makerspaces. Robotics largely accounted for the considerable growth in the cluster, despite it only receiving two of the seven predictions. The HRs predicted both 3D printing and makerspaces to have an impact starting in 2015 and this is somewhat reflected by an increased number of publications in that year. Given the discrepancy between the number of predications and publications, it seems that the HRs overemphasized the impact of maker technology on education overall and underpredicted the relative contribution of robotics to the maker movement.

Analytics technologies accounted for the third largest percentage of overall predictions (i.e., 5 or 12%) and had the fourth highest level of publications (see Fig. 8). Both the raw and weighted number of publications grew across 2011 to 2018, with a 1130% increase in weighted publications across the period. The HRs predicted an increased impact starting in 2015 and this is reflected by the 191% increase in weighted publications across 2012 to 2014 but a 250% growth across 2015 to 2018 (see Fig. 12). Within this cluster, the majority of publications are on learning analytics (cf., adaptive learning technology) and this too aligns with the HRs’ emphasis.

Gaming technologies accounted for the fifth largest percentage of overall predictions (i.e., 3 or 7%) but had the second largest impact on educational publications (see Fig. 10). Both the raw and weighted number of game publications increased steadily across 2011 to 2018, with a 255% increase in the weighted number of publications across the period. Within the games cluster (see Fig. 13), there were far more articles on games than the more specific gamification or game-based learning topics, but interest in gamification rose notably across the period by 2687%. Despite the HRs not predicting a major impact of games on education past 2015, the growth rate of game publications actually increased in this period. Overall, the data suggests that the HRs grossly underestimated the continued impact of games on education during this period.

Simulation technologies accounted for the fourth largest percentage of overall predictions (i.e., 4 or 9%) and had the third highest number of publications (see Fig. 10). Both the number of actual and weighted articles decreased from 2011 to 2012 but then steadily increased from 2013 to 2018, with a 554% increase in the weighted number of publications across the period. This pattern reflects the HR predictions, in that no predictions were made prior to 2012. Figure 14 shows the proportion of publications for each technology in the simulation cluster. Augmented reality generated the most publications followed closely by virtual reality, with virtual and remote laboratories in a distant third. This aligns with the HRs in that AR was predicted to have an effect on education earlier than VR and that VR’s effect on education was predicted to occur starting in 2018, which is when the number of VR articles matched the number of AR articles.

AI and other technologies

Artificial intelligence accounted for the lowest percentage of predictions from any of the clusters (i.e., 2 or 5%) and had the lowest number of publications (see Fig. 9). Both the raw and weighted number of publications decreased from 2011 to 2014 by 45% (weighted) but then increased from 2015 to 2018 by 841% (weighted). This increase is reflected in both the 2016 and 2017 HRs including AI in their far-term predictions. Given that only one technology populated the AI cluster, a figure of its individual publications is not included. Figure 15 shows the proportion of publications for each technology in the cluster ‘other’. Online learning consistently generated the highest number of publications across 2011 to 2018, such that it accounted for 84% of all publications in 2018 and is responsible for the other cluster ranking near the games and mobile technology cluster. This high level of impact is not reflected by the HRs, which only made one, rather general, near-term prediction for online learning in the 2016 HRs. In contrast, cloud computing and internet of things were the subject of more HR predictions but generated far fewer publications. Yet, the HRs predictions that cloud computing would have an early impact while internet of things would have a later impact is somewhat supported by the bibliometric results.

HR predications: accuracy and limitations

The preceding bibliometric analysis highlights how the HRs predictions are not always successful. To further illustrate this and to facilitate comparison between the present HRs’ predictions and the ones from Martin et al. (2011), Table 3 categorizes individual predictions across both studies according to their accuracy. The categorization reflects the accuracy evaluations made in the discussions by Martin et al. (2011) and in the preceding results. Martin et al. concluded that 37% of the individual HRs’ predictions were accurately predicted or slightly delayed whereas 41% of HRs’ predictions were deemed accurate or delayed in the present study. In both studies, a considerable number of individual predictions were deemed overestimations. These results further support the importance of evaluating the HRs’ predictions using bibliometric analysis and not just accepting them as pure reflections of actual technology trends.

Table 3

HRs prediction accuracy across Martin et al. (2011) and the current study

	Accurate	Delayed	Underestimated	Overestimated
Martin et al. (2011)	Social networking User-created content Game Virtual worlds Mobile	Grassroots video Collaborative web		Extended learning Personal broadcasting Social computing Massive gaming Personal web Social operating systems Knowledge web Learning objects Open content Augmented reality Ubiquitous computing Context-awareness
Current study	Mobile Wearable technology Robotics Learning analytics Virtual reality Augmented reality Artificial intelligence Internet of things	Cloud computing	Apps Game-based learning Online learning	BYOD Makerspaces 3D printing Adaptive learning technologies Analytics technologies Virtual and remoted laboratories Open content Natural user interface Personal learning environments Digital badges

Despite the evaluative value bibliometric analysis provides, using the number of publications on a given educational technology is not a perfect indicator of that technology’s influence on actual educational practice and is an imperfect substitute for directly observing technology use in classrooms. However, more direct data on educational technology adoption (e.g., school technology purchase rates) is largely not obtainable or limited to specific geographic regions. Further, studying broader technology adoption rates (e.g., overall purchase rates of tablets) runs the risk of assuming technology trends outside of schools are mirrored within them. This being said, the result of the bibliometric analysis should not be interpreted as directly reflecting the impact any one educational technology has on practice. Further, the extant body of educational technology research is often criticized for focusing on what is emerging (cf., pervasive) and on English speaking, developed nations. A similar critique can be made of the HRs themselves. As such, the present results and discussions should be interpreted with this limitation in mind.

Discussion

These results identify the K-12 educational technology trends predicted by the HRs from 2011 to 2021 and evaluate the accuracy of these predictions against the number of academic publications on these technologies. The HRs are an influential document with 500,000 downloads per year across 195 countries that are the product of deliberations among technology and education experts on how they see the future of educational technologies developing. Should teacher training and technology purchases be informed by the HRs? That is a difficult question to answer, but evaluating these reports provides a useful calibration for the numerous policy makers and educators who use them. Further, the HRs predictions are only a description of future potentials (i.e., models) and evaluating which predictions come to pass provides information on both what has occurred and on the prediction process itself.

Over seven-years of forecasts, the reports predicted that mobile technology would be the most influential educational technology from 2011 into the near future. Given that mobile technologies were the most impactful in the HRs from 2002 to 2010, this further reinforces the influence of mobile devices on education. Maker technology and games were predicted to impact education from 2015 to 2018 and 2012 to 2016, respectively. Analytics technologies’ impact was predicted to increase and would continue to influence learning along with other emerging technologies like VR and AI. Thus, the HRs predictions continue to highlight both pervasive (mobile) and emerging technologies (VR, AI, Maker) while recognizing the social webs’ declining influence on education.

The bibliometric analysis suggests that the HRs’ accurately predicted the most influential educational technology (i.e., mobile) and was fairly accurate for the fourth most influential technology (i.e., analytics technology). Predictions for maker technologies (i.e., 3D printing and robotics) were somewhat overstated and placed too great an emphasis on 3D printing and maker spaces over robotics. In contrast, the HRs’ predictions around games were far too conservative but did accurately foresee an increased interest in gamification. Thus, the prediction accuracy of the HRs was mixed. Some of these mixed results could be due to a fundamental assumption underlying the HRs; that the future of educational technology depends on larger societal trends. However, this assumption fails to consider the pedagogical value of a given educational technology and, perhaps more importantly, the additional barriers that prevent technologies from being adopted into K-12 classrooms. Mobile technologies are ubiquitous in society and are increasingly affordable. As such, it makes sense that the horizon reports accurately predicted their impact on education. In contrast, maker technologies are receiving a lot of attention at a societal level (e.g., news stories, featured in popular TV shows like Grey’s Anatomy) but they require considerable training to use and are relatively expensive to purchase and maintain. This may reflect how the HRs may ‘listen’ to popular discourse around technology more so than practitioners’ concerns. While evaluating the pedagogical merit and impact of each technology identified in this study would be beyond the scope of the present endeavor, Table 4 in Appendix contains a listing of recent systematic reviews for each technology cluster along with a brief overview for each paper. Having identified the technologies predicted to trend across 2011–2021, these systematic reviews will help evaluate their supposed merits and impact.

Table 4

Systematic reviews for each technology cluster

Technology cluster	Citation	# Studies reviewed	Overview
Mobile technology	Crompton and Burke (2017)	36	Focuses on mobile learning in mathematics. Most studies focused on mobile phone use in elementary settings and showed positive learning outcomes
	Crompton et al. (2016)	49	Focuses on mobile learning in science from 2000 to 2016. 51% of studies aimed at designing a system for mobile learning while 29% of the studies evaluated the effectiveness of mobile learning
	Liu et al. (2014)	63	Mobile learning in sciences, mathematics, and second-language learning. In comparative studies between mobile learning and traditional learning, majority showed learning gains
	Xie et al. (2018)	47	Mobile learning with and without disabilities. All studies reported positive effects of mobile learning in supporting students with disabilities
Maker technology	Benitti (2012)	10	This paper revealed that robotics were mostly applied in STEM courses and reported to improve academic achievement as well as problem solving skills
	Ford and Minshall (2019)	44	This paper summarized the use of 3D printing in six different education settings (e.g., elementary vs university)
	Ioannou and Makridou (2018)	9	Robotics involves students actively interacting with robots to construct knowledge and build social skills
Analytics technology	Bodily and Verbert (2017)	93	The article focuses on analytics reporting systems. Findings suggest mixed results for behavior and achievement but clear improvement for self-awareness and engagement
Games technologies	Byun and Joung (2018)	17	The paper reviewed digital game-based learning (DGBL)’s effect on students’ math achievement. Results indicate DGBL produces a small, positive effect
	Li and Tsai (2013)	31	This paper reviewed DGBL in science from 2000 to 2011. Two thirds of digital games studied were used to teach content knowledge, few promoted problem solving skills, engagement, or affect
	Merino-Campos and Fernndez (2016)	100	Studies on video games in physical education from 2010 to 2015; impact on students’ attitudes, cogntive skills, and motor skills discussed
Simulation technology	Hew and Cheung (2010)	15	Focus on 3D immersive virtual worlds. Three central topics include: affective domain, learning outcomes, and social interaction. In general, 3D immersive virtual can improve learning outcomes and foster social interactions
	Jensen and Konradsen (2018)	21	Application of head-mounted displays (HMDs) in education. HMDs are only helpful in improving cognitive skills, psychomotor skills, and affective skills under specific conditions
	Kavanagh et al. (2017)	99	Use of VR across diverse subjects. Improving student intrinsic motivation the main impetus for VR use. Problems associate with virtual reality deployment are also discussed
Artificial Intelligence	Magnisalis et al. (2011)	105	Use of intelligent systems to support collaborative learning. In general, potential to improve learners’ domain knowledge and collaboration skills, but effects limited by learning design and intelligent system’s sophistication
	Roll and Wylie (2016)	47	Discuses shifting foci of studies on AI in education. Shifts include change from system description and evaluation to modelling and from improving domain knowledge to motivation and collaboration skills

The tendency for educational technology adoption to follow societal factors is not limited to the HRs’ predictions. For example, both the year of prediction and the publication rates for emerging technologies seem to coincide with availability of the technology at a consumer level (i.e., affordable). Consumer level maker and VR technologies became available the same year they were included in the HRs and their publications rates increased in the two years following their commercial availability. This suggests that both predicted and actual trends in educational technologies are driven more by their availability than their educational affordances and exemplifies the longstanding criticism of the educational technology field as placing an overemphasis on ‘stuff’ (i.e., devices) at cost to pedagogical practice and theory building (Richey, 2008). Finally, the COVID-19 pandemic (which occurred during the revision of this paper) brings to light another factor affecting the educational technology industry, historical events and societal shifts. Predictions are based on the assumption that past and current behavior's determine future ones, but they cannot take into account unforeseen events (e.g., a global pandemic that moves education online). While the pandemic and the rise of online learning are an extreme example, more minor ones include societal shifts to and away from technologies for reasons unrelated to education (e.g., current disillusion with social media).

Allowing industry to direct educators and researchers’ gaze towards specific technologies is particularly problematic considering that many technology companies are just as quick to invest as to divest in a given technology. For example, Google entered the mobile VR market in October of 2017 with the affordable Daydream headset but abandoned the product line entirely in October of 2019 (Robertson, 2019). Researchers or educators turning to mobile VR because of Google’s investment would be left with devices that are now wholly unsupported. Thus, the trends identified in this study indicate a worrisome practice of researchers and educators following the investment whims of technology companies (i.e., a marketplace effect) but arguably being less able to course correct as quickly as the companies they follow. Interestingly, this marketplace effect on the HRs was not identified in the previous works by Martin et al. (2011, 2018). A lesson to be learned from this, for prognosticators, users, and researchers of educational technology, is to be less swayed by technologies that are cheap and available today (VR, 3D printing) and more focused on technologies that show signs of permanence (e.g., mobile).

The bibliometric analyses indicate that educational technology continues to be a growing field and topic within the greater educational research discourse and whether or not this growing interest is a net positive for education is up for debate. Both the actual and weighted number of publications on educational technology increased from 2011 to 2018, representing an approximately 300% increase in the amount of researcher discourse on educational technology across the period. While an increased interest in educational technology is warranted, given the influence of technology on society generally across this period, it does raise questions about the impact this increased level of research discourse will have on students. Tawfik et al. (2016) discussion on the consequences of technology in education made a strong case that an unmindful adoption of technology runs the risk of unintentionally increasing societal inequities in the classroom. Thus, the meteoric increase in educational technology discourse seen here could benefit students but only if the discussion considers who is included and who is excluded. For example, the largest trend in terms of predictions and research discourse was for mobile technologies. Much of this discourse within this trend assumes that students not only have a device (i.e., BYOD) but that they can access the internet on the device outside of school (i.e., anywhere learning). Discussions about the impact of mobile technologies on education thus run the risk of excluding or ignoring students who do not have devices or unlimited mobile internet access. Similar issues of equality of access likely exist for many of the technologies identified in this study and future works should use the approach forwarded by Tawfik et al. (2016) to critically examine each of the major trends identified herein.

Conclusions

This work provides an updated picture of K-12 educational technology trends in the past and near future by collating individual technologies predictions across seven Horizon Reports, identifying larger trends from these individual predictions, and evaluating the prediction accuracy using bibliometrics. The previous trend analysis by Martin et al. (2011) identified 7 technologies believed to affect educational practice from 2004 to 2010; including the social web, mobile, games, semantic web, human computer interaction, learning objects, and augmented reality (in order of impact). The present work identifies 6 technologies believed to affect education practice from 2011 to 2017; including mobile, games, analytics technologies, simulation technology, maker technology, and AI (in order of impact). A direct comparison between the two studies shows a deemphasis on social networks as an emerging educational technology, a continued influence of both mobile and game technologies, and an emerging influence of learning analytics and AI. Looking at both studies also highlights the importance of not relying on any one year of HR predictions but rather the long-term trends that arise from multiple reports, as reports in individual years are overly swayed by the availability of new technologies. Taken together, the present study and Marten et al.’s study provide a continuous tracking of major educational technology trends from 2004 to 2021, which can serve as a state of the field for researchers, policy makers, and educators interested in how technology has and continues to influence educational practice in the twenty-first century.