E-learning in higher education institutions during COVID-19 pandemic: current and future trends through bibliometric analysis.

The purpose of this study is to provide a comprehensive review of e-learning in higher education institutions (HEIs) during the COVID-19 pandemic. This study presents current research issues and future progress in HEIs e-learning in the wake of the COVID-19 pandemic from 2019 to 2020 through the Web of Science (WOS) database. The analysis comprises citation analysis, co-citation analysis, and co-occurrence of keyword analysis. This bibliometric review provides essential insight for researchers to identify the most influential publications and determine the fundamental structure. Furthermore, this review facilitates future studies on the influential research trends and emerging topics. Findings present several streams of discussion based on the clusters identified from the three analyses. Subsequently, the theoretical and methodological implications on the emergence of new sub-fields and interesting future works in connection with e-learning in HEIs are presented. The outcome would aid scholars and practitioners in advancing the technology and aid towards e-learning in HEIs to ensure students' education can be sustained in the state of emergency.

Introduction

The outbreak of the COVID-19 pandemic has caused uncontrollable downfalls and interruptions in many sectors, including education (Shahzad et al., 2021). Since then, e-learning or online learning has become one of the hottest and controversial topics (Liu et al., 2021). The unprecedented event has liberated e-learning as one of the most critical components in teaching and learning in higher education institutions (HEIs) worldwide (Pham and Ho, 2020). It has changed the way of conventional learning of face-to-face physical interaction shifting to e-learning. It has provided a sense of assurance that students are not deprived of education and learning is not halted, even during an emergency (Azzi-Huck and Shmis, 2020).

The application of computer technology and internet connection is considered the main component of e-learning (Maatuk et al., 2021). Despite the many terminologies of e-learning, such as online learning (Moore et al., 2011), distance learning (Elfirdoussi et al., 2020, Pham and Ho, 2020), virtual learning (Almarzooq et al., 2020), among others, all nomenclature reflects the nature of online learning. The utilization of accessible learning through the web, where students and instructors are distance away, with the internet connection through computer devices such as laptops and smartphones. Such learning environments can be categorized differently according to the learning realm education institutions face (Moore et al., 2011). These learning environments' design relies on the learning objective, accessibility (physical, virtual, or both), content, and audience (Moore et al., 2011).

The motivation behind this study is twofold. First, with the sudden interest in e-learning since the emergence of COVID-19, a plethora of research has emerged in the past two years. A comprehensive review on e-learning adoption in HEIs would be beneficial to scholars and researchers to review relevant research gaps to capture this intriguing field's intellectual structure and predict emerging trends and work in the near future (Leung et al., 2017). Despite having several qualitative and subjective nature of previous literature review studies on e-learning in HEIs during COVID-19 (Turnbull et al., 2021; Shahzad et al., 2021) and in developing countries' contexts (Zarei and Mohammadi, 2021). Other studies include Adedoyin and Soykan (2020), which provide a narrative review on online learning as an educational platform in HEIs and discussed the crisis response among faculty members, students, and administration. At the same time, Mseleku (2020) reviewed the challenges academics and students face, including the inability to access online tools, adjustment towards classes and lecturers, especially students living in rural areas, and association with mental health. On the other hand, Poudevigne et al. (2022) provide the perspective on medical and health students' mental health and the coping mechanism with virtual classes.

To the best of the author's knowledge, there has been no systematic review based on bibliometric analysis that captures the knowledge structure through mapping and visualization on the specific context of e-learning adoption in HEIs during COVID-19 pandemic. This study is the first that provides a quantitative review of the literature through the bibliometric analysis based on the Web of Science (WOS) database. Such an approach will help researchers explore and gain deeper insight into this area and predict future trends in HEIs e-learning. Moreover, this review will dictate the focus of past studies over the two years and the relationship among the influential clusters and themes of HEIs e-learning adoption that might be relevant in the coming years even after the COVID-19 subside.

Second, the interest in COVID-19 research has expanded tremendously, with thousands of documents published in a short period (Alam and Asimiran, 2021). With such an enormous amount of publication, it is intriguing to delve into the trending issue in e-learning adoption publications that might produce a better understanding of adapting e-learning in the context of HEIs. The method to cope with the pandemic in HEIs context from these publications with proper mitigation and intervention plan could provide an insightful understanding to HEIs in developing their plan based on best practices for the continuum of education.

Our study contributes to the e-learning literature in at least three domains. First, by presenting a holistic evaluation of scholarly contribution on e-learning in higher education and identifying the linkages to discern the intellectual structure of e-learning in HEIs during and post-pandemic. Second, the hype surrounding the adaptation of e-learning in education since the emergence of COVID-19, this study would critically connect the author's work from multiple disciplines by reviewing its evolutionary path (Hota et al., 2020). Finally, through co-citation analysis, this study would demonstrate the construction of the social relationship of the current topic at a particular time.

Literature review

Since the outbreak in February 2020, many countries have implemented social distancing and house quarantine for their people (Shahzad et al., 2021). After China has taken the necessary measures by closing all education institutions, including schools, colleges, and universities, other countries have followed suit. UNESCO reported that more than 160 countries implemented nationwide closures, which approximately impacted 87% of higher education students (Araú jo et al., 2020). The closure had impacted an estimated a billion students who rely on physical and face-to-face interaction in these education premises.

The COVID-19 pandemic has abruptly changed the education sector from physical education to e-learning and internet platforms. E-learning is not new to the HEIs system. It has been utilized before the pandemic, but not on a full scale. Physical classes were fully implemented, and e-learning only served as complementary. Since the outbreak, the state of emergency has forced HEIs to revive and resort to a full e-learning platform. Video and teleconferencing platforms have received a momentous inclination as the education sector seeks alternatives amid lockdown and home quarantine. The platform includes Google Classroom, Zoom, Webex, and Microsoft (Dash et al., 2021). These technologies provide the HEIs community and the general population opportunities to improve teaching and learning (Maatuk et al., 2021).

Despite the benefits of keeping physical distance in e-learning, there are several disadvantages and challenges, particularly for students (Maatuk et al., 2021). Several issues have arisen regarding student engagement with these virtual classes. Despite the opportunities from these technologies, some students face deliberate challenges such as internet connection, lack of gadgets especially coming from low-income families (Kapasia et al., 2020). Specific courses and majors could not implement e-learning when it involves hands-on and practical learning, especially in medical, technical, and engineering courses. Furthermore, concern on student mental wellbeing due to abundance of limitation during e-learning classes has also gripped the HEIs management. Accordingly, this study would provide the view from bibliometric analysis to discover the potential themes being discussed and discover the extent of opportunities the technology can offer for the sake of maintaining the status quo of HEIs (Almaiah et al., 2020).

The study would contribute to the body of knowledge in e-learning on the current issues and future trends that might influence e-learning in HEIs during and post COVID-19 pandemic. This study fills in the gap on the best practices of e-learning adoption in the context of HEIs during a state of emergency. This unprecedented event has led to widespread e-learning applications that HEIs need to capitalize on to ensure that students receive an education despite the emergency state. This bibliometric analysis of e-learning in HEIs contributes to the rich literature of e-learning adoption in the HEIs context by presenting a descriptive and quantitative overview through multiple bibliometric analytic techniques. Such an approach reduces the subjectivity while increasing the reliability of finding based on recent issues in HEIs e-learning. First, the author applies citation analysis by integrating all citations within the topic in the WOS. The citation analysis aims at identifying critical publications in the current topic to understand the issues at hand. Citation analysis is helpful for future scholars in the same area to start by referring to the most cited publications that would bridge to other related publications (Weerakoon, 2021).

Secondly, from the co-citation analysis, this review would facilitate scholars and researchers to capture the structure and central issue of the literature in e-learning adoption in the HEIs context (Zupic and Cater, 2015). The COVID-19 impact would be the same to every HEIs throughout the world, thus focusing on the critical issue from the recent publication. Best practices can be imitated and adapted to HEIs own education framework to assure education would not be compromised despite the state of emergency. Furthermore, COVID-19 is here to stay and is expected to evolve as an endemic (Deguma et al., 2021).

Thirdly, from the co-occurrence of keyword analysis, this study contributes to the semantic content overview of the topic by investigating the most relevant themes primarily discussed by authors in the literature (Bernatović et al., 2021). Co-occurrence analysis facilitates scholars to predict future research trends in HEIs adoption of e-learning. It analyzes the relationship among the keywords in all the publications by visualizing the knowledge structure from various perspectives (Van Eck and Waltman, 2014).

This study tends to achieve the following research objectives:

To identify central articles that provide a significant impact on e-learning in higher education during the COVID-19 pandemic.

To explore the structure of e-learning adaptation in higher education institutions.

To comprehend the evolution of e-learning by identifying the linkages among central articles and the evolution of such linkages over time.

Methodology

Bibliometric research

The bibliometric research method is a science mapping technique that investigates the relationship among disciplines, fields, scholars, and individual papers (Zupic and Carter 2015). Such a method has gained high interest among scholars as it maps the structure of the scientific field through the integration of classification and visualization (Boyack and Klavans, 2014; Van Eck and Waltman, 2014). It is considered one of the powerful methods in reviewing the literature based on scientometric database networks. Accordingly, based on the bibliometric approach, this study consummates a complete publication related to e-learning in HEIs, specifically during the COVID-19 pandemic.

Stage 1: data collection

A search string was developed consisting of three main sub-keywords and related keywords found in synonym, thesaurus, and past research 1) e-learning, 2) higher education institution, and 3) COVID-19. The search string for e-learning and related was mainly adapted from Sweileh (2021). The search string for COVID-19 includes "coronavirus 2019″ OR "covid-19″ OR "sars-cov-2″ OR "sars-cov2″ OR "covid19″ OR "covid 2019″ OR ″2019-ncov" while for HEIs include “higher education institution” OR “university∗” OR “college∗” OR “institution of higher learning” OR “faculty”. A detailed search string is presented in appendix 1. The search utilizes the "topic" field where the keywords will be searched in the publications' title, abstract, and keywords. Instead of the "all field", the study meant only to limit publications that exist in the said field instead of appearing in the text that is highly irrelevant to the context. No geographic, language barrier, or other constraints in the retrieval procedure was applied. All the articles retrieved was from the year 2020 and 2021. The search in the WOS database was conducted on October 3, 2021. We only adapted WOS database as it is the best, most robust database with more than 74.8 million scholarly data and datasets across 25 disciplines (Singh et al., 2021). The database has currently indexed more than 21,100 journals (Web of Science, 2022). The preliminary search returned 1683 publications.

Stage 2: screening

Articles are selected based only on journal publications and early access. Conference proceedings, review articles, books, book chapters, editorial, and letters were excluded to limit quality publications in reputable journals. Despite that, the amount of publication in the journal is more than enough. To ensure data integrity, all the publications were reviewed on the issue of consistencies and duplications before conducting the bibliometric analysis (Zupic and Cater, 2015). As a result, 1496 publications were screened for the next stage of analysis.

Stage 3: analyzing the data

Data analysis follows four main procedures. First, a descriptive statistic was conducted based on the WOS database to extract relevant descriptive analysis on the article publication, specifically on:

The number of relevant papers

Number of citation and self-citation

Number of h-index

After the descriptive analysis, the following database was analyzed using the VOSviewer software on the citation, co-citation, and co-occurrence of keywords (Van Eck and Waltman, 2014).

Citation analysis

Citation analysis provides an essential direction to researchers in a specific field by depicting the scientific interaction and bridging scholarly ideas among the conceptual links (Small, 1973). Citation analysis dictates that the most often-cited publications are highly influential in a particular subject area (Weerakoon, 2021). Identifying relevant, prominent articles in HEIs e-learning during COVID-19 would help researchers rely on the most influential articles as the primary basis to conduct studies in the current context.

Specifically, citation analysis helps researchers in many ways. In the current study, the analysis helps to 1) visualize citation network: the most frequently cited publications can be obtained, along with the citation relation between the publications and cluster it belongs 2) searching for relevant critical publications: most cited publications in a particular field is considered as the most relevant and critical (Van Eck and Waltman, 2017). The citation analysis also helps to retrieve such documents based on the title, author, journal, and year of publication.

Co-citation analysis

The co-citation analysis helps to systematically analyze e-learning in HEIs. It applies co-citation counts to assess the similarity between documents, authors, or journals (McCain, 1990). Co-citation analysis is a measurement of two publications cited by a third publication, hence known as co-cited publication (Small, 1973; Zupic and Carter, 2015). When more publication is co-citing two particular articles, the stronger the co-citation relation (Van Eck and Waltman, 2014). This approach looks into a forward perspective by showing changes and coherence in the literature over time by identifying publications or authors with other publications/authors (White and McCain 1998).

The current study applies the co-citation analysis based on articles instead of authors. The former analysis focuses on how two articles, rather than authors, are cited together (Hota et al., 2020). The article co-citation analysis is opted due to the current objective to map the intellectual structure of particular interest (e-learning in HEIs during COVID-19). Author co-citation analysis would distort the findings within such interest as authors may have contributed to other broader research areas (Acedo et al., 2006). Furthermore, author co-citation analysis disregards the contribution of co-authors where only the first author will be analyzed (McCain, 1990).

Co-occurrence of keyword

Co-occurrence of keywords has been studied extensively in the literature (Van Eck and Waltman, 2014; Tan Luc et al., 2020). The analysis evaluates the connection that exists between the keywords through the keywords that frequently appear, either in titles, abstracts, or keywords (Bernatović et al., 2021). The analysis is performed by extracting the keywords from the author-supplied keyword list in any article. The bubble size shows the frequency of the keywords, while the intensity of occurrence between two keywords is shown by the thickness of the line connected (Tan Luc et al., 2020).

The summary of answering the research question, the strategy, and the analysis technique is summarized in Table 1.

Table 1

Summary of the research question, strategies, and bibliometric techniques.

RO	Description	Strategy to answer RQ	Technique
1	To identify central articles that provide a significant impact on e-learning in higher education during the COVID-19 pandemic.	To identify articles in HEIs e-learning adoption during COVID-19 with the highest citation	Citation analysis
2	To explore the structure of e-learning adaptation in higher education institutions.	The emergence issue discussed on the relationship among the most co-cited documents	Co-citation analysis
3	To comprehend the evolution of e-learning by identifying the linkages among central articles and the evolution of such linkages over time.	Evaluate the keywords with the highest frequency and relations among the clusters	Co-occurrence of keyword analysis

The following section presents the bibliometric analysis of e-learning studies in HEIs during the COVID-19 pandemic. The analysis dealt with identifying the most influential articles (citation analysis), evaluation of e-learning publication structure in the wake of COVID-19 (co-citation analysis), and prediction of future trends (co-occurrence of keyword analysis).

Result and analysis

Descriptive analysis

The first search result in 1683 articles. After filtering only empirical articles in the journal (excluding review articles, book chapters, conference proceedings), the search ended with 1,496 articles. The sum of citations was 4,826 and 3,460 without self-citation. The average citation per item was 3.17, with an h-index of 27. This topic has received a tremendous amount of interest among scholars as the publication has increased almost double, 503 in 2020 and 902 publications in 2021 (even though data was extracted in October 2021, at the time of writing). It is anticipated that e-learning in HEIs will be further developed and extended, even after COVID-19 has subsided, contributing to its relevancy and potential to sustain educational activities across physical barriers.

Citation analysis

The most crucial hallmarks of COVID-19 in HEIs are identified through the bibliometric document citation analysis. In total, 43,345 references from 1,496 articles were extracted from articles related to e-learning in HEIs and COVID-19 pandemic. Selecting a cut-off of articles having five and more citations, the analysis led to 295 top articles. The top 10 most cited articles are ranked based on the number of citations presented in Table 2, including their title, number of citations, and links strength.

Table 2

Top 10 cited documents.

No.	Authors	Title	Citations	Links
1.	Bao (2020)	COVID-19 and online teaching in higher education: A case study of Peking University	295	14
2.	Murphy (2020)	COVID-19 and emergency eLearning: Consequences of the securitization of higher education for post-pandemic pedagogy	210	3
3.	Aristovnik et al. (2020)	Impacts of the COVID-19 pandemic on life of higher education students: A global perspective.	122	1
4.	Wang and Zhao (2020)	The impact of COVID-19 on anxiety in Chinese university students.	115	7
5.	Almaiah et al. (2020)	Exploring the critical challenges and factors influencing the E-learning system usage during COVID-19 pandemic.	106	8
6.	Adedoyin and Soykan (2020)	Covid-19 pandemic and online learning: the challenges and opportunities.	89	2
7.	Watermeyer et al. (2021)	COVID-19 and digital disruption in UK universities: afflictions and affordances of emergency online migration	79	2
8.	Kapasia et al. (2020)	Impact of lockdown on learning status of undergraduate and postgraduate students during COVID-19 pandemic in West Bengal, India	78	1
9.	Mukhtar et al. (2020)	Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era	78	1
10.	Longhurst et al. (2020)	Strength, weakness, opportunity, threat (SWOT) analysis of the adaptations to anatomical education in the United Kingdom and Republic of Ireland in response to the Covid-19 pandemic.	77	4

Bao (2020) is the most influential paper with the highest citation (293 citations and 11 links). The article was crucial based on its overview of online teaching in HEIs. The article suggested six instructional strategies and five principles in facing online education and classes. It provides useful theoretical and practical guidance for students and instructors in adapting to the new norm of online learning. The second most significant article is by Murphy (2020) that primarily discusses the COVID-19 and the emergency state of e-learning by drawing on the securitization theory to advocate e-learning in education. Murphy (2020) suggested an important point on the pedagogy issue that HEIs have to adapt, post COVID-19. The flexibility element of the asynchronous e-learning adopted during the pandemic has provided more comprehensive access to education to individuals who cannot attend the conventional full-time face-to-face class.

The subject of mental health, particularly among students, has been raised as a critical concern in HEIs e-learning. Wang and Zhao (2020) were among the first study assessing student anxiety state in the Chinese context. It was found that university student develops substantial anxiety due to COVID-19. Meanwhile, Aristovnik et al. (2020) also studied student mental state on their adaptation to e-learning with an even wider coverage by studying 30,383 students from 62 countries. Kapasia et al. (2020) discovered that poor internet connectivity and an unsuitable environment, among other issues, led to students' depression and anxiety.

Most articles discussed the challenges and barriers students and faculty members face in e-learning adaptation during the pandemic (Almaiah et al., 2020; Adedoyin & Soykan). Watermeyer et al. (2021) presented a study in the context of UK universities migrating to online learning in the state of emergency. Meanwhile, Almaiah et al. (2020) explore the HEIs community's critical challenges in adapting the e-learning system and the supporting factors during the pandemic. The study context was in six universities in Saudi Arabia and Jordan. In another study, the strength, weakness, opportunity, and threat (SWOT) analysis through a thematic approach on 14 different universities in the United Kingdom and Republic of Ireland (Longhurst et al., 2020). These articles are highly relevant to the broad domain among researchers as they discuss an in-depth analysis of e-learning adoption during the COVID-19 pandemic. Furthermore, the articles evaluate the importance and challenges of adopting e-learning in HEIs and future works that should be focused on. Furthermore, the issue of hands-on education, particularly raised in medical education, has shed the limelight (Mukhtar et al., 2020).

Co-citation analysis

Co-citation analysis helps the current HEIs e-learning adaptation during COVID-19 on the structure of references based on cited reference analysis function. Employing a threshold for number of citations of cited references to 10, the number of references that meet the required threshold of the 43,345 references was 234. Based on these 234 most cited references, the network analysis was built within the e-learning in HEIs and COVID-19. The most frequently cited references are Cao et al. (2020) (106 times), Bao (2020) (93 times), Dhawan (2020) (72 times), and Sahu (2020) (70 times). Result depicted that the 234 references are divided into five clusters, represented by different colors (Figure 1).

Figure 1

Result of Co-citation analysis.

Table 3 present the top 10 documents with the highest co-citation and total link strength. Figure 1 presents the co-citation network on the highest cited articles in e-learning studies during COVID-19. This network facilitates in identifying the node's strength in the entire network and positioning related citation. The size of the nodes and the line thickness show the citation strength level within the network. The color and lines of the nodes indicate the cluster to which articles belong (Wong et al., 2021).

Table 3

Top 10 documents with the highest co-citation and total link strength.

Documents	Citation	Total link strength
Cao et al. (2020), Psychiat Res, v287	106	274
Bao (2020), Hum Behav Emerg Tech, v2, p113	93	349
Dhawan (2020), Journal of Educational Tech, v49, p5	72	206
Sahu (2020), Cureus, v12	70	222
Fornell and Larcker (1981), J Marketing Res, v18, p39	68	249
Davis (1989), MIS Quart, v13, p319	65	263
Venkatesh et al. (2003), MIS Quart, v27, p425	50	206
Braun and Clarke (2006), Qual Res Psyhol, v3, p77	48	92
Hodges et al. (2020), Educause Rev	48	69
Unesco, 2020, Covid 19 ed disrupti	46	108
Rose (2020), JAMA- J Am Med Assoc, v323, p2131	45	102

Source: Author interpretation based on VOSviewer analysis.

Cluster 1 comprises 54 articles (red color). It is observed that most of these documents were published before 2010, primarily before the advancement of e-learning. Most of these documents discuss the emergence of e-learning, specifically on technology adoption. Cluster 1 is labelled as "e-learning and technology adoption." One of the earliest articles was Davis (1989) that laid out the variables of user-perceived ease of use and perceived usefulness on information technology acceptance. One of the most cited publications, Venkatesh et al. (2003), was considered the seminal article in the technology adoption model. Similarly, this cluster dealt with the structural equation modeling technique methodology to test the relationship among the variables in the model (Fornell and Larcker, 1981) and recent advance methodological approach of using the variance-based approach (Henseler et al., 2015; Hair et al., 2017). A much recent article by Almaiah et al. (2020) explicitly explores the adaptation of e-learning in higher education in the wake of the pandemic.

Cluster 2 comprises 53 articles (green color). The main theme in cluster 2 can be interpreted as “thematic COVID-19 analysis”. Braun and Clarke (2006) outline the methodological perspective in the thematic analysis as the representative document in this cluster. As the pandemic cases keep rising, many researchers adopted the qualitative approach to investigate the theme in HEIs e-learning. Moore et al. (2011) discuss the difference between e-learning and distance education. The main characteristic between the two is that the former can be characterized by the application of technological tools that are web-based or web capable with the presence of an internet connection. In comparison, the latter describes the access for learning based on cross geographical separation. Meanwhile, Hodges et al. (2020) discuss the challenge in implementing online learning and its related disruption to staff, faculty members, students, and everyone associated with HEIs. One crucial point addressed was that even though e-learning is necessary, classes should be conducted in the asynchronous mode rather than synchronous to ensure flexibility in facing the pandemic. Furthermore, Garrison et al. (1999) reported using computer conferencing as guidance to facilitate computer-mediated communication in higher education.

Cluster 3 consists of 47 articles (blue color). Most of the articles in cluster 3 deliberated the challenges and mitigation plans in e-learning. This cluster is labelled as "Challenges and mitigation of e-learning." The highest cited article is Dhawan (2020), which presents the strength, weaknesses, opportunities, and challenges of e-learning in India. Bao (2020) suggested instructional strategies in China context in applying online education and presented high-impact principles for academics and faculty members facing online education. The issue of access internet, monetary issues, lack of face-to-face interaction, and socialization was brought up in the Pakistan context (Adnan and Anwar 2020). Amid the issues in conducting e-learning in HEIs, Toquero (2020) suggests improving teaching and learning by strengthening the practices by constructing the curriculum to be more responsive to the students learning needs. Rapanta et al. (2020) implied the pedagogical content knowledge among non-expert academics in facing e-learning.

Cluster 4 consists of 46 articles (yellow color). This cluster is labelled as “Medical education during COVID-19”. This cluster involves articles discussing the issue of medical students facing the pandemic. The spread of the virus has led to uncertainty in medical education as medical education requires practical hands-on (Ferrel and Ryan, 2020). Faculty members and students are made to consolidate these changes and are grappling to advance in their careers. Mukhtar et al. (2020) delineated that the pandemic has posed several wins and losses to the medical students. It included comfort, accessibility, and remote learning, while the limitation involves difficulty and inefficiency to maintain academic integrity. In another view, the issue of patient safety is realizing that medical students in training may asymptomatic spread and contract the virus (Rose, 2020).

Cluster 5 represents 34 articles (purple). This cluster is primarily concerned with student wellbeing and mental health regardingg e-learning during COVID-19. This cluster is labelled as "student mental health." Many restrictions such as self-isolation, social distancing, and travel restrictions have caused massive losses in many sectors. Closing down education institutions, limited opening of business, reduced workforce, and the onslaught of economic crisis and recession have caused a severe mental distraction to the working population (Nicola et al., 2020). The impact of the pandemic has not spared students in HEIs. They are being forced to stay at home in the state of quarantine while at the same time expected to attend classes online has prompted unnecessary but expected stress and anxiety. Students were deficient in computer skills and heavy workloads that prevented them from improving (Aristovnik et al., 2020). They have to cope with the expectation from instructors and HEIs to attend online classes, involve in activities, complete the assignment and other responsibilities to execute their coursework despite having limited capacity and resources.

To conclude this section of co-citation cluster analysis, Table 4 summarizes each cluster and influential, representative articles.

Table 4

Co-citation clusters on e-learning during COVID-19.

No	Cluster label	Representative articles
1	e-learning and technology adoption	Fornell and Larcker (1981), Venkatesh et al. (2003)Henseler et al. (2015), Hair et al. (2017), Almaiah et al. (2020)
2	Distance education and COVID-19	Braun and Clarke (2006), Moore et al. (2011), Hodges et al. (2020), Garrison et al. (1999)
3	Challenges and mitigation of e-learning	Dhawan (2020), Bao (2020), Adnan and Anwar (2020), Toquero (202)
4	Medical education during COVID-19	Ferrel and Ryan (2020), Mukhtar et al. (2020), Rose (2020)
5	Student mental health	Cao et al. (2020), Sahu (2020), Aristovnik et al. (2020), Odriozola-Gonzá lez et al., 2020, Elmer et al. (2020)

Co-occurrence of keywords

The keyword co-occurrence analysis was based on author keyword analysis. The threshold of at least ten occurrences was set (Ahmad et al., 2021). From the 3,334 keywords, 84 keywords were processed. Keywords with higher frequency are a statement of popularity in HEIs e-learning during COVID-19. The highest word occurrence was COVID-19 (647, TLS = 1153), followed by online learning (253, TLS = 4476) and higher education (196, TLS = 387). Table 5 present the top 10 highest frequencies of the co-occurrence of keyword analysis.

Table 5

Top 10 keywords in the co-occurrence of keywords analysis.

Ranking	Keyword	Occurrences	Total link strength
1.	Covid-19	647	1153
2.	Online learning	253	476
3.	Higher education	196	387
4.	e-learning	187	357
5.	Distance learning	164	322
6.	Pandemic	107	228
7.	Covid-19 pandemic	105	175
8.	Online education	99	167
9.	Distance learning/self-instruction	58	203
10.	Education	57	123
11.	Blended learning	54	100

Figure 2 visualize the co-occurrence of keyword analysis. From the mapping, it can be concluded that there are five main clusters pertaining to e-learning in HEIs. The following are the description of the five clusters in the co-occurrence of keyword analysis:

Figure 2

Co-occurrence of keywords.

Cluster 1 (red) consists of 21 keywords. This cluster represents the theme "e-learning and pandemic" The main keywords are distance learning, e-learning, and pandemic. These keywords are relatable in the context of adapting to e-learning in times of pandemics. There is a distinguishable difference between e-learning pre-COVID-19 with post-e-learning adoption. The latter is characterized by "emergency remote teaching" by the current state of being delivered in pressing circumstances (Hodges et al., 2020).

Cluster 2 (green) present 19 keywords. It is labelled as "medical education and pandemic." Representative keywords include online education, education, medical education, and pharmacy education. This cluster is related to medical and health education in the wake of COVID-19. The requirement to do hands-on and clinical practice limits the advance in medical and health education. Even though delivering a curriculum for medical education is not as practical, it was suggested that novel online teaching methods must be implemented in the new medical curriculum, including methods to deliver online practical skills (Sandhu and de Wolf 2020).

Cluster 3 (blue) comprises 17 keywords with COVID-19, online learning, mental health, depression, anxiety, and stress. This cluster is labelled as "mental health among students." The mental health issue has become a prevalent topic as e-learning has several limitations compared to a physical class. The consequence of prolonged stress will manifest into severe psychological pain and suffering (Araú jo et al., 2020). In the same vein, scholars have provided many suggestions on dealing with online classes to reduce mental health issues (Gewin, 2020).

Cluster 4 (yellow) with 16 keywords dealt with “student self and independent learning." The main identified keywords are curriculum, internet/web-based learning, distance learning/self-instruction, multimedia-based learning, and computer-based learning. The terms used can vary (online learning, computer-mediated learning, blended learning), but this cluster's common theme is the ability to use computer-assisted technology connected to an internet network (Dhawan 2020). It provides the flexibility for students to learn without any geographical restrictions at any time and mean.

Cluster 5 (purple) consists of 11 keywords is labelled as “operationalization of e-learning." The related keywords are higher education, flipped classroom, blended learning, online teaching, Moodle, and online assessment. During COVID-19, HEIs and faculty are required to plan and strategize on practical e-learning sessions. These include the implementation of blended learning (Singh, 2021), flipped classrooms (Tang et al., 2020), and a systematic online assessment (Garcí a-Pe ñ alvo et al., 2021).

Among the five clusters analyzed, clusters 1, 2, 3, and 5 are closely inter-connected clusters, as shown in the figure. While cluster 4, particularly related to curriculum and internet/web-based learning, is considered a peripheral cluster due to its loose connection with the other four clusters. It is established that the pertinent topic and research area related to e-learning and pandemic lead to how students and instructors adapt to e-learning. Apart from that, the challenges and barriers to adopting e-learning have also become the topic of interest, especially to courses that require hands-on and practical face-to-face sessions like medical and health-science-related education. Furthermore, the issue of mental health and individual psychological state has also been related to e-learning as students are being isolated from meeting peers and instructors. Another mainstream topic is the operational issue in e-learning implementation that corresponds to the flipped classroom, blended learning, and online assessment in e-learning.

Discussion

In a short span of two years, 1496 related publications related to e-learning have been retrieved from the highly considerable search strategy implemented. Studies on e-learning in the state of pandemic emergency have only been rigorously studied since the worldwide outbreak in February of 2020. In this review, a bibliometric analysis was conducted on e-learning adoption in the context of HEIs that provides a detailed overview of its adoption during the COVID-19 pandemic, its dynamic literature, and intellectual structure. The present work determined that the publication trend and researchers' interest in e-learning will continue to rise, even after the COVID-19 subside. Understanding the research stream of e-learning for the past two years and the current central publications in this area would provide significant growth in e-learning for many years to come as the topic of e-learning would be very much relevant. To the best of the author's knowledge, this is the first study reviewing the literature on e-learning in HEIs context concerning COVID-19 pandemic through bibliometric analysis.

Despite being in an emergency or even in other delicate circumstances such as war or natural disaster, education is considered fundamental for human needs. E-learning has provided the ease of education and interaction remotely for students, instructors, and faculty management. Our investigations present a colorful palette of publications from authors and various topics and issues in e-learning during COVID-19. The observation offers valuable understanding and insights on the current state of e-learning research and specifies future directions such as pedagogical adaptation post-COVID-19. This review additionally extends the conventional technology adoption popularized in the early 2000s (Venkatesh et al., 2003) and presents the full implementation of e-learning in HEIs.

Among the issue of concern is the mental health state of students. The topic is currently being studied from various perspectives with mitigation and intervention plan. From the citation analysis, five of the top ten publications are related to this issue (Wang and Zhao, 2020, Almaiah et al., 2020, Adedoyin and Soykan, 2020; Kapasia et al., 2020; Mukhtar et al. (2020)). Similarly, cluster number 5 was labelled as student mental health in co-citation analysis, represented by the following publications (Cao et al., 2020; Sahu 2020; Aristovnik et al., 2020; Odriozola-Gonzá lez et al., 2020; Elmer et al., 2020). Meanwhile, in the co-occurrence of keyword analysis, cluster 3 exhibits the mental health cluster through related keywords (COVID-19, online learning, mental health, depression, anxiety, and stress). The mental health issue arises as many students and instructors are at high risk of developing it. Late graduation as examinations are being postponed and the thought of facing the job market due to global recession are among the issues students face (Sahu, 2020).

Another concern in HEIs e-learning adoption is the method of e-learning delivery. Instructors can adopt an interactive teaching method for students to engage better and improvise lecture delivery to attract attention. Every method in the HEIs through the lecture, digital open books, and examination should be transformed to ensure students can cope without compromising the quality of the education. Despite the innovative and interactive teaching approach, the online mode is discriminatory to underprivileged and poor students (Kapasia et al., 2020). Furthermore, students of special needs, such as the hearing impaired, are deprived of the chance for fair education, as they require tactile interaction with the instructors (Manzoor, 2020).

Specific majors and courses, particularly medical education that requires physical classes and interaction, are badly affected in this unprecedented time. New insights and innovative approaches for medical education must be adopted and familiarized, such as the one implemented by Imperial College London on the final year medical online examination (Sandhu and de Wolf, 2020). Other medical schools have carried out open book examinations to ensure students’ education continuity. Such a method should be highly considered to ensure clinical and practical skills can be delivered remotely to equip graduates with the possibility of achieving the same education outcome during clinical placements. The pandemic has established the advancement in medical education through the transformational and innovation of active curriculum for the many disciplines in medicine (Rose, 2020).

Implication

This study provides a plethora of research implications, in terms of theory and practical from the bibliometric analysis based on citation analysis, co-citation, and co-occurrence of keywords analysis.

Theoretical implications

Theories and model-based technology adoption are significantly important in understanding user adoption of e-learning in the context of an emergency, as in the COVID-19 pandemic. Previous technology adoption theories include the theory of reasoned action (Fishbein and Ajzen, 1975), theory of planned behavior (Ajzen, 1991), technology acceptance model (Davis, 1989), and the unified theory of acceptance and use of technology (UTAUT1 and UTAUT2) (Venkatesh et al., 2003). The pandemic presents an unprecedented and complex situation that forced students and instructors to adapt to abrupt education changes. Venkatesh's UTAUT2 might have been the latest theory, but given the current circumstances, the current theory might not explain the holistic understanding in the current context of assessing user adoption to technology, especially senior instructors. Pre-COVID-19, students and instructors have the option to choose e-learning or remain with the conventional face-to-face classes. However, since COVID-19, users are forced to use e-learning without any other option. This theoretical implication from the lenses of emergency adoption may include other critical variables such as fear of COVID-19 scale (Ahorsu et al., 2020) that can provide a revive understanding of how people are adapting or forced to adapt to the e-learning platform.

Another perspective that would provide different theoretical implications is from the view of Delone & Mclean Information System (IS) associated with IS quality and e-learning implementation success, especially among HEIs students. The model would provide the lens on the acknowledged system quality as it measures the ease of use, response time, usefulness, availability, flexibility, and reliability (Shahzad et al., 2021). The unprecedented COVID-19 pandemic has caught the HEIs management by surprise. The administrator, instructors, and students are unprepared to face sudden shifts from physical classes to online classes. This drastic change would have caused inadequate facilities and equipment preparation in conducting full-fledged e-learning, without the need to meet physically. Furthermore, students' and instructors' mental and emotional readiness were caught off guard with demands and pressure to accomplish e-learning with shortcomings in facilities quality dimensions.

Practical implications

This bibliometric analysis on the adoption of e-learning in HEIs has contributed to the practical implications twofold. Firstly, this paper has clearly and holistically provided the structure of current e-learning issues and research stream in HEIs, investigating the evolution of e-learning and predicting future trends of publications in a short span of 2 years. The findings would provide the basis for scholars to quickly identify relevant target publications, the current state of research and relinquish the possible research work in e-learning post-COVID-19. Moreover, this bibliometric review can be utilized as a guidebook for scholars when deciding to submit their articles to journals as stipulated by the studies presented.

Secondly, the critical issue conferred is on the mental health among students and instructors. Students are struggling to connect to classes, especially those who are underprivileged and coming from a poor background. Many issues include internet connection and lack of proper device studying from home. From the perspective of instructors, faculty members, and staff who are already struggling to balance work-life in teaching, research work, and other obligations in university are facing many burdens since the pandemic started. Faculty members have to deliver classes from home and prepare all the teaching materials without technical support (Rapanta et al., 2020; Hodges et al., 2020). Furthermore, the issue of retrenched and lay-off in the workforce has left the low- and middle-income families on job security and its impact on the labor market (Almeida and Santos, 2020). Therefore, HEIs have to balance between ensuring well-balanced education in the time of COVID-19 while at the same time not burdening students and faculty members with the troublesome and unnecessary workload.

Limitation and future work

This review has its fair share of limitations. The articles included in this review were restricted to only journal publications. The reason to include only journal publications is to ensure the findings are of high quality and standard, despite that there could be relevant studies published in books, book chapters, and conference proceedings. Since the total publications (journals only) were 1496 articles, it is justifiable that the number of publications was large enough and acceptable. Hence, the quality and quantity of articles analyzed in this bibliometric study are dependable to meet the objectives stipulated. Other studies interested in a specific topic that may result in a lower volume of publications might include all types other than journals for an inclusive and broad bibliometric analysis.

We only utilized articles indexed in the WOS database in the present study. Other databases might have produced different results, such as Scopus and PubMed, that might include more indexed publications (Mongeon and Paul-Hus 2016). Despite this limitation, it is believed that the decision to adopt only the WOS database had limited the number of publications indexed in reputable journals in terms of quality over quantity. Furthermore, most of the studies related to the COVID-19 refer to WOS as the primary source of database (Ho et al., 2021). Limiting only the WOS database is better to filter only relevant publications related to e-learning in HEIs during the COVID-19 outbreak.

Another limitation of this study is the author's subjective judgment in classifying the themes and labeling the publication's content, which might have led to biases or tend to classify based only on the author's understanding. The limited keywords in specific publications might not capture the actual content of the articles in the co-occurrence of keyword analysis. The clusters were labelled based on the most significant keywords, which the author had qualitatively analyzed based on individual articles. Specific keywords that are missing due to their low occurrences, at the same time, might be of high importance to a particular cluster. Due to the dynamic analysis of the VOSviewer software, a specific cluster might have been missed, thus was not analyzed.

Conclusion

1496 publications related to e-learning in HEIs during COVID-19 were retrieved from the WOS database. Findings depicted that e-learning studies during COVID-19 have proliferated in a short span of two years. HEIs must continue their educational activities, and students must not be penalized for education rights regardless of the COVID-19 outbreak. This study is precious to researchers and scholars in HEIs education on the current state of e-learning adoption and the future of education even though COVID-19 would be receded and becoming endemic. Findings also contribute to the e-learning literature by identifying the knowledge network and conceptual structure and, most importantly, predicting the future direction of e-learning adoption in HEIs. This current bibliometric analysis through citation, co-citation, and co-occurrence of keyword analysis serve as the major contribution for the trends and future works in student adaptation of e-learning in HEIs. The outcome fills the questions gap on 1) central articles, 2) centralization of e-learning studies, and 3) evolution of e-learning. Fundamentally, this study shed light on the importance of e-learning in HEIs in particular and in the broader context of education, especially on the younger generation.

Declarations

Author contribution statement

MUHAMMAD ASHRAF Fauzi: All authors listed have significantly contributed to the development and the writing of this article.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Field

Search string

“Topic”

(("blended learning" OR ″b-learning" OR "blearning" OR "online learning" OR "online education" OR moocs OR "massive open online courses" OR “m-learning” OR "mobile learning" OR "mlearning" OR "virtual learning" OR "web-based learning" OR "digital learning" OR moodle OR ″e-learning" OR "elearning" OR "electronic learning" OR "internet learning" OR "distributed learning" OR "network∗ learning" OR "tele-learning" OR "computer assisted learning" OR "web-based learning" OR "distance learning" OR "learning management system" OR "computer-based learning" OR "interactive learning" OR "learning management system " OR "adaptive learning" OR "electronic assessment" OR ″e-assessment" OR "eassessment" OR "interactive learning" OR "web-based learning" OR "digital learning" OR "computer-assisted instruction" OR "web-based learning" OR "internet-based learning" OR "multi-media learning" OR "technology-enhanced learning" OR "distributed learning" OR "virtual patients" OR "virtual microscopy" OR "virtual environment" OR "virtual learning") AND ("coronavirus 2019″ OR "covid-19″ OR "sars-cov-2″ OR "sars-cov2″ OR "covid19″ OR "covid 2019″ OR ″2019-ncov") AND ("higher education institution" OR "universit∗" OR "college∗" OR "institution of higher learning" OR "faculty")