Academia's responses to crisis: A bibliometric analysis of literature on online learning in higher education during COVID-19.

This paper aimed to provide a holistic view of research that investigated online learning in higher education around the globe during COVID-19 utilizing a bibliometric analysis. The researchers used co-citation analysis and text mining afforded by VOSviewer to document and analyze research patterns and topics reported in peer-reviewed documents published between January 2020 and August 2021. Findings of this study indicated that scholars from 103 countries or regions from the Global North and Global South investigated a wide array of topics, such as use of various technologies and strategies, redesigned curriculum, student perceptions and psychological impacts of the pandemic-imposed online learning. Many researchers applied technology acceptance theories and structural equation modeling to investigate factors associated with adoption and impacts of the pandemic-imposed online learning. Of the large quantity of research, medical education and chemical education were the most investigated disciplines. Inquiry-based learning, discovery learning, hands-on learning and collaborative learning emerged as instructional approaches frequently discussed or utilized across the target studies. This paper discussed (a) ongoing and emerging challenges to online higher education, (b) placing innovative pedagogies at the forefront of online learning, and (c) rapid, but imbalanced distribution of evolving literature based on the findings.

INTRODUCTION

Higher Education Institutions (HEI) have experienced a steady growth of online learning over the past decade. For example, the number of degree‐seeking students in 2012 in the United States totaled 4,559,494 (Seaman et al., 2018). By 2019, that number had expanded to 7,313,623 students enrolled in online learning classes or programs at degree‐seeking institutions (National Center for Education Statistics, 2020). Because of its broad concept, online learning takes multifaceted forms across HEIs and has a range of synonyms and meanings attached to it (eg, distance learning, e‐learning and remote learning; Moore et al., 2011; Singh & Thurman, 2019). In this article, we refer to the definition from Singh and Thurman (2019) that captures the core elements of online learning. Drawing upon a systematic literature review and content analysis of the definitions of online learning, Singh and Thurman define the concept as education delivered in an environment through the use of the Internet for teaching content to enhance synchronous or asynchronous learning activities as well as learning that is not dependent on student physical or virtual location.

Well‐designed online learning in HEIs has such potential benefits as accessibility, affordability and flexibility (Castro & Tumibay, 2021; Dhawan, 2020), which may have served as a contributing factor for the increasing enrollment in online classes and programs across HEIs. In the meantime, research exploring what constitutes well‐designed online learning and its effectiveness is accumulating (eg, Castro & Tumibay, 2021). This growing body of research focused on different aspects, such as instructional design strategies (eg, learner‐led, instructor‐led), student populations (eg, undergraduate, graduate, certificate seekers), and desired outcomes (eg, course completion, academic performance, motivation). For example, extensive research has verified the importance of self‐regulated learning skills in ensuring student achievement in undergraduate and graduate online courses (eg, Artino & Jones, 2012; Broadbent & Poon, 2015; You, 2016) and strategies that foster online social interactions among faculty and students within the framework of community of inquiry (CoI; Garrison & Arbaugh, 2007). More recent studies found that providing job‐relevant online content with customizable local support improved completion rates of professional degree and certificate programs (Littenberg‐Tobias & Reich, 2020) and that providing automatic feedback increased student performance and reduced instructor efforts in grading large‐scale online courses (Cavalcanti et al., 2021).

Recently, HEIs across all continents were forced to transition to online learning in response to the COVID‐19 pandemic (Xie et al., 2021). Due to the unexpected disruption, educators and students in almost all educational settings including K‐12 and higher education have had first‐hand experiences of online teaching and learning. The accelerated transition to online learning prompted scholars to quickly investigate factors regarding the design, delivery and evaluation of pandemic‐imposed teaching and learning. These quick responses by the global research and practice communities help generate a large corpus of knowledge about online learning. The purpose of this study is to conduct a bibliometric systematic review to explore the body of emerging knowledge, with a focus on uncovering research and practice trends in the pandemic‐imposed online higher education.

Extant literature reviews

Research has emerged to document and reflect on how different entities (eg, countries, universities, school districts) responded to the pandemic‐imposed online learning. For example, Crompton and colleagues (2021) reviewed studies on K‐12 online learning across 48 countries during emergencies, such as human‐caused and natural disasters. The researchers found that 57 out of 60 reviewed studies focused on student learning during the COVID‐19 pandemic and identified several strategies being used during remote learning, including (1) finding multiple ways to communicate with students and parents (eg, online survey, FaceTime, text messages), (2) using different technologies to deliver instructions (eg, Zoom, Skype, Google Suite, radio, television) and (3) partnering with community organizations or companies for resources. This review provided comprehensive understandings of what technologies and strategies have been used to address emergency‐related challenges and opportunities in K‐12 educational settings.

In addition, several literature reviews have emerged that analyzed research investigating how online teaching and learning in higher education around the globe was addressed during the COVID‐19 pandemic. Abu Talib et al. (2021) reviewed 47 studies from high‐quality, prestigious journals that examined how the transition from in‐person to online education impacted academia and students. Findings from this review suggest that the rapid transition has negative impacts on students' mental health. Reliance on recorded lectures, fatigue from prolonged staring at screens and lack of personal contact may have resulted in decreased engagement among college students. Despite these challenges, the authors argued that being forced into online learning might have served as an impetus for innovative teaching approaches, integration of modern technology and lower cost of education.

Carrillo and Flores (2020) analyzed 134 empirical studies published from 2000 to 2020 on online teaching and learning practices in teacher education, specifically in relation to social, cognitive and teaching presence within the CoI framework. Although not all reviewed studies specifically addressed the COVID‐19, the authors discussed the implications of their findings in the context of the pandemic. Another study analyzed 26 empirical studies published from February 2020 to October 2020 that explored the role of educational technologies during HEIs' transition to online learning (Turnbull et al., 2021). The authors applied the framework of Technological Pedagogical Content Knowledge (TPACK) to analyzing pedagogy, content and technology emerging from the literature. According to their findings, frequently utilized technologies during the transition include Zoom, Learning Management System (LMS) and social media. Moreover, the authors identified five challenges in relation to technologies, including access to technology, synchronous/asynchronous learning tool integration, faculty and student online competence, privacy and confidentiality, and academic dishonesty.

These extant reviews provide valuable information on specific technologies, impacts or challenges related to the pandemic‐imposed online higher education from different aspects. However, research and practices are evolving rapidly in response to the ongoing pandemic. This suggests a need to update understandings of the topic. Additionally, the previous reviews only focused on articles published in high‐impact journals, empirical studies or research conducted prior to the pandemic. Other research and practices conducted by a wider community of researchers and practitioners might have been left unexamined; thus, there is a lack of a comprehensive map illustrating HEIs' rapid and ongoing responses to the crisis.

Rodrigues et al. (2020) initiated an effort to map the scientific literature on the COVID‐19 disruption to the areas of education and management using a bibliometric mapping and analysis technique. Three distinct themes emerged from the literature map, including (1) COVID‐19 and online education, (2) COVID‐19 from a management perspective and (3) COVID‐19 in Canada. However, this work was not specifically focused on HEIs. Therefore, there is still a lack of research that comprehensively analyzes scientific knowledge about online higher education generated by the global research and practice communities. To fill the gap, the present review applies the bibliometric methodology to map academia's responses to the crisis by identifying patterns and themes in research and practices in online higher education.

Purpose of the study

This paper aims to provide a holistic view of research that investigated the pandemic‐imposed online learning in higher education around the globe from January 2020 to August 2021. First, we investigate how HEIs responded to the sudden shift to online teaching and learning drawing upon bibliometrics of research articles published during the pandemic. Second, the focus of this review is to identify research themes emerging from the literature. Specifically, the following research questions guide our research efforts:

What is the basic bibliometrics of research investigating the pandemic‐imposed online learning in higher education around the globe? To uncover an intellectual structure of emerging research, we focus on the following basic bibliometrics: (a) geographical distribution of publications, (b) most co‐cited references and (c) journal with most publications. Analysis of geographic distribution helps reveal sources of scholarly participation and document the breadth of knowledge across the globe (Hallinger & Kovačević, 2019). Metrics on most co‐cited references and journals with most publications are used to identify influential research topics and the state‐of‐the‐art of a scientific field (Zupic & Čater, 2015).

What themes have emerged from the literature investigating the pandemic‐imposed online learning in higher education around the globe?

METHOD

We utilized bibliometric analysis methodology to address the research questions. This methodology allows researchers to analyze topics in a research field, trends of the topics and interrelationships among these topics within a large corpus of literature (Ellegaard & Wallin, 2015; Murphy et al., 2007). A bibliometric analysis draws upon descriptive publication data on authors, institutions, journals, keywords, disciplines and citations to generate networking knowledge maps within a research field as well as advanced text mining techniques to identify research themes and promising future research directions (Ziegler, 2009). In recent years, bibliometric analysis has gained traction as an approach to reviewing educational research (eg, Fellnhofer, 2019; Hallinger & Kovačević, 2019). In this review, we used VOSviewer, a tool for constructing and visualizing bibliometric networks (van Eck et al., 2010a), to analyze the bibliometric data. We used co‐citation analysis and text mining afforded by VOSviewer to document and analyze research patterns and topics reported in peer‐reviewed publications about online higher education during the pandemic.

Database and search procedures

Web of Science (WoS) was used as the database for this literature review. WoS is considered as the largest multidisciplinary database of scientific literature and is widely used by researchers around the globe (Li et al., 2019). We followed the Preferred Reporting Items for Systematic Reviews and Meta‐Analysis (PRISMA; Liberati et al., 2009) to search for and screen documents (see Figure 1). We applied a Boolean search that combines three parts of the search string: (1) online learning, (2) higher education, and (3) COVID‐19. A wide range of keywords or phrases were used to identify more documents related to online higher education during COVID‐19 (see Table S1 for detailed search terms).

FIGURE 1

PRISMA flowchart: data search and screening process

We set the time frame starting from January 2020 to August 2021. The initial search generated 1658 documents. To ensure the quality of included documents, we set peer‐reviewed journal articles or conference proceedings as a search parameter. Additionally, we only included documents published in English. After applying the two parameters, the process yielded a total of 1394 articles, which include 931 open‐access articles, 216 early‐access articles, 246 published in traditional journals and one conference proceeding. There are several possible explanations for the surprisingly small number of proceedings. First, multiple conferences were cancelled during the pandemic. Second, even if a conference went online, articles accepted to the conference might have reported studies conducted before the pandemic given that it usually takes several months from submitting a proposal to being accepted. Third, WoS selectively covers conference proceedings from high‐impact, high‐frequency science journals and a great majority of these are from biomedical subject areas (Clarivate, 2021).

Inclusion and exclusion criteria

To begin, we scanned the title and abstract of 1394 documents to determine their eligibility for inclusion. In order to generate a comprehensive map of the literature, we included (1) documents that reported on empirical studies using data collected from experiments, observations or surveys, (2) documents introducing specific technologies or strategies used to address the pandemic‐imposed online learning in higher education (or related terms; see Table S1) and (3) documents describing faculty's experiences of redesigning courses, using technology or applying instructional practices as responses to the sudden transition to online. Given that the purpose of this study is to generate a holistic view of quick responses and solutions to the pandemic‐imposed online learning, we did not limit research to only empirical studies. However, a document was excluded if (1) it did not emphasize online higher education during the pandemic; (2) it only explored the impact of COVID‐19 rather than online learning during COVID‐19 on participants' mental health or other psychological states or (3) it was a conceptual piece, editorial, philosophical paper, literature review or commentary. The screening process removed 333 records, which resulted in a final database of 1061 documents for further analysis (see Figure 1).

Data analysis

Bibliometric data of included documents, including information on author name, affiliation, title, keywords, abstracts and cited references, were downloaded from WoS and imported into the free software VOSviewer for analysis. First, a descriptive statistical analysis of the number of documents by country and by journals was performed using VOSviewer. When working with WoS files, VOSviewer processes data on origins of documents based on the first author's information. The results regarding origins of documents were illustrated through a heat map of geographical distribution of publications (see Figure 2). The results regarding journals with most publications were shown in Table 1.

FIGURE 2

Geographical distribution of publications

TABLE 1

Journal citation report™ rank of top 10 journals with most publications/papers

Rank	Journal	Number of publications/papers	Publication frequency	Open access	JCI	Journal citation report™ (based on JCI in 2020)
						Category (SSCI/SCIE/ESCI)	Rank (Quartile)
1	Journal of Chemical Education (JCE)	134	12 issues/year	No	0.94	Chemistry, Multidisciplinary (SCIE)	49/219 (Q1)
						Education, Scientific Disciplines (SCIE)	24/79 (Q2)
2	Sustainability	51	24 issues/year	Yes	0.56	Environmental Sciences (SCIE)	163/306 (Q3)
						Green & Sustainable Science & Technology (SSCI, SCIE)	38/66 (Q3)
						Environmental Studies (SSCI)	111/156 (Q3)
3	Education and Information Technologies (EIT)	35	6 issues/year	No	1.82	Education & Educational Research (SSCI)	82/724 (Q1)
4	Education Sciences	28	4 issues/year	Yes	1.03	Education & Educational Research (ESCI)	250/724 (Q2)
5	Journal of Microbiology & Biology Education	18	3 issues/year	Yes	0.41	Education, Scientific Disciplines (ESCI)	52/79 (Q3)
6	BMC Medical Education	14	1 issue/year	Yes	1.28	Education & Educational Research (SSCI)	174/724 (Q1)
						Education, Scientific Disciplines (SCIE)	14/79 (Q1)
7	Ecology and Evolution	14	24 issues/year	Yes	0.71	Evolutionary Biology (SCIE)	30/52 (Q3)
						Ecology (SCIE)	84/178 (Q2)
8	Interactive Technology and Smart Education	14	4 issues/year	No	1.02	Education & Educational Research (ESCI)	254/724 (Q2)
9	Pharmacy Education	14	4 issues/year	Yes	0.22	Education, Scientific Disciplines (ESCI)	66/79 (Q4)
10	Revista Romaneasca Pentru Educatie Multidimensionala	14	3 issues/year	Yes	0.70	Education & Educational Research (ESCI)	350/724 (Q2)

Abbreviations: ESCI, emerging sources citation index; JCI, journal citation indicator; Q, quartile; SCIE, science citation index expanded; SSCI, social sciences citation index.

Second, we identified prominent journals and documents using co‐citation analyses. In bibliometrics, citation analysis and co‐citation analysis techniques are frequently employed to identify prominent units, such as author, reference/document, and source/journal, within a scientific domain (Fellnhofer, 2019). Citation analysis measures the extent to which a unit was cited by documents included in the reviewed database. Heavily cited documents usually made significant contributions to advancing knowledge within a given scientific network (Hood & Wilson, 2001). As a variant of citation analysis, co‐citation analysis examines the frequency with which two units are co‐cited together by the same document (ie, citing document) in the reviewed database; the analysis is based on inclusion of the co‐cited units in the reference list of the citing document (Hallinger & Kovačević, 2019). Compared to citation analysis, co‐citation analysis provides a more reliable and broader measure of scientific contributors and contributions (Fellnhofer, 2019; Hallinger & Kovačević, 2019). Thus, we employed co‐citation analysis to identify prominent documents with most co‐citations in our database (see Table 2).

TABLE 2

Rank order of 20 most highly co‐cited documents in the database

Rank	Document	Geographical Divide (Country)	Content or Focus	Co‐citations
1	Bao (2020). COVID‐19 and online teaching in higher education: A case study of Peking University	Global South (China)	Empirical Study: Six specific instructional strategies	59
2	Hodges et al. (2020). The difference between emergency remote teaching and online learning	Global North (United States)	Review/Conceptual Article: Various terms of online learning	58
3	Fornell and Larcker (1981). Evaluating structural equation models with unobservable variables and measurement error	Global North (United States)	Methodology Article: Structural Equation Modeling	53
4	Sahu (2020). Closure of universities due to coronavirus disease 2019 (COVID‐19): impact on education and mental health of students and academic staff	Global South (Trinidad & Tobago)	Review/Conceptual Article: Potential impact on education and mental health of students and academic Staff	50
5	Cao et al. (2020). The psychological impact of the COVID‐19 epidemic on college students in China	Global South (China)	Survey Study: Psychological impact of the COVID‐19	48
6	Davis (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology	Global North (United States)	Technology Acceptance Model	48
7	Crawford et al. (2020). COVID‐19:20 countries' higher education intra‐period digital pedagogy responses	Global North (Australia)	Content Analysis: HEIs' responses to COVID‐19	41
8	Venkatesh et al. (2003). User acceptance of information technology: Toward a unified view	Global North (United States)	Technology Acceptance Model	40
9	Braun and Clarke (2006). Using thematic analysis in psychology	Global North (New Zealand)	Methodology Article: Thematic Analysis	37
10	Murphy (2020). COVID‐19 and emergency eLearning: Consequences of the securitization of higher education for post‐pandemic pedagogy	Global North (Canada)	Content Analysis: Policies on Emergency e‐Learning	37
11	Hair et al. (2010). Multivariate data analysis: A global perspective	Global North (United States)	Methodology Book: Multivariate Regression	33
12	Dhawan (2020). Online learning: A panacea in the time of COVID‐19 crisis	Global South (India)	Content Analysis/SWOC: e‐Learning Modes	32
13	Rapanta et al. (2020). Online university teaching during and after the Covid‐19 crisis: Refocusing teacher presence and learning activity	Global North (Portugal)	Empirical Study: Investigating expert insights into online teacher presence	32
14	Adnan and Anwar (2020). Online learning amid the COVID‐19 pandemic: Students' perspectives	Global South (Pakistan)	Empirical Study: Pakistani student perspectives of online learning	30
15	Freeman et al. (2014). Active learning increases student performance in science, engineering, and mathematics	Global North (United States)	Literature Review: Active Learning in STEM	30
16	Toquero (2020). Challenges and opportunities for higher education amid the COVID‐19 pandemic: The Philippine context	Global South (Philippines)	Review/Conceptual Article: Opportunities & Challenges	29
17	UNESCO (2020). COVID‐19 Educational disruption and responses	Global North (France)	Statistics: COVID‐19 impact on education	26
18	Moore et al. (2011). e‐Learning, online learning, and distance learning environments: Are they the same?	Global North (United States)	Literature Review: Differences in e‐Learning, online learning, and distance learning	24
19	Rose (2020). Medical student education in the time of COVID‐19	Global North (United States)	Viewpoint Article: Potential impact of COVID‐19	24
20	Venkatesh and Davis (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies	Global North (United States)	Technology Acceptance Model	24

The geographical divide is based on whether the first author's affiliation is located in Global North or South.

VOSviewer's co‐occurrence mapping functionality, which leverages text mining techniques, was applied to analyze connectedness among publications based on keywords and terms that appeared in the publications. The stronger publications are connected, the closer publications are located near one another on a networking map, and the more concepts are interconnected (van Eck et al., 2010b). Networks among publications and clusters generated by VOSviewer allowed us to explore structural features of the literature base and examine themes emerging from it.

RESULTS

A total of 1061 documents that reported various factors in relation to how HEIs around the globe responded to COVID‐19 were found. Researchers from a wide range of disciplines have investigated, designed and/or evaluated the pandemic‐related challenges and opportunities for online higher education. In this section, we present the results of bibliometric characteristics and co‐occurrence analyses to address our research questions.

Bibliometric characteristics of the literature

Geographical distribution of publications

Figure 2 illustrates the geographical distribution of documents published by scholars from 103 countries or regions. The Global North countries or regions accounted for 61.96% (n = 808) of all the included documents, and 38.04% (n = 496) was contributed by scholars from the Global South. The North‐South divide of the world is often considered on account of political, socio‐economic and scientific developments of a country (Confraria et al., 2017). For the purpose of this analysis, we categorized identified countries or regions in two groups following the United Nations' categorization of “developing” economies (Global South) and “developed” economies (Global North; United Nations, 2019). Additionally, the United States, India, China, Spain and Saudi Arabia are the top five countries that produced the most articles investigating the pandemic‐imposed online higher education.

Journals with most publications

Within a relatively short period of time, a large number of journals published articles investigating online higher education during COVID‐19. The journals vary widely in the number of articles published, ranking and publishing model. Table 1 lists the top 10 journals in terms of the number of articles. Among all journals, Journal of Chemical Education (JCE), Sustainability, and Education and Information Technologies (EIT) produced the most articles. We adopted publisher‐neutral data and statistics provided by Journal Citation Reports (JCR) to measure the scientific influence of scholarly journals. Two widely used JCR metrics include Journal Impact Factor (JIF) that measures journal‐level metric calculated from WoS‐indexed data and Journal Citation Indicator (JCI) that measures average category normalized citation impact of citable publications by a journal over the recent three years (Clarivate, 2021). We used JCI as the main metric given that not all journals have a JIF index. Additionally, JCR organizes journals into different categories based on discipline areas, and there are journals that have multiple categories because of their interdisciplinary nature. Further, JCR provides journal ranking data based on JCI within a specific category.

Of the top ten journals, eight have an education‐related category. Two education‐related journals with social science citation index fall under the first quartile (Q1; indicating top 25% of journals in the field of education and educational research), and four are Q2 journals. Moreover, seven journals are fully open‐access journals, which publish research articles that are peer‐reviewed, online and freely available to readers. Three journals publish more than 10 issues per year. Some of these journals published a relatively large number of articles per issue. For example, JCE publishes 30–40 articles per issue, and BMC Medical Education publishes more than 500 articles per issue.

Most co‐cited documents

Using co‐citation analysis, we identified the most co‐cited documents across the included publications to evaluate scholarly impact and patterns. Table 2 lists the top 20 documents that were most co‐cited in our database. The highly co‐cited documents span the period from 1981 to 2020. Of the top 20 documents, 12 were published in 2020, such as Bao (2020), Sahu (2020) and Cao et al. (2020). Other co‐cited documents prior to the year of 2020 were methodological or conceptual papers that informed the reviewed papers on online higher education from January 2020 to August 2021. To further analyze the geographic distribution, 70% of these co‐cited documents were produced by the first author, whose affiliation was located in the Global North, and 30% in the Global South.

Documents on strategies and mental health

The most co‐cited article (Bao, 2020) reported six specific instructional strategies for improving student engagement based on observations of Peking University's online education during COVID‐19. These strategies include making emergency plans for unexpected problems, chunking content into smaller units, an article investigating the effectiveness of active learning on improving performance of STEM students in higher education prior to the pandemic (Freeman et al., 2014) was co‐cited 30 times across the reviewed publications. Three more highly co‐cited empirical studies reported the psychological impact of the pandemic on Chinese college students (Cao et al., 2020), expert insights into pedagogical content knowledge related to online learning (Rapanta et al., 2020), and Pakistani college students' attitudes towards distance learning courses (Adnan & Anwar, 2020).

Documents on relevant policy and viewpoints

There are three highly cited documents that explored relevant policy that addressed the sudden transition to online learning using content analysis methodology. For example, using university and government sources such as university websites and news articles, Crawford et al. (2020) analyzed how HEIs across 20 countries responded to COVID‐19. The researchers found great variability in crisis‐response measures from having no contingency strategies to rapidly redeveloping curriculum for fully online offerings. Dhawan (2020) conducted a strengths, weaknesses, opportunities and challenges (SWOC) analysis of e‐learning modes as well as described the growth of EdTech startups during the pandemic using secondary sources of data such as journal articles, reports and company websites. In another document analysis, Murphy (2020) analyzed shared characteristics in 25 declarations of emergency e‐learning at American universities based on securitization theory. Murphy found universities' responses to emergency e‐learning as a securitization of face‐to‐face schooling and increasing efforts to normalize e‐learning. It was argued that normalizing emergency e‐learning might result in normalizing “a form of education that perpetuates structural inequalities of class, race, and support” (p. 501). Thus, the author posited that the importance of desecurizing face‐to‐face schooling after COVID‐19 for the future possibility of emancipatory pedagogy, be it face‐to‐face or online.

In addition, three viewpoint or review articles were highly cited (ie, Rose, 2020; Sahu, 2020; Toquero, 2020). These articles discussed potential impacts of the pandemic on mental health of students and academic staff (Sahu, 2020) as well as on pre‐clinic curricula and learning environments for medical students (Rose, 2020). Toquero (2020) described opportunities for HEIs to respond to pandemic‐related challenges, such as integrating environment and health courses into the curriculum, scaling up professional training in online instruction, and strengthening research efforts and evidence‐based practices. In addition, researchers cited two articles that distinguished various terms, including the difference between emergency remote teaching and online learning (Hodges et al., 2020) as well as the variations in e‐learning, online learning and distance learning (Moore et al., 2011). Lastly, UNESCO's website providing data on school closures around the globe (UNESCO, 2020) was widely cited.

Documents on methodology

Aligned to research efforts, three articles or books discussing research or data analysis methods were frequently cited, including structural equation models (SEM; Fornell & Larcker, 1981), thematic analysis (Braun & Clarke, 2006) and multivariate data analysis (Hair et al., 2010). In addition, articles that introduced and investigated various technology acceptance models (TAM) and theories were frequently referenced. These models include User Acceptance of Information Technology (UAIT; Davis, 1989), Unified Theory of Acceptance and Use of Technology (UTAUT; Venkatesh et al., 2003), and a theoretical extension of the TAM that integrates social influence and cognitive instrumental processes (Venkatesh & Davis, 2000).

Emerging themes from the literature

Using VOSviewer keyword co‐occurrence analysis, we created a map that visualized similarities among keywords and terms emerging from the literature (see Figure 3). The output of a co‐occurrence analysis displays a network of keywords as a means to identify the topical foci and their relations within the literature that represent the research trends of an emerging knowledge base and intellectual structure of a field (Hallinger & Kovačević, 2019). In this study, we created a co‐occurrence map by using “All Keywords” (ie, keywords in title, index, defined by author) with a threshold of minimum 5 times of occurrences of a keyword. For this study, the threshold was set for the purpose of identifying a wider range of potential topics.

FIGURE 3

Keyword mapping. This graphical map illustrates themes emerging from the literature on the pandemic‐imposed online higher education based on co‐occurrence of keywords that appeared in the same publications with a minimum occurrence threshold of 5

VOSviewer identified 258 keywords for the analysis. We deleted keywords that represent overlap of our search terms for article retrieval, such as COVID‐19, online learning and higher education, as well as other general terms, such as students, education, learning and teaching. Occurrences of general terms carry limited value or have no defined meaning for the analysis (Li et al., 2019). For this study, the deletion of these keywords was to determine nuances of themes regarding online learning in HEIs. See the full list of deleted keywords and co‐occurring keywords in Table 1 and Supplemental Spreadsheet in the Appendix, respectively.

Connection between two keywords that occur together is represented by a link; each link has a strength (ie, the size of a link) that indicates the number of publications in which two terms occur together (van Eck & Waltman, 2020). Terms together with the links between the items constitute a network. Parameters set for link strengths in VOSviewer, such as methods to calculate and normalize link strengths, impact the clustering of co‐occurrence maps. In this study, we used full counting to calculate link strengths and association strength method to normalize link strengths when creating the co‐occurrence map (For more details, see the discussion on association strength and full counting in van Eck & Waltman, [2020]). To illustrate, the term “impact,” which is situated at the center of the map (see Figure 3), has the largest circle and numerous links in different sizes to other terms. In the network visualization, the larger the size of a circle, the more the number of occurrences of a term in all documents (van Eck & Waltman, 2020). This means that “impact” is the most frequently occurring keyword in Cluster 1 (red) and across all documents, highlighting the status of “impact” as a central concept emerging from the research on the pandemic‐imposed learning.

Seven clusters, illustrated through different colours, emerged based on keyword co‐occurrence. Each cluster contains keywords with varying levels of co‐occurrence (see Supplemental Spreadsheet for the frequency of keywords in each cluster). The keywords with high co‐occurrence within each cluster represent the most investigated concepts that constitute an emerging theme. We leveraged the semantic connectedness among the highly co‐occurring concepts and the most co‐cited documents identified in this review to label each cluster. For example, “impact,” “mental health,” “medical education,” “anxiety” and “stress” had the highest frequency in Cluster 1. Drawing upon the most co‐cited documents, mental health, anxiety and stress suggest a research focus on investigating psychological impacts of COVID‐19 on student learning. Thus, we labelled Cluster 1 as Medical Education and Psychological Impact (red; 52 keywords). Following the same procedure, we labelled the other clusters: Cluster 2‐Chemistry Curriculum and Laboratory Instruction (green; 37 keywords), Cluster 3‐TAM (blue; 31 keywords), Cluster 4‐Student Perceptions and Satisfaction (yellow; 24 keywords), Cluster 5‐Student Performance and Design (purple; 23 keywords), Cluster 6‐Classrooms in Distance Learning Settings (cyan; 19 keywords) and Cluster 7‐Support and Community (orange; 17 keywords). Compared to the first four clusters, we observed less prominent themes within Clusters 5, 6 and 7. In the following sections, we describe and discuss the most noticeable topical foci identified from the clusters.

Cluster 1—Medical education and psychological impact

In Cluster 1, keywords such as “medical education,” “dental education,” “anxiety,” “stress,” “depression” and “mental health” co‐occur frequently. These keywords indicate that there was a large amount of medical education literature during COVID‐19. It is possible that this trend features prominently because of the necessity to continue medical training in situ and the accessibility to medical students, trainees and samples to be recruited for a rapid scientific investigation versus other fields. Moreover, the co‐occurring keywords could also suggest that a certain amount of research has explored the impact of COVID‐19 and emergency online learning on student mental health (eg, Guse et al., 2020; Vishwanathan et al., 2021; White et al., 2021). We note that the term “digital divide” has dense links with “anxiety” and “stress.” An example of such a link was found in Shin and Hickey (2021), a survey study revealing that students, more likely females, experienced issues regarding mental health, digital divide, inequity and accessibility during emergency remote learning. To address the digital divide and ensure equity, some research found that faculty highlighted flexibility, reducing coursework to essential content, and personalization as ways to extend culturally sustaining pedagogy to support online learning (eg, Goin Kono & Taylor, 2021).

In addition, “assessment” emerged as a keyword associated with student stress and mental health. A large number of universities used online proctoring systems, such as artificial intelligence‐driven proctoring systems (eg, Jia & He, 2021), when assessing student learning during lockdowns (Raman et al., 2021). Instead of focusing on online tools for summative assessment, however, researchers also explored the importance of “formative assessment” (eg, Sharadgah et al., 2020) and associated tools (eg, Areed et al., 2021) in supporting active online learning. Other technological innovations investigated in Cluster 1 range from “simulation”, “social media” and “telemedicine.” In particular, Facebook and Twitter emerged as the main social media used for “communication” during the quarantine (eg, Greenhow & Galvin, 2020; Zarzycka et al., 2021). Such online communications were highly impacted by participants' “digital literacy” (eg, De la Hoz et al., 2021; Mollenkopf et al., 2020). These findings, consistent to previous research on online learning, point to the importance of establishing a sense of community and providing socio‐emotional support tailored to student needs (eg, Fatani, 2020; Li & Yu, 2020).

Cluster 2—Chemistry curriculum and laboratory‐based instruction

Chemistry education, which includes organic, analytical, physical, applied, bio‐, medical, inorganic and general chemistry, features in this cluster. Other keywords frequently co‐occuring in Cluster 2 include “curriculum”, “laboratory instruction”, “student‐centered learning”, “science”, “professional development” and “spectroscopy”. Illustrated by the emerging terms, researchers have explored curriculum and laboratory instruction, especially for chemistry, from student‐centered instructional approaches when applied to online learning settings. These approaches include inquiry‐based/discovery learning (eg, Spitha et al., 2021), hands‐on learning using manipulatives (eg, Luse & Rursch, 2021), and collaborative/cooperative learning (eg, Serafin & Chabra, 2020). The frequent occurrence of “professional development” and its connection to “curriculum” and varying student‐centered instructional approaches suggest the pressing needs, challenges or opportunities of supporting faculty in providing effective online instruction (eg, Ó Ceallaigh, 2021; Rupnow et al., 2020). Regarding technology, many studies investigated how clickers, videos, YouTube and online platforms supported student learning (eg, Alfayez, 2021; Chiu, 2020).

Cluster 3—TAM

Cluster 3 is mainly composed of terms associated with TAM and determinants of technology adoption. Aligned to previous research, a majority of studies in this cluster focused on examining participants' perceptions, intentions, behaviours and attitudes towards technology use in terms of ease of use and perceived usefulness (eg, Alshurafat et al., 2021; Antee, 2021), gender‐differences (eg, Tang et al., 2021) and quality (eg, Istijanto, 2021). As a prominent methodology used in TAM research, SEM was widely used to explore relationships among these constructs and factors within the context of the pandemic‐imposed online learning. Another key term emerging from this cluster is “developing countries,” revealing the origin of these studies investigating factors associated with participants' adoption of technology during the pandemic.

In addition, multiple terms that were frequently investigated in online learning research emerged in this cluster, such as “self‐regulation”, “mobile learning”, “MOOCs” and “information technology”. These terms have dense links with keywords or terms such as “TAM”, “acceptance” and “TAM” within this cluster as well as keywords such as “perceptions”, “motivation” and “satisfaction” in other clusters. To illustrate the potential connections, a study investigated multiple factors impacting Indian students' adoption of MOOCs (eg, motivation, self‐regulation, perceived usefulness) using TAM as the framework and SEM as the statistical analysis method (Singh & Sharma, 2021). Another study examined relations among mobile learning acceptance constructs, motivation and behavioural intention based on the UTAUT model and SEM (Sitar‐Tăut, 2021).

Cluster 4—Student perceptions and satisfaction

Cluster 4, which consists of co‐occurring keywords “perceptions”, “satisfaction”, “model”, “motivation”, “self‐efficacy”, “readiness”, “social presence”, “student satisfaction” and “student engagement”, is located at the center of the left‐hand side of the map. This cluster also demonstrates wide connections with keywords in other clusters than Cluster 2, which is located on the right‐hand side of the map. In particular, most keywords in this cluster are considered to be participants' perceptions and beliefs towards emergency remote learning, which relate to students' psychological symptoms and well‐being emerging from Cluster 1.

Similar to Cluster 3, researchers tended to apply SEM to explore impacts of student beliefs (eg, self‐efficacy; El‐Sayad et al., 2021), perceived usefulness and ease of technology use (eg, Zaidi et al., 2021), and perceptions towards social presence (eg, Park & Kim, 2020) on engagement and satisfaction with online learning. Additionally, “self‐regulated learning” co‐occurred frequently with other psychological factors from Cluster 4. To explore these connections, some researchers identified different resource‐management strategies applied by students to self‐regulate emergency remote learning (eg, Biwer et al., 2020). Other researchers investigated how self‐regulation skills impacted student well‐being, academic performance, motivations and/or emotions (eg, Pelikan et al., 2021; Reinhold et al., 2021).

Cluster 5, 6 and 7

Co‐occurring keywords in these clusters are dispersed over a wider region of the map, which indicates less connectedness among arising concepts compared to themes identified in other clusters. Nevertheless, multiple technologies and innovations emerged from these clusters. For example, Cluster 5 illustrates that researchers investigated the design and effectiveness of flipped classrooms (Collado‐Valero et al., 2021), augmented reality (eg, Eldokhny & Drwish, 2021; Solmaz et al., 2021) and virtual reality (eg, Akdere et al., 2021) on student performance, especially for chemistry (eg, Dunnagan et al., 2020; Rodriguez et al., 2020). Cluster 6 illustrates that the concept of gamification (eg, Fontana, 2020; Petchamé et al., 2021), as well as digital tools such as Zoom (eg, Ghounane, 2020) and Moodle (Molchanova et al., 2020) were frequently used to support classroom experiences in distance learning settings.

Co‐occurring keywords “community” and “support” in Cluster 7 demonstrate a research focus on investigating ways and tools for building online communities, where students could support each other (eg, Sobaih et al., 2020). Aligned to the focus, researchers used the CoI framework as a theoretical foundation for designing and examining online learning (Orlowski et al., 2021; Tan, 2021). In addition, keywords “barriers” and “equity” co‐occurred frequently from Cluster 7, which have strong links to the keyword “digital divide” in Cluster 1. This reveals the research efforts in investigating equity issues and barriers at the level of technology, individual, family, institution and community (Baticulon et al., 2021) within the pandemic‐imposed online learning environment. For example, lack of internet connectivity (Olum et al., 2020), no access to suitable devices (Bashitialshaaer et al., 2021) and external distraction (Joshi et al., 2021) were identified as important barriers to online learning. To further address barriers, some researchers discussed experience‐based strategies for promoting equity and inclusion during synchronous classes (eg, Castelli & Sarvary, 2021; Iyer & Chapman, 2021).

DISCUSSION

This study reported on a bibliometric analysis of the literature on online higher education during COVID‐19, providing a holistic, bird's‐eye‐view perspective of rapid responses to the disruption around the globe. Because of the comprehensive list of search terms and intentionally broad inclusion criteria, we identified a large amount of research (as reported in 1061 documents) published after January 2020 that investigated online higher education. As a global phenomenon, the sudden transition to online learning caused by the pandemic has attracted unprecedented attention from researchers in various disciplines, which are not limited to education, to quickly engage in problem exploration and solving regarding challenges brought to higher education. Thus, it is not surprising that many more articles were identified for this review, compared to previous reviews that only focused on empirical studies or were limited to specific aspects of emergency online learning in higher education (eg, Turnbull et al., 2021). Moreover, the number of documents included in this study far exceeded that of the bibliometric analysis conducted by Rodrigues et al. (2020), which was published in September 2020. This difference suggests that in addition to different inclusion criteria, one more year has witnessed a vast number of publications pertaining to the impact of COVID‐19 on online higher education.

One interesting finding from the study is that journals with most publications did not include more prestigious, high‐impact journals focused on online learning and educational technology. There are several possible explanations. First, high‐impact journals may implement more rigorous peer‐review procedures that take a longer time to publish COVID‐19 articles. Second, these journals may produce fewer issues and articles than journals with most publications identified in this study. Third, COVID‐19 has presented a large number of abnormalities in journal publishing procedures, which might have resulted in skewed distribution of publications across journals.

Another important finding is that a majority of identified publications are open‐access articles. Multiple highly co‐cited documents are also open access, such as Cao et al. (2020; 2948 citations), Bao (2020; 1124 citations), Sahu (2020; 1058 citations) and Crawford et al. (2020; 1056 citations). One observation is that the turnaround time from submission to acceptance of these articles is four to six days. The rapid turnaround might be that these publishing journals had sped up the peer‐review process as a quick response to the needs for information on how to address challenges caused by the pandemic. Alternatively, it might be that these publishing journals, such as Psychiatry Research in which Cao et al. (2020) was published, provide rapid publication of research reports, regardless of COVID‐19. Moreover, a number of journal publishers made COVID‐19 articles permanently or temporarily open access as an effort to support distribution of scientific findings (Fraser et al., 2021). It is beyond the scope of this study to investigate whether such efforts could be found in the current literature base. However, it will be worthwhile for future research to examine the impact of rapid publishing of and free access to research on online learning during the COVID‐19 on recognition and dissemination of scholarly findings over the long term.

Overall, scholars from 103 countries or regions in the Global North and Global South investigated a wide array of topics, such as use of various technologies and strategies, redesigned curriculum, student perceptions and psychological impacts of the pandemic‐imposed online learning. We also found more articles produced by scholars from the Global North, and also the documents that were co‐cited across articles in the review literature. Specifically, scholars from the United States, India, China, Spain and Saudi Arabia published most articles. From the perspective of theoretical and methodological approaches, many researchers applied technology acceptance theories and SEM to investigate factors associated with adoption and impacts of the pandemic‐imposed online learning. Of the large quantity of research, medical education and chemistry education were the most‐investigated disciplines. Additionally, inquiry‐based learning, discovery learning, hands‐on learning and collaborative learning emerged as instructional approaches frequently discussed or utilized across these studies.

Ongoing and emerging challenges to online higher education

Mirroring previous discussions around online learning (eg, Singh & Thurman, 2019), findings of this review revealed that researchers utilized and explored a wide array of concepts or terms regarding online teaching and learning, such as distance learning, online teaching, emergency online education and remote teaching. As discussed in Hodges et al. (2020) and Moore et al. (2011), which are frequently co‐cited articles in the current literature, various terms in relation to online learning encompass different meanings, but are often used interchangeably. Regardless of various terminologies, this review found that researchers around the globe rapidly investigated unexpected challenges related to online higher education learning environments. Some examples include low‐level of student motivation and satisfaction, mental health issues of faculty and students, lack of digital competence and self‐efficacy, barriers to adopting new learning modalities, lack of access to technological infrastructure and software, and other inequity issues. These challenges are not new, but became prominent during the COVID‐19 lockdowns implemented across the world.

One thing that was not prominent in the literature was investigating strategies, impacts and challenges of online learning for postsecondary students with disabilities. Through the abstract search, we only identified eight relevant articles from the large literature base (eg, McMahon et al., 2021; Meleo‐Erwin et al., 2021; Spencer et al., 2021). As reported in Meleo‐Erwin et al. (2021), students with disabilities continued to face barriers to learning during the pandemic due to the lack of accessible information on remote instruction and counseling. The absence of sufficient research and challenges accelerated by the pandemic suggest a great need for more efforts to investigate online learning that ensure equitable education opportunities for students with disabilities.

Awareness of the role of technology in supporting online learning in HEIs is critical for access and implementation. However, technology alone does not address profound educational challenges of establishing more equitable, responsive, and sustainable education systems during and beyond COVID‐19 (Facer & Selywn, 2021). In addition, it is important to note that advances in technology are outpacing educational research (Njenga & Fourie, 2010). For example, researchers have explored the use and effectiveness of various technologies, such as virtual reality, simulations and synchronous/asynchronous learning tools, in both K‐12 and higher education settings (eg, Rogers, 2011; Smyth, 2011; Wu et al., 2020). Before gaining a full understanding of how these technologies can be applied to facilitate teaching and learning, emerging technologies and practices, such as artificial intelligence, micro‐credentialing, blockchain and open educational resources, are reported and predicted to have the potential to shape the future of global higher education teaching and learning (OECD, 2021; Pelletier et al., 2021). An important implication for policymakers, researchers and practitioners is to leverage technology to engage with innovative pedagogy, rather than solely focusing on technology.

Placing innovative pedagogies at the forefront of online learning

Researchers have long posited that innovation involving use of technologies should put student learning as its central concern and focus on pedagogies that could help establish core relations among students, instructors and the conditions in which students would be able to shape learning experiences appropriate for them (Clegg et al., 2010). Although the pandemic has imposed unprecedented challenges to the global education systems, there are strategies that exist in previous literature that can address these challenges. For higher education, prior studies found promoting interactions among students, peers and instructors as well as fostering collaboration in small groups based on similar interests or roles were effective in building trust, supportive and inclusive online learning communities (eg, Carrillo & Flores, 2020). In addition, implementing strategies that support student time management, effort regulation and critical thinking in online higher education learning environments could predict positive learning performances (Broadbent & Poon, 2015).

Findings from the current study showed that many researchers and practitioners highlighted inquiry‐based learning, discovery learning, hands‐on learning, collaborative learning and self‐regulated learning as pedagogical strategies for emergency remote learning. These strategies highlight students' role as collaborative, active and self‐regulated learners, which have been frequently discussed in the previous literature about online learning in higher education (eg, Broadbent & Poon, 2015; Kirschner et al., 2004). Nevertheless, it is not clear whether instructional approaches and strategies identified from the literature apply to all students with diverse educational and mental health needs. On the macro‐level, the success with online higher education is associated with economic, political, social and cultural influences within different contexts (Clegg et al., 2010).

Rapid, but imbalanced distribution of evolving literature

It is promising to witness an unprecedented level of participation in online higher education investigations from the international communities of research and practice. Our data suggest that open access publishing contributed to the rapid distribution of research findings. Previous research has shown that open access would widen the readership among researchers, practitioners and other stakeholders (Beck et al., 2020). These nimble responses generated a large corpus of knowledge that has potential to reach a broad readership. On the other hand, although open access helps reduce barriers of knowledge dissemination and stimulate a wider global participation in scientific investigations (Evans & Reimer, 2009), it has such controversies as substandard peer‐review process and quality control (Wicherts, 2016). Thus, it is also imperative for researchers to ensure dissemination of high‐quality scholarly work towards advancing knowledge on online higher education.

This review also reveals an imbalanced distribution of the evolving literature and most co‐cited documents between the Global North and South, which accords with previous evidence of the dominance of the Global North in the production of knowledge (Collyer, 2016). This begs the question of how to promote more equitable production, dissemination and transfer of knowledge regarding new challenges and opportunities for online higher education around the globe. Because of the global crisis in education, numerous higher education institutions, faculty and students within different contexts have had experiences with implementing and engaging with online learning in varying formats. Drawing upon large‐scale, cross‐section research approaches, future research can explore country‐, university‐, faculty‐ and student‐level factors that can contribute to the effectiveness of different pedagogical practices for online learning.

Limitations and future research

While this study offered a holistic knowledge map of research investigating online higher education during the COVID‐19 pandemic, it has some limitations worth noting. First, while this study provides valuable information on rapid responses to the pandemic from the global research and practice communities, the short timeframe as a search parameter might affect the type and depth of studies included in this review. Nevertheless, the large volume of research indicated a high level of involvement from multiple stakeholders in a wide range of rapid responses to the unprecedented disruption in education. The purpose of this bibliometric review was to provide a holistic view of how these quick responses were positioned in the research; thus, current findings of this study could inform future research. Follow‐up studies are needed to provide a more fine‐grained analysis.

Second, the bibliometric analysis applied in this study limits our focus to overall topical themes emerging from the large body of literature. Thus, our analytical strategy leaves a few questions unanswered, such as the quality of rapid publications, a more fine‐grained picture of applications of various technologies and pedagogies, and effectiveness of investigated strategies on student learning during the pandemic. Therefore, investigations on the design, use and effect of various technologies and online learning strategies within different higher education systems at a more fine‐grained level are much needed. Additionally, we used the default threshold of minimum 5 occurrences to identify keywords, which might bias the results. Future research can apply different thresholds to examine similarities and differences in emerging themes.

Third, our search parameters of including documents provided by WoS and published in English only may lead to an exclusion of relevant studies that were not available in WoS or written in other languages. In particular, previous research showed that high‐income or Western countries publish more research in English compared to other countries (Crompton et al., 2021). Moreover, although WoS is frequently used as the largest multidisciplinary database of scientific literature and publishes conference proceedings as items in journals, it may not include conference proceedings about online learning and higher education during the pandemic given its preference to high‐impact biomedical sciences journals. Thus, it is important to note that these parameters might impact the findings of this study. Future bibliometric investigations of research on multidisciplinary, global issues can consider expanding search parameters to include more studies.

Fourth, there are studies exploring how universities responded to the COVID‐19 pandemic that had not yet been published by the time we conducted the search. In particular, some journals implement a prolonged peer‐reviewed and rigorous publishing process. This may leave a number of to‐be‐published studies unanalyzed within the current study. Therefore, researchers can consider conducting a follow‐up analysis in the future.

CONCLUSION

Bibliometric analysis in this study allowed us to examine a large corpus of studies investigating how HEIs responded to the sudden transition to online teaching and learning through research and practices conducted by scholars across disciplines during the crisis of COVID‐19. The large volume of evolving research, especially published in open‐access journals, indicates an unprecedented level of participation from scholars around the globe in rapidly exploring a wide range of topics associated with the pandemic‐imposed online higher education.

Prominent documents identified based on co‐citation analysis revealed that intellectual relatedness between previous and evolving research on online higher education was mainly focused on identifying effective instructional practices, widely adopted models and statistical analysis techniques, and impacts of the pandemic. These foci were reinforced by the results of keyword co‐occurrence analysis, which identified topics from the review database, such as psychological impacts of the pandemic, student perceptions towards online learning and factors affecting technology acceptance and provision of supports (eg, technologies, instructional strategies) for a successful transition.

The rigor and quality of rapid responses identified from this emerging literature is not clear. However, it is promising to witness the unprecedented level of scholarly participation from both the Global North and South in generating knowledge to address a global challenge to education. Furthermore, we encourage researchers and practitioners in HEIs to place innovative pedagogical practices at the forefront when striving to address ongoing and unexpected challenges and opportunities for online higher education moving beyond COVID‐19.

CONFLICTS OF INTEREST

The authors declare no potential conflicts of interest regarding the research, authorship and/or publication of this article.

ETHICS STATEMENT

Data of this study were collected from the existing, published articles. No data were collected from human subjects.

Funding information

This research did not receive any specific grant from funding agencies in the public, commercial, or not‐for‐profit sectors.

Table S1

Click here for additional data file.

Supplementary Material

Click here for additional data file.