COVID-19 in the Twitterverse, from epidemic to pandemic: information-sharing behavior and Twitter as an information carrier.

In this study, we defined a Twitter network as an information channel that includes information sources containing embedded messages. We conducted stage-based comparative analyses of Twitter networks during three periods: the beginning of the COVID-19 epidemic, the period when the epidemic was becoming a global phenomenon, and the beginning of the pandemic. We also analyzed the characteristics of scientific information sources and content on Twitter during the sample period. At the beginning of the epidemic, Twitter users largely shared trustworthy news information sources about the novel coronavirus. Widely shared scientific information focused on clinical investigations and case studies of the new coronavirus as the disease became a pandemic while non-scientific information sources and messages illustrated the social and political aspects of the global outbreak, often including emotional elements. Multiple suspicious, bot-like Twitter accounts were identified as a great connector of the COVID-19 Twitterverse, particularly in the beginning of the global crisis. Our findings suggest that the information carriers, which are information channels, sources, and messages were coherently interlocked, forming an information organism. The study results can help public health organizations design communication strategies, which often require prompt decision-making to manage urgent needs under the circumstances of an epidemic. © Akadémiai Kiadó, Budapest, Hungary 2021.

Introduction

Since the first case of the novel coronavirus was detected in Wuhan, the capital city of Hubei province in China, on December 31, 2019, the outbreak quickly became a global crisis and greatly impacted many people’s lives. Under the massive quarantines instituted worldwide due to the rapid spread of the virus, social media platforms have become one of the most essential information channels to the world and between users for all-around, real-time, non-physical communication. There are 3.5 billion active social media users, equal to approximately 45% of the global population, and visiting social networking websites is one the most popular Internet activities with the highest user engagement (Park & Park, 2020; Park et al., 2021; Tjepkema et al., 2020). The exponential growth of social media use elevated the presence of public-centered media environments (Matsa & Shearer, 2018; Yoon & Chung, 2020). Social media platforms can thus be the best places to learn about people’s interests and concerns about the outbreak of the new epidemic because information sharing and the diffusion of information about the novel coronavirus occurs simultaneously within social media networks. Among the popular social media platforms, Twitter facilitates interactions between online users, and with minimal access restrictions to the platform, any users or accounts can become opinion leaders or influential by functioning as an information or communication hub in the network (Chong & Kim, 2019). Twitter has become a “model organism” for research due to its extensive use and relatively open data policy (Tufekci, 2014).

However, existing studies have gaps in analyzing information sharing about the new coronavirus on Twitter. For example, in the COVID-19 Twitter networks, some studies found a large number of non- or less credible information sources, including uniform resource locators (URLs) (Allen et al., 2020; Broniatowski et al., 2020; Yang et al., 2020a), while other studies reported evidence of a distinguished presence of public authorities and experts instead of low-quality information promoted by retweet activities on Twitter (Gligorić et al., 2020; Park et al., 2020; Shahi et al., 2020). Moreover, a great amount of fake news and misinformation circulated regarding this new epidemic (Hern, 2020, March 4; Shmerling, 2020). Alex Hern, technology editor of The Guardian, said that “Twitter has become a hotbed of inaccurate and dangerous advice, while others – such as Pinterest and WeChat – restrict users’ ability to communicate about the outbreak at all” (Hern, 2020, March 4, para. 1).

Including the 2009 H1N1 pandemic, Twitter has often been used for public health surveillance (PHS) (Jones 2011; Mandeville et al., 2014; Merchant et al., 2011; Rocklöv et al., 2019) and crisis management (Cho et al., 2013; Du et al., 2019; Gunawong et al., 2019; Jung & Park, 2014; Shan et al., 2019). Despite the frequent application of Twitter as PHS due to its popularity and accessibility, there is little research viewing Twitter as an information channel during a pandemic, despite its heavy use for information sharing in disaster settings. Drawing from the concept of PHS, we coined the term “public health information surveillance” (PHIS) and defined a Twitter network as an information channel functioning as a PHIS application. Most COVID-19 studies, including those cited above that yielded different results, analyzed a dataset without considering the deepened circumstances of the epidemic. In this study, we examined the phases of the epidemic via three chronologically different Twitter networks: the local epidemic phase in January 2020, the global epidemic phase in February 2020, and the pandemic phase in March 2020.

Therefore, by investigating the new coronavirus Twitter networks as information channels performing as PHIS during a global public health crisis and further examining the conflicting findings from the early COVID-19 research studies, we examined the characteristics of information sharing among three Twitter networks with temporal gaps. This stage-based approach was particularly imperative because it uncovered new findings from those reported in existing COVID-19 research. The policy suggestions based on our findings can facilitate public health policy and public health information policy. This study also contributes to the applicability of Twitter as PHIS.

Theoretical framework

The development of information communication technology has enabled Internet-based disease surveillance that uses digital data to “nowcast” and forecast an epidemic (Park et al., 2018). According to the World Health Organization (WHO), PHS is “the continuous, systematic collection, analysis, and interpretation of health-related data needed for the planning, implementation, and evaluation of public health practice” (World Health Organization, 2017). PHS can help researchers discover trends in disease prevalence and emerging public health issues and determine intervention points. PHS has undergone many developments over the years, as shown in Fig. 1 (Aiello et al., 2020). However, Google Flu Trends failed to predict the H1N1 (swine flu) pandemic in 2009 and overestimated the number of flu cases during the 2012–2013 flu season, predicting more than double the number of cases reported by the Center for Disease Control and Prevention (CDC). This provoked criticism from scholars, which resulted in removal of the site in 2015 (Aiello et al., 2020; Lazer et al., 2014). Although the multiple erroneous predictions of Google Flu Trends sparked awareness of the biases ingrained in digital surveillance, Internet-based search tools and social media have expanded the scope of PHS by providing real-time data.

Fig. 1

Major events in digital PHS (Aiello et al., 2020)

Since the 2009 H1N1 pandemic, public health organizations have increasingly been using social media for the distribution of health information, preparation for emergency situations, and communication related to disease management (Jones 2011; Merchant et al., 2011; Thackeray et al., 2012). Social media functions as an effective information carrier for facilitating quick communication with the public in addition to offering possible benefits for infectious disease management and surveillance (Mandeville et al., 2014). Aiello and his colleagues (2020) claimed that the components and characteristics of social media users’ posts can offer researchers opportunities to extract further information through natural-language processing and image analysis.

The most popular PHS application of Twitter data includes modeling flu infection rates through tweet frequencies (Rodríguez-Martínez & Garzón-Alfonso, 2018). Moreover, the geolocational information of tweets allows researchers to model the spread of diseases through human geographic mobility, possibly allowing their models to achieve greater precision (Rocklöv et al., 2019). While Twitter has been by far the most-used social media site in the context of PHS since 2006, there is a lack of research on its use as an information channel for the global community during a pandemic. Moreover, given the urgency of efforts to stem the COVID-19 pandemic, the international community has been striving to find ways to enhance PHS. We introduce the concept of PHIS while responding to the urgency and filling the gap in the PHS research. Examining the nature and type of information distributed and shared during the epidemic can be as important as controlling the spread of the virus because disinformation or misinformation could exacerbate negative consequences, particularly in the digital age (Chong, 2019).

Twitter, as an information carrier, allows its users to include information that motivates them to seek and share content from diverse information sources. The literature has primarily discussed three types of information carriers: channels, sources, and messages. Channels have been described as “an information transmission system” (Goldenson 1984, p. 137) or “the means by which the message gets from the source to the receiver” (Rogers & Shoemaker, 1971, p. 24). They are often cast as restraints, as in the pipeline metaphor; that is, they keep messages within themselves (Axley, 1984; Putnam & Boys, 2006). A channel is the largest aggregate among the concept of information carriers, which is the stem of an individual’s decision tree regarding the trajectory of information-seeking and sharing (Johnson & Case, 2012). In this study, we viewed Twitter as an information channel.

A source produces a specific node or venue for information. “A source is an individual or an institution that originates a message” (Rogers & Shoemaker, 1971, p. 251) or “[a] work, etc., supplying information or evidence (esp. of an original or primary character) as to some fact, event, or series of these. Also, a person supplying information, an informant, a spokesman” (OxfordEnglishDictionary.com 2020). Information sources embedded in Twitter may include news articles, scholarly content, personal content, links to websites, and images that share similar or different amounts of reliability, credibility, attributes, trustworthiness, or dynamics (Johnson & Case, 2012).

A message is composed of words, symbols, or indications that convey a certain type of content emerging from a certain source within a certain channel (Johnson & Case, 2012). Berlo (1960) defined messages as “the expression of ideas (content), expressed in a particular way (treatment), through the use of code” (p. 169). Therefore, we expected the content of the sources embedded in the tweets sampled in this study to contain messages that carried specific implications regarding the novel coronavirus outbreak. Social media content generally includes many sentiment-incorporated sentences. Analyzing sentiments requires mining attitudes and emotions, such as happiness, sadness, or anger, which lie in the realm of personal impressions rather than facts, in addition to reflecting an individual’s perspective on a specific topic (Chong & Chang, 2018). On Twitter, sentiment analysis research has been popular and was largely conducted at the word level due to Twitter’s strict word-limit policy. In addition, we examined scientific information sources and messages shared on Twitter via Altmetric measurements because the attributes of scientific sources and messages shared on Twitter could provide helpful insights on information sharing when the deadly virus was rapidly expanding worldwide in early 2020. Altmetrics aggregates the impact of scientific research studies included in social and mass media, which supplement the impact factor score (Cress, 2014).

Therefore, by taking a stage-based approach, we aimed to examine information sharing on Twitter related to the novel coronavirus outbreak by applying the theoretical framework of information carriers (Johnson & Case, 2012). According to Johnson and Case (2012), “Information carriers are the primary repositories of information available to individuals within their information fields” (p. 31). In this study, we defined Twitter as an information channel for the novel coronavirus outbreak and defined embedded (news) media, influencers, influential tweets, scholarly content, and popular websites in the Twitter networks as information sources in the channel. We defined information, ideas, emotions, signals, and implications as messages included in the information sources. This study aimed to answer the four research questions below:

RQ1: What are characteristics of the Twitter networks regarding the COVID-19 outbreak?

RQ2: What are the characteristics of the information sources included in the Twitter networks regarding the COVID-19 outbreak?

RQ3: What are the characteristics of the messages included in the information sources of the Twitter networks regarding the COVID-19 outbreak?

RQ4: What are the characteristics of the scientific information sources and messages regarding the COVID-19 outbreak shared on Twitter?

Data collection

Twitter data were collected on January 24 (N1), February 14 (N2), and March 14 (N3). Each dataset was retrieved with an application programming interface (API) using the data import function of NodeXL, a type of social network analysis (SNA) and visualization software (Smith, 2015). We restricted the datasets to a maximum of around 18,000 tweets per retrieval and examined a total of 74,466 relationships on Twitter. In the Twitter network retrieved on January 24, 2020 (N1), 18,075 vertices created tweets including “coronavirus” or replied to or mentioned tweets containing the term. In the Twitter network retrieved on February 14, 2020 (N2), 17,720 vertices were created using tweets that included the term “coronavirus” or were a reply to or mention of tweets containing the term. In the Twitter network retrieved on March 13, 2020 (N3), 23,889 vertices were created tweets that included the term “coronavirus” or were a reply to or mention of tweets containing the term. Extra tweets included in the datasets were retrieved from prior periods, generally one week to about ten days earlier, which enlarged the complete sample period covered by the data.

We considered the chosen dates as important for several reasons. The WHO declared a Public Health Emergency of International Concern on January 30, 2020. Thus, the Twitter network retrieved on January 24 (N1) can provide insight into the characteristics of information sharing on Twitter before COVID-19 became a global concern. On February 11, 2020, Situation Report 22 published by the WHO reported 43,103 confirmed cases worldwide, and 99% of the cases came from China. On the same day, the WHO officially named the novel coronavirus “COVID-19,” an amalgamation of “coronavirus,” “disease,” and “2019” (World Health Organization, 2020). Thus, the Twitter network retrieved on February 14 (N2) can provide insight into the characteristics of information sharing on Twitter after COVID-19 became viewed as a threat to global public health. On March 11, 2020, a total of 118,319 incidents had been reported globally from 115 nations, and the WHO declared it a pandemic. Therefore, the Twitter network retrieved on March 13, 2020 (N3) can provide insight into the characteristics of information sharing among Twitter users shortly after COVID-19 was declared a pandemic in which a novel virus spread globally due to the public’s lack of immunity (Davey, 2020).

To obtain scientific information sources and messages for the periods examined in the study, we retrieved Altmetric data from altmetric.com. We retrieved scholarly journal articles that were shared on Twitter via a PubMed search query associated with public health sciences research at altmetric.com on April 5, 2020. We applied the four keywords “COVID-19,” “Coronavirus,” “Corona virus,” and “2019-nCov” for the three data-collection points (January 24, 2020; February 14, 2020; and March 13, 2020) to match the articles with non-scientific information sources and messages.

Data analysis

To answer the proposed research questions, we examined and compared differences and similarities between the overall attributes of the entire networks, characteristics of the embedded sources, and features of the messages included in the information sources in the retrieved Twitter networks. We applied SNA to examine the coronavirus Twitter network as an information channel under the PHIS concept. SNA can illustrate relationships and disclose influencers via centrality measures. Wetherell et al., 1994) explained SNA as follows:

Social network analysis (1) conceptualizes social structure as a network with ties connecting members and channeling resources, (2) focuses on the characteristics of ties rather than on the characteristics of the individual members, and (3) views communities as “personal communities,” that is, as networks of individual relations that people foster, maintain, and use in the course of their lives. (p. 645)

We took a typological approach to evaluate the structural features of each network, and, to identify top influencers of the three networks, we used betweenness centrality, a measure of how frequently a Twitter user is located in the shortest path between two other users and how the user links clusters by bridging gaps in the network (Hansen et al., 2011). A Twitter account with high betweenness centrality plays a significant role in linking the major groups and many participants; otherwise, many tweets would be fragmented or isolated in the network. Furthermore, high betweenness centrality indicates a high level of influence and connectivity (Freeman, 1978).

We performed SNA using NodeXL (Hansen et al., 2011). Four types of relationships (edges) were found on Twitter: retweets, replies-to, mentions, and tweets that were self-looped edges or were neither replied to nor mentioned by any Twitter users. Unique edges were defined as unique relationships on the network. N1, N2, and N3 included 22,086, 25,178, and 12,414 edges, respectively.

Each dataset was processed by calculating metrics such as indegree, outdegree, and betweenness centrality for N1, N2, and N3. After removing 8287 duplicated edges, we examined 51,391 unique connections in this study. For network visualization, the Clauset-Newman-Moore algorithm was applied to create clusters. The algorithm calculates the dominant clusters in the network by categorizing vertices into the best-fitting group based on patterns of interconnectedness; this grouping algorithm defines a few main groups and several smaller groups overall (Wakita & Tsurumi, 2007). All three networks were illustrated by applying the Harel-Koren Fast Multiscale layout algorithm to the data. The most frequently mentioned URLs, domains, hashtags, words, word pairs, and sentiment in N1, N2, and N3 were calculated.

Findings

RQ1: What are the characteristics of the Twitter networks as a communication channel regarding the COVID-19 outbreak?

People sharing the same information field also share a context that provides the foundation of information for further interaction (Fisher et al., 2005). The context of this study is the global novel coronavirus outbreak. Twitter is an effective and easy-to-access information channel that most people can join with few restrictions if they have Internet access. To answer the first research question, we analyzed N1, N2, and N3 in terms of communication channels regarding the COVID-19 outbreak while focusing on the major influencers in each network. Figures 2, 3 and 4 provide a network visualization and the most frequently mentioned words in the N1, N2, and N3 networks, respectively. The network visualization of N1, N2, and N3 demonstrates the characteristics of “community clusters” (Smith et al., 2014). However, under close examination, it is apparent that each network presented typological differences. The inter-group connections are relatively sparse in N1 compared to the other two networks, which implies that the novel coronavirus emerged suddenly as a global topic on Twitter. As the virus has spread internationally, inter-group borders have become blurry. Figure 3 shows a seeming peak in people’s focus on the coronavirus as the entire configuration looks like a tight crowd. Figure 4 shows that, while the largest group remains, a few medium-sized groups have gradually dissolved into smaller clusters. This is mainly because the coronavirus has become a worldwide pandemic with varying levels of interest to different regions and populations.

Fig. 2

Top 20 groups in N1 with the top keywords in each

Fig. 3

Top 20 groups in N2 with the top keywords in each

Fig. 4

Top 20 groups in N3 with the top keywords in each

Some researchers (Smith et al., 2014; Song et al., 2019) also claimed that specific information sources and topics triggered several conversations while nurturing their own audiences and/or communities. This phenomenon can encourage the appearance of various perspectives on a topic or event depending on its importance or applicability to diverse audiences, uncovering various opinions on a topic among social media users (Smith et al., 2014). N1, N2, and N3 largely possess the attributes of “community clusters.” For example, all of them had several isolates or self-loop vertices, as in G2 of N1, G1 of N2, and G2 of N3. All three of the networks are composed of multiple medium- or small-sized groups instead of a single dominant, centralized group. Despite the common topic of the COVID-19 outbreak, the Twitter users formed multiple small- and medium-sized conversation groups based on their interests and the relevance of the event with relatively little impact from influencers compared to other conversational archetypes on Twitter, such as polarized crowd networks and broadcast networks (Smith et al., 2014).

The results indicate that actors with high betweenness centrality carry a large amount of traffic with the shortest paths, which makes them significant actors in linking conversations in a weakly connected network (Easley & Kleinberg, 2012). They facilitate connections between Twitter networks by reducing or removing information discrepancies within the network. Therefore, vertices with high betweenness centrality are often considered “top influencers” in the network compared to those who simply have many followers.

As shown in Table 1, international news media outlets, such as @cnn, @ajnews, and @skynews, presented high betweenness centrality in N1. As the primary issue-relevant institution of COVID-19, @WHO was a substantial gap-bridger since the beginning of the epidemic. In a Twitter network, laypersons often achieve high betweenness centrality when they create or retweet tweets that are popular within a specific network. For example, @siwuol is a man who has around 6000 followers but retweeted a tweet that graphically described the prevention tactics for the novel coronavirus, which was retweeted 186,200 times and liked 129,400 times. This single retweet critically positioned him in the N1 network. Despite having few followers, other ordinary users, including @988partick, @old_plot1996, and @smilenio2 in N1; @ischinar in N2; and @hyejoohobi, @slashtrashqueen, @elkhalifag, @jasminebri_anna, and @miguelrmzcorro in N3, played the role of “information connectors” due to their locational merit in each network.

Table 1

Top ten Twitter handles by betweenness centrality in N1, N2, and N3

	N1		N2		N3
Rank	Vertex (Twitter handle)	Betweenness centrality	Vertex (Twitter handle)	Betweenness centrality	Vertex (Twitter handle)	Betweenness centrality
1	@siwuol_	49,624,320.04	@who	13,021,484.63	@realdonaldtrump	44,562,214.18
2	@988patrick	26,691,857.41	@jenniferatntd	9,973,854.964	@hyejoohobi	13,714,341.13
3	@cnn	22,124,504.28	@ischinar	7,152,190.048	@sanchezcastejon	13,176,038.85
4	@old_plot1996	21,175,873.5	@sharonhoole	7,072,644.326	@eugenegu	12,707,562.65
5	@naturezlife	17,747,097.37	@kamalaharris	6,874,271.04	@slashtrashqueen	12,546,038.98
6	@who	13,430,056.16	@howroute	6,034,819.301	@elkhalifag	10,408,211.45
7	@ajenews	12,711,023.93	@livecrisisnews	6,012,616.286	@jasminebri_anna	8,496,679.512
8	@skynews	11,197,578.91	@cnn	5,674,626.199	@miguelrmzcorro	8,159,209.971
9	@conflits_fr	11,077,305.5	@spectatorindex	5,279,425.897	@melenaperezjaen	8,159,209.971
10	@smilenio2	9,516,981.411	@viriyabot	5,129,610.853	@ivanduque	5,982,395.825

In N2, @WHO was the top influencer in the entire network. Among the other influencers, two—@jenniferatntd, a writer, and @kamalaharris, a U.S. senator—were public figures. Interestingly, @cnn was the only news outlet among the top ten influencers in N2. Twitter.com suspended the accounts of @ischinar, @howroute, and @livecrisisnews while notifying users that “Twitter suspends accounts which violate the Twitter Rules” (Twitter.com). As its Twitter handle suggested, @viriyabot, including @spectatorindex, in N2 was a bot account. In N3, after the coronavirus outbreak was declared a pandemic by the WHO, several prominent political leaders emerged as top influencers, including the U.S. president (@realdonaldtrump), the prime minister of Spain (@sanchezcastejon), and the president of the Republic of Colombia (@ivanduque). Table 2 summarizes the top ten Twitter handles in terms of betweenness centrality in N1, N2, and N3 by account type. The bar graph in Fig. 5 presents the composition of the top influencers by their account types.

Table 2

Top ten Twitter handles by betweenness centrality in N1, N2, and N3 by account type

Betweenness centrality rank	N1		N2		N3
	Vertex (Twitter handle)	Account type	Vertex (Twitter handle)	Account type	Vertex (Twitter handle)	Account type
1	@siwuol_	Individual	@who	Public health	@realdonaldtrump	Politician
2	@988patrick	Individual	@jenniferatntd	Iindividual	@hyejoohobi	Suspended
3	@cnn	News media	@ischinar	Suspended	@sanchezcastejon	Politician
4	@old_plot1996	Individual	@sharonhoole	Bot	@eugenegu	Public figure
5	@naturezlife	Individual	@kamalaharris	Politician	@slashtrashqueen	Individual
6	@who	Public health	@howroute	Suspended	@elkhalifag	Individual
7	@ajenews	News media	@livecrisisnews	Suspended	@jasminebri_anna	Individual
8	@skynews	News media	@cnn	News media	@miguelrmzcorro	Individual
9	@conflits_fr	Suspended	@spectatorindex	Bot	@melenaperezjaen	Politician
10	@smilenio2	Suspended	@viriyabot	Bot	@ivanduque	Politician

Fig. 5

Bar graph of the top influencers in N1, N2, and N3 by Twitter account type

RQ2: What are the characteristics of the information sources included in the Twitter networks regarding the COVID-19 outbreak?

We extracted the top domains and URLs included in the tweets in the N1, N2, and N3 networks for use as representative information sources. Among the top 30 domains presented in Table 3, except for two social media platforms, Twitter.com and YouTube.com, and one government website, whitehouse.gov, all the top domains were news outlets. Theguardian.com, nytimes.com, and reuters.com appeared in all three networks as popular information sources. This indicates that Twitter users were seeking information regarding the COVID-19 outbreak largely through reliable news sources, and the most popular news sources for those who shared information regarding the outbreak were the Internet versions of The Guardian, The New York Times, and Reuters. Due to the global scale of the event, geographically diverse national news outlets were also identified, such as scmp.com (South China Morning Post), albertonnews.com, and elnactional.cat. Interestingly, breitbart.com, an American news outlet known for its far-right politics and conspiracy theories, was ranked as one of the top information sources in N2.

Table 3

Top domains in the tweets in N1, N2, and N3

N1		N2		N3
Top domain	Classification	Top domains	Classification	Top domains	Classification
twitter.com	social media	twitter.com	social media	twitter.com	social media
trib.al	social media	trib.al	social media	nytimes.com	news website
theguardian.com	news website	youtube.com	social media	trib.al	social media
infobae.com	news website	reuters.com	news agency website	whitehouse.gov	government website
sky.com	news website	theguardian.com	news website	reuters.com	news agency website
scmp.com	news website	breitbart.com	news website	youtube.com	social media
nytimes.com	news website	albertonews.com	news website	elnacional.cat	news website
reuters.com	news agency website	bloomberg.com	news website	theguardian.com	news website
youtube.com	social media	nypost.com	news website	washingtonpost.com	news website

As in Table 3, Twitter users appeared to be largely interested in news stories and information reported by major national and international news outlets. In terms of the URLs, at the beginning of the outbreak, the most frequently shared URLs were those of Chinese websites, such as scmp.com and http://nbl.whiov.ac.cn (the National Biosafety Laboratory, Wuhan). After COVID-19 became a global concern, resulting in severe disinformation and backlash against Asians, thebreitbart.com and NYpost.com, which is an American news outlet known for sensationalism and its conservative bias (according to the Columbia Journalism Review), was ranked multiple times as one of the most frequently shared information sources in N2. In the N3 network, which was assessed after the COVID-19 outbreak was declared a pandemic and the number of confirmed cases and death toll had risen in the U.S., seven out of ten most frequently shared URLs were U.S.-based websites, such as https://www.whitehouse.gov/briefings-statements/president-donald-j-trump-mobilized-full-resources-federal-government-respond-coronavirus/, https://www.nytimes.com/2020/03/14/technology/coronavirus-purell-wipes-amazon-sellers.html and https://www.foxnews.com/world/hindu-group-india-cow-urine-coronavirus.

RQ3: What are the characteristics of the messages included in the information sources in the Twitter networks and tweets regarding the COVID-19 outbreak?

We extracted the top words, top word pairs, and top hashtags from the data to examine the characteristics of messages included in the tweets in N1, N2, and N3 (Table 4). The top words and hashtags presented messages in line with the information sources. For example, the top words, top word pairs, and top hashtags in N1 showed that many tweets in N1 referred to the novel coronavirus as “breaking news” while mentioning its origin, Wuhan, China. When discussing topics similar to those in N1, in N2, Twitter users mentioned public health and death tolls. The official term for the virus, COVID-19, was the top hashtag as well as one of the most frequently mentioned words in N2. As in the cases of the top domains and top URLs, a conservative orientation, reflected in the inclusion of catchphrases like “MAGA” (“make America great again”) and “americafirst” among the top hashtags, was observed. In N3, many more Twitter users used the term “COVID19” in their tweets and retweets than in N2. N3 also showed increased awareness of the status of the coronavirus outbreak as a pandemic and the suspension of the UEFA Champions League’s season (suspende, fútbol). One of the most frequently shared URLs in N3 was a link to a document titled “Mental Health and Psychological Considerations During COVID-19 Outbreak” (https://www.who.int…). The popularity of this information source appeared to mirror the messages implied in some of the top word pairs, such as “human, rights” and “providing, human,” as the document consisted of a series of messages that “can be used in communications to support mental and psychosocial well-being in different target groups during the outbreak” (https://www.who.int…).

Table 4

Top words, word pairs, and hashtags in tweets in N1, N2, and N3

Rank	N1			N2			N3
	Top words (487,680)	Top word pairs	Top hashtags	Top words (518,304)	Top word pairs	Top hashtags	Top words (663,967)	Top word pairs	Top hashtags
1	#coronavirus (10,078)	#ไวร, สโคโรนา (2430)	coronavirus (7442)	coronavirus (12,059)	coronavirus, outbreak (930)	coronavirus (5846)	coronavirus (16,069)	covid,19 (819)	coronavirus (4254)
2	coronavirus (9448)	save, life (1991)	wuhan (1149)	#corona- virus (7876)	#coronavirus, #coronavirus- outbreak (682)	china (1179)	#coronavirus (5515)	due, coronavirus (532)	covid19 (909)
3	china (4331)	life, #coronavirus (1977)	ไวรัสโคโรนา (1043)	china (4099)	covid,19 (638)	covid19 (1026)	Trump (1719)	sick,leave (332)	covidー19 (520)
4	น (3653)	stay, safe (1424)	china (691)	people (1665)	death, toll (506)	wuhan (577)	people (1588)	coronavirus,pandemic (330)	covid_19 (261)
5	wuhan (3390)	safe, #coronavirus (1014)	coronavirus-outbreak (623)	#covid19 (1638)	hong,kong (476)	coronavirus-outbreak (480)	#covid19 (1302)	suspende, fútbol, (320)	coronavirusupdates (89)
6		joke, share (984)	coronavirus (166)		public,health (456)	covid2019 (358)		human, rights (318)	breaking (60)
7		share, guys (984)	chine (126)		#coronavirus, #covid19 (450)	coronavirus- china (158)		government,due (312)	corona- pocalypse (58)
8		guys, stayn (984)	Ncov (121)		health, officials (436)	maga (121)		coronavirus,providing (312)	coronaviruspandemic (57)
9		wuhan, coronavirus (900)	wuhan- coronavirus (120)		hubei, province (412)	2019ncov (113)		providing, human (312)	corona-outbreak (53)
10		'ฮ,น (860)	Breaking (99)		cruise,ship (375)	americafirst (108)		fútbol suspende, (280)	corona (53)

Table 5 illustrates the results of the word-level sentiment analysis of N1, N2, and N3. Overall, negative sentiments were stronger than positive ones in all three networks. N1 displayed the biggest gap, 1.8%, between positive and negative sentiments. This negativity might have been caused by fears and worries about the new coronavirus outbreak, even though the outbreak was not yet a global concern. Although positive sentiment increased during the examination periods, N2 contained the largest number of negative sentiments, 3.5%, as well as the most violent or angry words (706), as shown in Table 7. The rapid spread of the virus worldwide or the backlash detected in N2 (or both) might have caused this reaction. However, as the sentiments in N3 demonstrated, after the COVID-19 outbreak was declared a pandemic, Twitter users used more positive, as well as fewer negative, words than in N1 and N2 with a gap of only 0.6% between the two sentiments. This might have occurred partially because people began to encourage each other after the panic at the beginning of the outbreak.

Table 5

Word-level sentiment in N1, N2, and N3

	N1	N2	N3
Types	Count	Count	Count
Positive	5059 (1%)	7034 (1.4%)	11,151 (1.7%)
Negative	13,572 (2.8%)	18,189 (3.5%)	15,125 (2.3%)
Gap between positive and negative sentiment	1.8%	2.1%	0.6%
Angry/violent under negative category	91	706	524

Table 7

Angry or violent words under negative category

Rank	N1		N2		N3
1	kill	77	kill	649	hate	304
2	hate	10	hate	31	kill	159
3	destroy	4	burn	12	hurt	36
4			hurt	9	destroy	18
5			destroy	5	bomb	7
Total		91		706		524

Tables 6 and 7 present the top ten positive and negative words and all the angry or violent words in the negative category in N1, N2, and N3, respectively. These largely shared positive and negative words can reveal the trajectory of messages shared among the users in the three networks. For example, while “safe,” shared 2,796 times, was the dominant concern, the users might not have taken the coronavirus outbreak event seriously, as indicated by the high use of the word “joke” in N1. As the outbreak incident became a global concern, the users in N2 promoted supportive messages by including words like “support,” “positive,” and “encourage” in their tweets, and this trend continued to N3 by the sharing of words such as “positive,” “support,” and “thank” when the virus was declared a pandemic. In terms of negative messages, “kill” and “death” were widely shared in N2, revealing the view of this event as a “crisis.” The case as pandemic was highly regarded as a “crisis” and was expressed as an emotionally annoying event, as indicated by the inclusion of the words “fuck or fucking” in the messages in N3. “Racism” appeared in the top 20 words of N2 and N3. However, “virus” and “outbreak” were shared the most in all three networks, suggesting that Twitter functioned as an information sharing channel of the COVID-19 event.

Table 6

Top ten positive and negative words in N1, N2, and N3

Rank	Positive	N1			N2				N3
			Negative		Positive		Negative		Positive		Negative
1	safe	2796	joke	1988	work	494	outbreak	1595	free	543	virus	846
2	lead	266	virus	1933	support	361	virus	989	positive	391	sick	679
3	patient	230	outbreak	1700	patient	265	kill	649	good	376	crisis	517
4	better	180	deadly	687	positive	235	death	880	right	326	outbreak	438
5	helped	151	emergency	350	well	167	toll	531	support	288	die	331
6	great	151	infected	330	good	167	breaking	461	protect	249	breaking	290
7	helped	120	breaking	274	right	152	epidemic	447	well	238	emergency	267
8	lead	120	infection	257	solidarity	149	infected	438	relief	219	refusing	221
9	right	101	death	247	top	145	symptoms	346	work	209	death	216
10	good	92	dead	230	encourage	119	crisis	324	thank	198	fuck (fucking)	373

RQ4: What are the characteristics of scientific information sources and messages regarding the COVID-19 outbreak shared on Twitter?

Tables 8, 9 and 10 present the research studies that obtained the highest Altmetric Attention Scores (AASs) from the three data-retrieval points, January 24, 2020; February 14, 2020; and March 13, 2020. The AAS is an indicator of how frequently a research study has been mentioned on several types of online platforms, such as news sites, blogs, Twitter, Facebook, Wikipedia, and LinkedIn (Altmetric.com; Holmberg & Park, 2018; Park & Park, 2018, 2021). The AAS is a weighted score calculated via a pre-established algorithm, and it represents the attention that a study has attracted online rather than its quality. We collected the top ten research studies, as measured by the AAS, for each Twitter data-retrieval point because we could expect that the AASs of the studies were likely largely determined by the attention they received from Twitter users.

Table 9

Top ten research studies shared on Twitter from the February 14, 2020 AAS dataset

Rank	Research Output Title	Journal/ Collection Title	Publication Date	Altmetric Attention Score
1	Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China	The Lancet	February 1, 2020	12,167
2	Persistence of coronaviruses on inanimate surfaces and their inactivation with biocidal agents	Journal of Hospital Infection	March 1, 2020	11,841
3	First case of 2019 novel coronavirus in the United States	New England Journal of Medicine	March 5, 2020	9062
4	Transmission of 2019-nCoV infection from an asymptomatic contact in Germany	New England Journal of Medicine	January 30, 2020	8722
5	Severe acute respiratory syndrome coronavirus as an agent of emerging and reemerging infection	Clinical Microbiology Reviews	October 12, 2007	8598
6	A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence	Nature Medicine	November 9, 2015	7996
7	Coronavirus latest: confirmed cases cross the one-million mark	Nature.com	April 4, 2020	7165
8	Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus–infected pneumonia in Wuhan, China	Journal of the American Medical Association	March 17, 2020	6148
9	Early transmission dynamics in Wuhan, China, of novel coronavirus–infected Pneumonia	New England Journal of Medicine	March 26, 2020	6093
10	Remdesivir and chloroquine effectively inhibit the recently emerged novel coronavirus (2019-nCoV) in vitro	Cell Research	February 4, 2020	5890

Table 8

Top ten research studies shared on Twitter from the January 24, 2020 AAS dataset

Rank	Research Output Title	Journal/ Collection Title	Publication Date	Altmetric Attention Score
1	Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China	The Lancet	February 1, 2020	12,167
2	A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence	Nature Medicine	November 9, 2015	7996
3	Coronavirus latest: confirmed cases cross the one-million mark	Nature.com	April 1, 2020	7165
4	Cross‐species transmission of the newly identified coronavirus 2019‐nCoV	Journal of Medical Virology	February 19, 2020	5356
5	A familial cluster of pneumonia associated with the 2019 novel coronavirus indicating person-to-person transmission: a study of a family cluster	The Lancet	February 1, 2020	4422
6	A novel coronavirus from patients with pneumonia in China, 2019	New England Journal of Medicine	January 24, 2020	4192
7	Bat coronaviruses in China	Viruses (1999–4915)	March 2, 2019	2780
8	Coronavirus infections—more than just the common cold	Journal of the American Medical Association	January 23, 2020	2433
9	Deposition of respiratory virus pathogens on frequently touched surfaces at airports	BMC Infectious Diseases	August 29, 2018	1896
10	Bat cave solves mystery of deadly SARS virus — and suggests new outbreak could occur	Nature	December 1, 2017	1732

As shown in Table 8, many scientific studies were produced and shared frequently on Twitter since the first confirmed incident of the novel coronavirus at the end of December 2019. These studies mainly examined the medical aspects of the newly identified coronavirus and were particularly focused on investigations of the incidents in Wuhan, China, where the virus was first reported (Nos. 1, 4, and 5). Almost half of the studies that were shared were past studies about coronavirus infections in bats (Nos. 2, 7, and 10).

As shown in Table 9, as the epidemic progressed and became a global event, new findings from scientific studies attracted Twitter users’ attention. For example, some of the most widely shared studies were about examining the infection route (Nos. 2 and 4), developing a possible vaccine for the novel coronavirus (Nos. 4 and 10), and confirmed incidents in countries besides China, such as the U.S. and Germany (Nos. 3 and 4).

Multiple studies in Table 10 referred to the novel coronavirus as “COVID-19” (Nos. 1, 4, and 5), and scientific studies that examined the clinical characteristics, including the respiratory syndrome, of the virus and risk factors for deaths resulting from COVID-19 (Nos. 2, 4, 5, and 7) attracted a lot of attention on Twitter as the coronavirus outbreak became a pandemic. While four studies published before the outbreak are included (Nos. 2, 7, 9, and 10) in Table 8, only one additional existing study is included in Table 10 (No. 5), and all studies presented in Table 10 were either new or were conducted after the outbreak of COVID-19. “A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence” is the only study that appears in Tables 8 (No. 2), 9 (No. 6), and 10 (No. 10), indicating that the study progressively gained attention and was valued by Twitter users as a significant information source in relation to the novel coronavirus among the past relevant studies. However, the new findings from the studies about COVID-19 became more popular and frequently shared on Twitter, as shown in Table 10. For example, “Covid-19 — navigating the uncharted,” “Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) Outbreak in China,” and “Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study” obtained AASs of 12,453 (No. 1), 11,214 (No. 4), and 11,187 (No. 5), respectively, when they first appeared in Table 10.

Table 10

Top ten research studies shared on Twitter from the March 13, 2020 dataset by AAS

Rank	Research Output Title	Journal/ Collection Title	Publication Date	Altmetric Attention Score
1	Covid-19 — navigating the uncharted	New England Journal of Medicine	March 26, 2020	12,453
2	Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China	The Lancet	February 1, 2020	12,167
3	Persistence of coronaviruses on inanimate surfaces and their inactivation with biocidal agents	Journal of Hospital Infection	March 1, 2020	11,841
4	Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China	Journal of the American Medical Association	February 24, 2020	11,214
5	Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study	The Lancet	March 1, 2020	11,187
6	First case of 2019 novel coronavirus in the United States	New England Journal of Medicine	March 5, 2020	9062
7	Clinical characteristics of coronavirus disease 2019 in China	New England Journal of Medicine	February 28, 2020	8862
8	Transmission of 2019-nCoV infection from an asymptomatic contact in Germany	New England Journal of Medicine	January 30, 2020	8722
9	Severe acute respiratory syndrome coronavirus as an agent of emerging and reemerging infection	Clinical Microbiology Reviews	October 12, 2020	8598
10	A SARS-like cluster of circulating bat coronaviruses shows potential for human emergence	Nature Medicine	November 9, 2015	7996

Overall, as shown in Tables 8, 9 and 10, the extensively shared scientific information sources regarding the novel coronavirus on Twitter were or will be published in high-impact clinical journals, such as The Lancet (5 times), New England Journal of Medicine (7 times), and the Journal of the American Medical Association (3 times). Many studies were conducted by joint teams of researchers from medical and academic institutions.

Conclusions and discussion

In this study, we identified the COVID-19 Twitter network as a PHIS application and defined it as an information channel that included information sources containing embedded messages. We reported findings from comparative analyses of three “coronavirus” Twitter networks with temporal gaps by examining the characteristics of scientific and non-scientific information sources and messages. Overall, the three Twitter networks demonstrated the characteristics of “community cluster networks” (Smith et al., 2014). This result indicates that diverse small and medium-sized groups discussing coronavirus relevant topics were created as a communication channel on Twitter.

In the local epidemic phase in N1, trustworthy information and news sources were identified as the top influencers and top domains and URLs, which confirmed the findings of some previous COVID-19 studies (Gligorić et al., 2020; Yang et al., 2020a, 2020b). On the contrary, during the global epidemic phase in N2, discontinued and suspicious Twitter accounts, such as @ischinar, @howroute, and @livecrisisnews, and bot accounts, such as @viriyabot (3.9), @sharonhoole (4.1), and @spectatorindex (4.4), were examined as top connectors. These bot accounts are automated Twitter accounts controlled by bot software. The scores in the parentheses indicate Botometer scores out of 5, and a higher score implies more bot-like activities (Yang et al., 2020a; 2020b). These results corresponded to the findings from some early COVID-19 studies that investigated infodemic on social media during the COVID-19 epidemic (Allen et al., 2020; Broniatowski et al., 2020; Chong, 2020; Yang et al., 2020a; 2020b).

In addition, U.S.-based far-right or conservative news media, such as breitbart.com and NYpost.com, were also widely shared domains in N2. The word-level sentiment was the most negative in N2. The findings may suggest that, when a local epidemic became a global crisis, those who wished to spread manipulative information saw an opportunity to take advantage of the critical circumstances by spreading fake news, malicious information, and hate to certain groups. We suggest that social media companies and governments take prompt actions to prevent an infodemic before a regional epidemic becomes a global crisis.

In N3, the public were actively communicated to the top political leaders, including @realdonaldtrump, @sanchezcastejon, and @ivanduque, while sharing information sources and messages they posted on Twitter.

These high-profile international politicians were determined as top influencers of the pandemic network in N3, which included a government website, such as whitehouse.gov, as one of the primary information sources. Carried messages examined through top word pairs, such as “government due” and “human rights,” were relevant to governance under the pandemic situation as shown in Table 4. Interestingly, @siwuol, which was neither a popular media outlet nor a political or public figure, presented the highest betweenness centrality in the entire network. In fact, several top influencers who played important roles as information super spreaders and connectors within the networks were ordinary individuals. This finding suggests that any Twitter user can become a significant influencer by linking multiple clusters and boosting information sharing across the network. Interestingly, no celebrities were recognized as top influencers, although celebrities are often identified as top influencers in various Twitter networks due to their popularity and numerous followers (Chong & Kim, 2019). This unusual absence of celebrities from the list of top influencers during the public health crisis could be an avenue for future studies.

The characteristics of the information sources analyzed through the top domains and URLs and the implied messages investigated via the top words and sentiments within the Twitter content suggests that the elements of information carriers (channels, sources, and messages) were coherently interlocked, forming an organism of information. Our study results indicate that public health organizations should make more efforts to communicate with the public during a global epidemic. It seemed troublesome that the WHO was the only public health organization that appeared as one of the top influencers in the N1 and N2 stages, and no such organizations were examined in the pandemic phase. As demonstrated in Fig. 5, bot-like accounts and suspended accounts heavily affected the Twitter network while outperforming other types of influencers in the global epidemic phase. Thus, promoting credible information sources and instantaneous updates by public health agencies, as well as the active presence of political leadership, could alleviate the harms resulting from an infodemic created by fake news, misinformation, disinformation, and bot-made content on Twitter. Based on Park and Chung (2020) and the findings of our study, we strongly recommend building a transnational PHIS platform to effectively deal with the pandemic crisis.

There is a dearth of research on examining scientific information sharing behavior via social media during a global epidemic crisis though Bornmann et al., (2020) attempted to investigate Twitter as social-spatial sensors in disseminating scientific information on certain disease, such as HIV, malaria, and tuberculosis. In this regard, the implication of current research is on a par with recent other studies. This study discovered that the largely shared scientific information was about transmission dynamics and clinical features of the disease and case studies of the new coronavirus patients especially infected in Wuhan, China. These findings suggest that clinical scientists quickly initiated investigations on the new coronavirus to inform the world by diminishing an informational deficit with respect to the new infectious disease. Moreover, the trended scientific information per different stage affirms the use of Twitter as PHIS during the pandemic. The results of this study could help public health organizations as well as the governments design information strategies, which often requires prompt decision-making to manage urgent needs during a pandemic crisis.