Making sense of tweets using sentiment analysis on closely related topics.

Microblogging has taken a considerable upturn in recent years, with the growth of microblogging websites like Twitter people have started to share more of their opinions about various pressing issues on such online social networks. A broader understanding of the domain in question is required to make an informed decision. With this motivation, our study focuses on finding overall sentiments of related topics with reference to a given topic. We propose an architecture that combines sentiment analysis and community detection to get an overall sentiment of related topics. We apply that model on the following topics: shopping, politics, covid19 and electric vehicles to understand emerging trends, issues and its possible marketing, business and political implications.

Introduction

Online social networks (OSNs) have been burgeoning in recent years (Alamsyah et al. 2021). This rapid growth of social network, combined with easily accessible data and discussions on multitude of topics provides great research potential for customer analysis, product analysis, sector analysis and digital marketing. Different data science and machine learning techniques such as clustering, association rule mining, ensemble models, deep learning and sentiment analysis, are used in conjunction with digital marketing and product analysis (Saura 2020; Alsini et al. 2018). People use social networks to discuss wide ranging topics and share opinions on them (Wu et al. 2011). Given the scale of information on OSNs, there arises a need to apply different data mining techniques to get actionable insights from them (Davenport 2014; Saura et al. 2019).

OSNs such as Facebook, Twitter, Instagram and so on encourage people to participate and collaborate, forming virtual online communities (Leskovec et al. 2008; Reyes-Menendez et al. 2018). This encouragement is in various forms such as likes, shares, retweets, use of hashtags, comments and mentions. In these OSNs, authors write about their life, share opinions on various topics and discuss wide-ranging issues (Wu et al. 2011). Also, the use of collaboration features, as discussed above, facilitates studies like community detection by allowing formation of multidimensional networks based on friend/follower network (Deitrick and Hu 2013), network based on hashtags (Xiao et al. 2014; Lorenz-Spreen et al. 2018), sentiment based network (Xu et al. 2011) and so on. Multidimensional networks are networks that may have multiple connections between any pair of nodes (Berlingerio et al. 2013) for this purpose, multidimensional analysis is required to gain valuable insights from them. Among different OSNs, Twitter is one of the most studied OSN for social network research (Kumar and Sebastian 2012). One of the main advantages of platforms like Twitter for research is that, on these platforms, users are organized in networks, which makes it possible to investigate groups of people, or communities, united by common interests, rather than individual profiles or personalities which is enabled by extensive use of hashtags, mentions and retweets that forms a complex network (Hubert et al. 2017), which in turn is important for big data analysis and digital marketing (Karataş and Şahin 2018; Saura 2020; Hu et al. 2013).

To gauge profitability of a product or a business, one needs to consider two main things: (1) attractiveness of a business or product and (2) competitiveness level (Chevalier-Roignant and Trigeorgis 2012). Finding attractiveness of a product is important as with time, trend changes. These changes in trends often demand changes to existing business models in order to sustain in the market and to alleviate the inevitable risks involved (Direction 2021). As an example, recent trend to use sustainable energy led to the growth of electrical vehicles shifting the focus from petrol or diesel based vehicles (Hsieh et al. 2020; Hall and Lutsey 2020). A growth in trend is generally accompanied with positive opinion for that topic consequently text analysis techniques are often used to identify public opinion about a trend (Hassani et al. 2020). In many cases, it is often important to get a broader understanding of the topic to understand key players and overall opinion for that sector. Broader understanding of a topic also allows us to better understand emerging trends and public opinion about them. For this purpose, traditional methods tend to be more time consuming as it involves finding relevant topic and then applying sentiment analysis over it (Chandrasekaran et al. 2020). This takes time because topic modeling is a slow process and often involves qualitative human intervention (Reyes-Menendez et al. 2020). Also, existing topic modeling methods does not allow changes to generality of the found topics (Boon-Itt and Skunkan 2020; Chandrasekaran et al. 2020; Reyes-Menendez et al. 2020; Saura and Bennett 2019). With this as our motivation, in this paper we propose a framework that can be used to get related trends and topics accompanied by their overall public sentiment along with a parameter that can be used to change generality of the found topics. Finding recent trends and topics is important for businesses, politicians and marketing agencies alike. We can use the results from our proposed model to answer questions like what is the overall sentiment for a given topic? What are the emerging issues and public opinion about them? How is a product faring compared to other products? What is the general market trend? and who are the key players for a given trend? For this purpose, we apply our model over wide ranging topics like shopping, electric vehicles, covid19 and politics. We then compare our results with recent exploratory analysis in these topics.

In conclusion, the contributions of this paper include: (1) overall topic sentiment classifier model; (2) evaluation of different trends based on our model findings. In this, our goal being to propose a model that can be used to effectively find related topics along with their overall sentiment from a given topic on twitter platform. Furthermore, in our model, we provide a key metric that can be used to vary how general the resultant related topics are with respect to our given topic. We also note that user generated content are qualitative and, therefore, should be used for exploratory analysis (Kim et al. 2013; Pfeffer et al. 2018).

Background

Rapid growth of OSNs and massive data flow through social networks have given rise to research on the analysis of social networks (Alamsyah et al. 2021; Tavakolifard and Almeroth 2012; Kulshrestha et al. 2015). OSNs have also changed the dynamics of how consumers buy products and interact with one another (Lăzăroiu et al. 2020). This change of dynamics combined with modern data mining techniques has led to its use in digital marketing and targeted customer analysis (Saura 2020). In particular, sentiment analysis for opinion mining (Diamantini et al. 2019) and community detection for customer targeting, segmentation and topic modeling (Karataş and Şahin 2018) are widely studied. We use sentiment analysis to understand how people orient themselves about a topic given a piece of text (Yadav and Vishwakarma 2020). Particularly, we aim to determine whether the given text is of positive connotation, negative connotation or neutral connotation (Kontopoulos et al. 2013). It helps us understand public opinion given a text corpus of a given topic. As an example, we can try to identify public opinion about covid-19 based on tweets about it (Boon-Itt and Skunkan 2020).

Another task for understanding broader market dynamics is to retrieve closely related subtopics for which we can perform the above mentioned analysis (Chandrasekaran et al. 2020). To get closely related subtopics, we can use community detection over a topic based network (Lorenz-Spreen et al. 2018). We define community detection as a way in which we attempt to find a set of clusters such that it minimizes intraconnection between them and maximizes interconnection within the cluster in a given set (Fortunato 2018). Another possible approach is using latent Dirichlet allocation (LDA) over corpus of text to find embedded topics within them (Saura and Bennett 2019). Although LDA is a good choice to detect themes discussed in a set of text corpus, it fails to incorporate intrinsic twitter feature ‘hashtag’ that inherently is used for expressing the topic that particular tweet is about (Kumar and Sebastian 2012; Davidov et al. 2010). Furthermore, there is no means using which we can induce the generality metric to find related topics for the given tweets.

In real world, networks are often multidimensional. To get actionable insights from such networks, we require multidimensional analysis to distinguish among different kinds of interactions or equivalently look at interaction from different perspectives. Dimensions can either be explicit that directly reflect interactions such as friend-follower network or it can be implicit that reflect interesting qualities of interactions that can be inferred from the available data, for instance, hashtag network (Berlingerio et al. 2013). In our work, we focus on multidimensional network with two explicit dimensions (1) hashtags and (2) opinions about topics. There can be different interactions between two users. They can be connected to each other with same set of topics with similar opinion. They can be connected to each other with same set of topics with different opinion. They can also be connected to each other with partial set of topics with same or different opinion.

Organization

The rest of the paper is organized as follows. In Sect. 2, we discuss prior works on community detection and sentiment analysis. In Sect. 3, we take a look at community detection and sentiment analysis. The architecture for overall topic sentiment classifier (OTSC) is described in Sect. 4, mentions our results in Sect. 5 and discusses it in Sect. 6. In Sect. 7, we conclude by stating our contributions, and discuss managerial implications and practical/social implications for marketers. Finally, we discuss limitations and future research in Sect. 8.

Related work

It is important to understand emerging trends and their public opinion for making informed business and political decision (Bello-Orgaz et al. 2020; Ansari et al. 2020; Puthussery 2020). Interactions among people in OSN lead to formation of a multidimensional complex network. (Berlingerio et al. 2013) lays foundations of multidimensional network and its analysis.

Among different OSNs, Twitter is one of the most studied OSN (Hubert et al. 2017). (Pak and Paroubek 2010; Kouloumpis et al. 2011; Kumar and Sebastian 2012) analyzes twitter tweets using sentiment analysis. Furthermore, there are many works related to development and discussion about different sentiment analysis models (Kontopoulos et al. 2013; Bhatnagar et al. 2020; Zhang et al. 2021; Yadav and Vishwakarma 2020). These models can be used to understand sentiment of a given text. To train a model for classification purposes, we need labeled data. For this purpose, some automatic data collection methods have been researched, for instance, (Read 2005) used emoji’s to collect and label data while (Davidov et al. 2010) used hashtags for the same.

Other major topic of research for OSNs includes community detection (Karataş and Şahin 2018). Community detection is not a well-defined topic and requires some degree of arbitrariness and/or common sense. Given that, Fortunato (2010) performed a thorough review of community detection algorithms. Karataş and Şahin (2018) studied various applications of community detection such as criminology, public health, politics, customer segmentation, smart advertising, targeted marketing, network summarization, social network analysis, recommendation systems, link prediction and community evolution prediction. Furthermore, in rare instances, community detection and sentiment analysis have been combined, for instance, Deitrick and Hu (2013) focus on using sentiment analysis to enhance community detection. They use different twitter specific features to further enhance the detected community.

Saura and Bennett (2019) proposed a three stage method for text mining using LDA for topic modeling followed by sentiment analysis which is followed by application of text mining techniques. Application of similar procedure can be found in Reyes-Menendez et al. (2018), Saura et al. (2019), Chandrasekaran et al. (2020) and Boon-Itt and Skunkan (2020). Reyes-Menendez et al. (2020) used model proposed in Saura and Bennett (2019) to analyze and understand business implication of #metoo in twitter, further highlighting key takeaways for businesses and advertisers. Liu et al. (2017) also proposed a framework that integrates LDA and sentiment analysis and answer several brand-related questions using it. Liu et al. (2019) used model proposed in Liu et al. (2017) in part to get trendiness metrics for analysis of luxury brands.

All these works use LDA as a base algorithm for finding topics given a text corpus. They then apply different text analysis tasks on those topics. Using LDA on twitter is a bad choice as it fails to incorporate twitter intrinsic feature such as hashtag, fails to provide generality for topics and requires human intervention. To our knowledge, the model proposed in our paper (OTSC) is the first model that identifies these shortcomings and solves them.

Community detection and sentiment analysis

Community detection and sentiment analysis are core components of our architecture. There are various preprocessing tasks required for both stages, and in this section, along with discussing our choice of algorithm, we will also look at preprocessing steps involved for that particular stage.

Community detection

In this section, we discuss community detection briefly in the context of social network analysis. For a more detailed introduction to community detection, refer to Fortunato (2010).

Preprocessing for community detection

To apply a community detection algorithm, we need to model tweets as mathematical graphs. Generally, a friend follower network is selected because community detection (Solomon et al. 2019; Luo et al. 2020), in general, used to detect closely related groups. We are more interested in closely related topics than groups; hence, we use hashtags to model our network (Xiao et al. 2014). Hashtags by nature represent the topic of discussion in that tweet, thus giving considerable information about that tweet (Kumar and Sebastian 2012). To form communities based on hashtags, we first extract hashtags from the tweet and lowercase it to retain meaning irrespective of capitalization. After successfully extracting hashtags, we form combinations of 2 and link all the combinations together. This process is repeated on all tweets to make a network of hashtags. To make this network of hashtags (hashmap), we need general topic G. G would be used to collect data from twitter such that the hashmap generated from it would cover wide ranging topics.

Community detection algorithm

Communities are defined as sets of vertices which are densely interconnected whereas sparsely connected with the rest of the vertices (Parés et al. 2017). We have various community detection algorithms such as Newmans leading eigenvector (Newman 2006), Label Propagation (Raghavan et al. 2007), Louvian method for community detection (Blondel et al. 2008), infomap (Rosvall and Bergstrom 2008) and many more (Chunaev 2020). For our purposes, we want to find k communities instead of some random number of communities. Using k we can change the generality of our result, i.e., the higher the value of k, the more specific a community would be. This is due to the fact that k denotes number of communities found and if there are lesser number of communities, the more general a community is. Hence to find k communities, we use the fluid communities algorithm as it allows us to provide insights into the graph structure at different levels of granularity (Parés et al. 2017).

Fluid communities algorithm is a propagation-based algorithm that is capable of identifying variable number of communities in a network. It is based on the idea of introducing number of communities within a non-homogeneous environment where communities will expand and compete until a stable state is reached. Given a graph where V is set of vertices and E is set of edges in the graph, fluid communities algorithm initializes k communities, i.e., , where . Each community is initialized in a different random vertex and is associated with density d as described in Eq. (1).Fluid community algorithm operates in supersteps and updates communities using an update rule until assignment of a vertex to that community does not change for two consecutive supersteps (Parés et al. 2017).

Sentiment analysis

In this section, we take a look at sentiment analysis and the preprocessing steps required for performing sentiment analysis on tweets.

Preprocessing for sentiment analysis

To train sentiment classifier, we require labeled tweets which we gather using method described in Go et al. (2009). In this method, we use emoji’s to collect data and label them based on polarity of that emoji. In this method, we assume that tweets containing happy emojis like ‘:-), :), :D’ will correlate to a positive tweet and tweets containing sad emoji’s like ‘:-(, :(, =(’ will correlate to a negative tweet. After data gathering is done, we first filter our tweets by converting text to lowercase, removing all hashtags, removing retweet designations (‘RT’), usernames and URLs. After this step, we remove all the stopwords from the NLTK corpus, and we perform tokenization and remove punctuations from the collected tweets.

Sentiment analysis algorithm

After preprocessing, we need to extract features from our tweets, and for that, we apply TFIDF Vectorization [refer Eq. (4)] (Singh and Shashi 2019). In Eq. (2), is the raw count of a term t in the document d. In Eq. (3), N is the total number of Documents, i.e., |D|. The resulting features are passed to the Multinomial Naive Bayes Classifier (Kibriya et al. 2004), which classifies tweets positively or negatively.Naive Bayes classifier is based on Bayes theorem (Anthony 2007) where s is a sentiment, M is a Twitter message. Because we have equal sets of positive and negative tweets we can simplify the equation as:After training our sentiment classifier on our training data for sentiment classification, it performs with 77% accuracy. When we calculate f1 score, we get 76% for positive labels and 78% for negative labels.

Overall topic sentiment classifier

Figure 1 gives a general overview of our architecture. The first step involves collecting data and preprocessing it for training our sentiment classifier and generating a hashmap based on a general topic G (Refer Sect. 3.1.1). After preprocessing is done, we use the hashmap to detect communities using fluid community detection algorithm. It takes in k that determines how many communities should be formed (Parés et al. 2017). By default, we divide our hashmap into ten communities (i.e., ). We can fine-tune these hyperparameters based on our requirements. After finding communities and training our classifier, the trained sentiment classifier and detected communities (C) are passed for analyzing overall sentiments for related topics. In this step, we pass a topic t that we use along with C to find related topics and perform sentiment analysis for those topics.

Fig. 1

Architecture for overall topic sentiment classifier model

Analyzing overall sentiments

This module takes in a general topic G, detected communities C, hashmap from the previous step and a topic t. It first uses a hashmap and t to calculate the most valued, directly related topics . We can find this by looking at neighboring nodes of t, and we pick at max ten topics with the highest weight. We apply Eq. (8) to find suitable community () for t.In Eq. (8), C is set of communities we detect using FluidC algorithm, and is set of directly connected topics we select using hashmap and topic t.After this step, we apply Eq. (9) to all the nodes within and pick 10% nodes with highest Quality factor . It ensures that the topics we pick in a community are of high quality, i.e., have high degree and its combined weight with adjacent edges is high. In Eq. (9), w is total weight of node with its adjacent nodes and d is the degree of the node under consideration. We pass those 10% selected topics along with a value n to our sentiment classifier. n denotes the number of tweets to fetch for each topic for sentiment analysis. To demonstrate, we take and use Twitter API to fetch tweets for our selected topics. The sentiment classifier then classifies sentiment for each tweet. To keep track of overall sentiment, we initialize T with 0 and increment it by 1 for every positive tweet and decrement by 1 for every negative tweet. To normalize, we divide T by n. Finally, the result we get is in the range -1 to 1 where 1 denotes every post encountered is a positive post and -1 denotes every post encountered is a negative post. The greater the output sentiment, the more positive it is.

Results

Twitter users post messages about a range of topics unlike other sites which are designed for a specific topic. Users use hashtags (#) to mark topics a tweet talks or is related about (Kumar and Sebastian 2012). We propose an OTSC model that uses this feature of twitter to make a hashmap of a general topic G and use this hashmap to find closely related topics of a given topic t. Furthermore, it finds overall sentiments of top 10% topics among the found topics. We use our proposed OTSC model and apply it to G = #summer and t = #shopping with k set to 10, 15 and 20. Similarly, we apply our model to G = #covid19 and t = #vaccine, G = #politics and t = #issues and G = #electricvehicles and t = #tesla with k set to 10, 15 and 20.

The found topics can be referred in Figs. 2, 3, 4 and 5. In each of these figures, we added 3 nodes k10, k15 and k20. All the topics connected to k10 are found when , similarly, k15 corresponds to topics found when and k20 for . Sentiments related to corresponding topics can be referred to in Table 1 and 2. In Fig. 2, node k10 is connected with 18 topics, k15 is connected with 8 topics, and k20 is connected with 6 topics. k10 and k15 have 1 topic in common, k10 and k20 have 1 topic in common, whereas k15 and k20 have 2 topics in common. In Fig. 3, node k10 is connected with 12 topics, k15 is connected with 10 topics, and k20 is connected with 8 topics. k10 and k15 have 4 topics in common, k10 and k20 have 4 topics in common, whereas k15 and k20 have 7 topics in common. In Fig. 4, node k10 is connected with 16 topics, k15 is connected with 10 topics, and k20 is connected with 7 topics. k10 and k15 have 3 topics in common, k10 and k20 have 1 topic in common, whereas k15 and k20 have 1 topic in common. Finally for Fig. 5, node k10 is connected with 17 topics, k15 is connected with 11 topics and k20 is connected with 9 topics. k10 and k15 have 10 topics in common, k10 and k20 have 8 topics in common, whereas k15 and k20 have 7 topics in common.

Fig. 2

Resultant closely related topics for #shopping and #summer (, and )

Fig. 3

Resultant closely related topics for #covid19 and #vaccine (, and )

Fig. 4

Resultant closely related topics for G = #politics and t = #issues (, and )

Fig. 5

Resultant closely related topics for G = #electricvehicles and t = #tesla (, and )

Table 1

Overall sentiment table for G = ‘#summer,’ t = ‘#shopping’ and G = ‘#covid19,’ t = ‘#vaccine’

	Summer-shopping	Summer-sentiments	Covid-vaccine	Covid-sentiments
0	#love	0.80	#vaccine	-0.72
1	#shopsmall	0.85	#health	0.22
2	#spring	0.63	#covidvaccine	-0.29
3	#ootd	0.91	#wearamask	-0.04
4	#cute	0.81	#covid19ab	-0.40
5	#summer	0.54	#patients	-0.37
6	#happy	0.89	#yeg	0.05
7	#zazzle	0.94	#yyc	-0.15
8	#funny	0.75	#abpoli	-0.14
9	#games	0.60	#nhs	-0.24
10	#retailtherapy	0.47	#covid19on	-0.14
11	#sweater	0.40	#abhealth	-0.41
12	#beach	0.70	#pandemic	-0.13
13	#consignment	0.68	#saturday	0.56
14	#weship	0.66	#travel	0.62
15	#resale	0.45	#medicine	-0.45
16	#tomball	0.72	#today	-0.12
17	#rtresale	0.64	#insurance	-0.48
18	#womens	0.88	#onpoli	-0.24
19	#accessories	0.73
20	#handmade	0.99
21	#swimsuit	0.86
22	#bags	0.73
23	#teepublic	0.83
24	#mothersday	0.88
25	#dress	0.93
26	#fitness	0.58
27	#sport	0.59

Table 2

Overall sentiment table for G = ‘#politics,’ t = ‘#issues’ and G = ‘#electricvehicles’ and t = ‘#tesla’

	Politics-issues	Politics-sentiments	Ev-tesla	Ev-sentiments
0	#government	-0.13	#evs	-0.24
1	#podcast	0.82	#tesla	0.53
2	#racism	-0.51	#cars	0.11
3	#history	-0.04	#battery	-0.36
4	#feminism	0.42	#bmw	0.35
5	#leadership	0.74	#vw	-0.04
6	#religion	0.09	#eugreendeal	0.35
7	#digitalmarketing	0.85	#renault	0.38
8	#bjp	0.38	#autos	0.23
9	#culture	0.75	#volvo	0.13
10	#shootings	-0.77	#batteries	-0.19
11	#feminist	0.69	#daimler	0.22
12	#laws	-0.18	#climateactionnow	0.11
13	#podcasts	0.84	#energystorage	-0.01
14	#left	-0.29	#hydrogen	-0.35
15	#economics	-0.71	#stocks	0.60
16	#democrats	0.18	#stockmarket	0.20
17	#investment	0.68	#cleanenergy	0.33
18	#nyc	0.42	#electriccar	0.31
19	#trading	0.12
20	#acquaman	0.56
21	#stopasianhatecrimes	-0.14
22	#pakustv	-0.23
23	#china	-0.20
24	#democracy	-0.09
25	#police	-0.52
26	#gop	0.06
27	#freespeech	-0.19
28	#asiapacific	0.21

We find most topics positively viewed for Fig. 2 and most topics negatively viewed for Fig. 3 (Refer Table 1). For Fig. 4, some topics are positive and some are negative, whereas for Fig. 5 most topics are positive while some being negative (Refer Table 2).

Discussion

It is important to understand emerging trends and public opinion about them to make informed decision and gain actionable insights (Saura 2020; Alsini et al. 2018). These trend can be used for marketing analysis as used by Reyes-Menendez et al. (2020) to understand implications of emerging trend and how advertisements can be made while keeping them in mind. Similar to this, much research has been done to analyze emerging trends and their marketing implications (Iyengar et al. 2011; Alalwan et al. 2017; Saura et al. 2019; Kim et al. 2013; Boon-Itt and Skunkan 2020; Reyes-Menendez et al. 2018; Lorenz-Spreen et al. 2018; Chandrasekaran et al. 2020). We can also observe policy changes of government to incorporate and promote sustainable development (Li et al. 2016; Hishan et al. 2019; Elkerbout et al. 2020).

Using these research, we can understand the importance of analysis of emerging trends. Given that, most of the work done to find topics are based of latent Dirichlet allocation (LDA) (Saura and Bennett 2019), which works well but it fails to incorporate hashtags that are primarily used for marking topics in a tweet (Kumar and Sebastian 2012). Furthermore, there is no means using which we can change generality of the found topics when we use LDA. To include these features, we use fluid community detection algorithm that allows us to find k communities using which we can change granularity of the resultant communities (Parés et al. 2017).

We applied our model to find trends in the following (G, t) pairs: (#shopping, #summer), (#covid19, #vaccine), (#politics, #issues), (#electricvehicles , #tesla) for . In each cases, we found maximum number of topics for and minimum number of topics for . From this, we can infer that the topics found tend to be more specific due to the shrinkage of number of nodes in individual communities as we increase k(Number of communities) (Parés et al. 2017).

In Fig. 2, we found some interesting topics like #ootd which stands for outfit of the day and its corresponding sentiment (Refer Table 1) seems to be among the highest. This provides a potential advertising keyword and technique that marketing agencies can use which is also supported by paper (Dar and Tariq 2021). Also emerging topics like swimsuit, beach, accessories, handmade with above average sentiment suggest that people may tend to buy items related to these topics. This also opens a door for business opportunity in accessories, handmade products, bags and custom designs made using zazzle. Topics such as love, cute, shopsmall, weship, mothersday and retail therapy can be used by marketing agencies to promote their products as they correspond to a positive public opinion. Given that, one should be careful to use the keyword ‘retail therapy’ as its below average public opinion.

For topics relating to (#covid19, #vaccine), we found a general negative sentiment. Particularly for #vaccine which might point toward vaccine hesitancy. A research (Rosenbaum 2021) suggests that about 31% of Americans wish to take wait and see approach and about 20% remain quite reluctant about it. Other reason for its negative sentiment might be because several European nations are suspending the use of Astrazenca covid-19 vaccine (Mahase 2021). Furthermore, a negative sentiment in patients might indicate increasing number of covid patients. A positive travel sentiment paired with #yeg (Edmonton International Airport) and #yyc (Calgary International Airport) might suggest ease of lockdown and possible investment opportunity in travel sector (Nayak et al. 2021). A positive sentiment on #health asserts that people are health conscious which provides opportunity related to healthcare and organic products (Tandon et al. 2020). A neutral sentiment of #wearamask suggests that many people have negative sentiment about it where our results match with a similar research which states that among 4099 respondents only 53.3% of symptotic participants reported wearing a mask in preceding week and about 62% people without symptoms did not wear a mask in prior week (Egan et al. 2021) pointing toward need for education about importance of mask.

Results from (#politics, #issues) give us a list of pressing issues such as racism, shootings and free speech. It also points out emerging Asian hate (#stopasianhatecrimes) that recently emerged due to current covid19 pandemic (Xu et al. 2021). Additionally, a positive viewpoint on OSN shows that people tend to view it positively, suggesting that schemes promoting equality might be viewed positively (Reyes-Menendez et al. 2020). Podcasts being connected to all three nodes (k10, k15 and k20) might suggest that people are often using podcasts to listen to or share opinions on politics and pressing issues. Advertisers might want to use that medium to advertise related content.

Finally, Fig. 5 is one of the most overlapping graphs among the topics we explored. These overlaps mostly points toward competitors of tesla such as renault, bmw, daimier, volvo and vw(volkswagen). Looking at the sentiments, we can infer that tesla might be one of the most positively viewed vehicle in electric vehicle domain (Thomas and Maine 2019). Furthermore, the occurrence of topics such as stocks and stockmarket indicates that people that talk about tesla or electric vehicles might also be related to investment community. A positive sentiment in #climateactionnow and #eugreendeal indicates that people are optimistic of sustainable alternatives (O’Riordan 2004) that can be considered while entering a business domain or while creating an advert. It also indicates that governments work on related policies might be positively viewed (Li et al. 2016; Hishan et al. 2019; O’Riordan 2004).

It is also important to note that in most cases overlapping topics are of greater importance as compared with non-overlapping topics, e.g., beach, swimsuit and accessories for Fig. 2, vaccine and health and patients in Fig. 3, podcasts and government in Fig. 4 and different automakers such as renault, bmw, daimer, volvo and volkswagen in Fig. 5.

Conclusion

Analysis of emerging trends is important to both businesses and policy makers. In this paper, we propose an OTSC model (Fig. 1) that can be used to get emerging trends along with their public sentiment based on Twitter tweets. We propose using fluid community detection algorithm instead of generally used LDA for finding related topics in Twitter. This is because LDA fails to incorporate Twitters intrinsic features such as hashtags and does not provide metrics to change generality of the found topics. With the help of fluid community detection algorithm, our model is capable of changing the granularity of underlying community thereby changing the generality of the found topics. We further assert this by applying our model to following (G, t) pairs: (1) (#summer, #shopping), (2) (#covid19, #vaccine), (3) (#politics, #issues) and (4) (#electricvehicles, #tesla) for and 20. Our resulting topics are presented in Figs. 2, 3, 4 and 5. Their corresponding sentiments are in Tables 1 and 2. We found that for , the number of topics found were maximum and for they were minimum pointing that as number of community increase, the topics tend to be more specific thereby less general. We further analyzed our results in context of business analysis and policy analysis and found that we can answer questions like what are the emerging trends, positively viewed keywords, key competitors, find pressing and emerging issues and general sentiment around a topic. This information can then be used to make informed business and political decisions.

Managerial implications

Discovering emerging trends and issues have several applications in business analysis and management. Our research proposes an OTSC model to discover and analyze public sentiments about those emerging trends. Emerging trends can be used to understand a broader market picture potentially helping with business and managerial changes. One such example that we discussed is about emerging trends for shopping in summer that helps us understand how people are more positive about handmade products, swimwear and potentially fitness and sports products during summer. Furthermore, we can understand declining trends such as for sweater as compared with other trends for the same query. A closer look at electric vehicles and tesla points us that clean energy is positively viewed at. One can apply this knowledge to create a positive customer/client outlook by promoting environmental friendly approach. Given this information, it is important to understand how this might affect business and a prior knowledge about the domain is required. Prior knowledge is also necessary to set apart topics for business use and keywords for adverts in the found topics. Our model gives a broader view of the subject, but to take decisions, one need to understand specifics of the topic of interest keeping in mind its broader implications.

Practical/social implications for marketers

Interactivity on the internet shifts the ways in which users perceive advertising. This research provides practical implications on how advertisers can use interactions among users to understand keywords that might give a better perception of their adverts. For instance, we found ootd, love, cute, retailtherapy and shopsmall for shopping in summer. Based on user interactions, our model might also suggest relevant advertisement means (Podcasts for political advertisements) as found in Table 2. Other than that, corresponding sentiments related to keywords can also help advertisers understand how a keyword might affect advertisement. For instance, retailtherapy have below average sentiment score as compared to other found topics. This may indicate that advertisers should take caution while using this keyword and better understanding of using this keyword might be required.

Future work and limitations

Our proposed model uses fluid community detection algorithm, and it does not always return the same result during each run (Sun et al. 2020). Finding a suitable value of k requires trial and error furthermore results may vary at each run. We hand-picked topics for analysis which might include some bias. Furthermore, topics found are open to interpretation and hence subjective. In future, we can try an iterative model that uses OTSC as a base and applies it for different values of k. This work can also be extended for analysis of different trends similar to the following works (Boon-Itt and Skunkan 2020; Reyes-Menendez et al. 2018; Lorenz-Spreen et al. 2018; Chandrasekaran et al. 2020). This work can also be extended to better understand importance of overlapping topics for different values of k.