Latent topics resonance in scientific literature and commentaries: evidences from natural language processing approach.

Resonance is generally used as a metaphor to describe the manner how the information from different sources is combined. Although it is an attractive and fundamental phenomenon in human behavior studies, most studies observed semantic resonances in well-controlled experimental settings at word level. To make up the missing link between word and document level resonances, we devoted our contributions to topic resonances in a novel and natural setting: academic commentaries. Ninety-three academic commentaries from ninety-three authors, along with their references and original papers, are analyzed by a latent Dirichlet allocation based natural language processing approach. This approach can decompose a corpus written and read by an author into several topics with different weights, which can reveal the phenomena ignored at word or document level. We found that (1) topic resonances commonly exist between commenters' fundamental input and output topics; (2) output words are re-allocated by commenters to echo salient input topics; (3) commenters are more prone to associate references which focus on the non-dominant input topics; and (4) topic resonance can even be predicted by a Hebbian-like model which matches the aforementioned findings. These findings will continue to enrich our understanding on the relationship among probe, feedback and context.

Introduction

Memory resonance was originally used by Richard Semon (a German scientist) as a metaphor to describe the manner in which information from different sources is combined. He characterized the memory resonance (called homophony in his monograph) as follows.

“At the ecphory of a combination of engrams … what is given is not a single indissoluble blend of mnemic excitations --‘coalescence’ some physiologists call it--but a unisonant chorus in which the single components of an apparently uniform combination of engrams, distinct indeed from each other as to their time of origin, may be individually discerned” (Semon, 1921, p. 165).

Although resonance is an attractive and fundamental phenomenon in human behavior studies, resonance itself has been used as a verbal metaphor much more frequently than an explanatory theory on how an input invokes the output (Tzeng et al., 2005; Cook and O'Brien, 2014; Yeari and Broek, 2016). Situation model (Graesser et al., 1994) is one of the few models tried to explain the interaction between text-based propositions and reader's prior knowledge. However, the traditional materials used to study resonance are mostly narratives or news with limited length. For a different discourse type, e.g., expository or persuasive essay, or even a longer narrative essay with several stages or sections, words are supposed to be stemmed from an intermediate level between the word level and the document level firstly, and then to the document level. Nevertheless, such intermediate level of resonance is seldom investigated (Kintsch and van Dijk, 1978; Murdock et al., 2017). As a result, three vital questions are left open at this intermediate level:

Q1: Does this intermediate level resonance exist? If yes, then is it related to demographical variables, such as gender, discipline, seniority?

Q2: Which components in the prior knowledge spectrum will be most likely activated?

Q3: Is there a model to quantify the resonance pattern at this intermediate level?

We believe this intermediate level resonance study would become an important but nascent link for a complete vision on semantic resonance process. In this paper, we contributed our efforts to the above three research questions on an intermediate level: topic. We attempted to utilize a computer-based natural language processing method on topic distributions (topic modeling) to extend our understanding in the nature of semantic resonance process. More specifically, we designed a quasi-experiment setting for our studies, i.e., observing the reading materials and the writing outcomes of academic commenters.

To our best knowledge, our work is one of the first studies to explore individual topic resonance by topic modeling, though there are two topic resonance studies reflecting similar design philosophies.

The first one is Kintsch and van Dijk (1978) when topic modeling approaches (Blei et al., 2003; Foulds et al., 2013a,b) were not yet developed. Kintsch and van Dijk (1978) asked the participants to write a short summary after recalling a whole report as much as possible. They found that the macro-operations (a.k.a., topic level operations, e.g., summarization) are under the control of a schema, which reflects the comprehender's goals. Although their model can predict macro-processes (a.k.a., reducing the information by deletion and various types of inferences to its gist), they admitted that a crucial role in macro-processes, general world knowledge is a missing link.

The second similar study (Murdock et al., 2017) figured out a smart way to make up this missing link by assuming the full-text of books listed in Charles Darwin's chronologically-organized reading journals as his prior knowledge. By using an unsupervised Bayesian model, they investigated how this celebrated scientist traded off between exploitation of past discoveries and further exploration when he was searching in an environment with an uncertain resources distribution.

Different from their studies, we observed the topic resonances at a large scale extension from our previous pilot study work (Wang et al., 2016), exploring more general findings in more common cases. Our previous pilot study only involves one sample, which lacks of basic validation, let alone other patterns which can only emerge and be tested in a large scale samples.

The rest of this paper is organized as follows. Section “Related Work” reviews literatures. Section “Materials & Methods” describes the materials and specific methods adopted in this paper. Section “Example” uses a sample study to illustrate the procedure to extend a Latent Dirichlet Allocation (LDA) based topic modeling for our setting. Section “Results” presents the findings of Study 1, 2 and 3. Study 1 and Study 2 were designed for question Q1 and Q2. Study 3 was designed for question Q3. Section “Discussion” discusses the connections between our findings and other related studies. Section “Conclusions” concludes the whole paper.

Related work

Resonance is one of the cornerstones scaffolding many human behavior studies, ranging from text comprehension in Cognitive Psychology to knowledge activation in Social Psychology. In text comprehension, the resonance theory suggested that the linguistic input can automatically and quickly activate any information in memory that matches the input semantically or phonologically (Myers & O'Brien, 1998; Sparks, 2012). The resonance studies in this field focus on the interactions between working memory (Baddeley and Hitch, 1974) and long-term memory during the reading process (Beker et al., 2016). In knowledge activation field, the resonance phenomenon is studied under the name of priming, which refers to facilitative effects of some events or actions on subsequent associated responses (Molden, 2014). The resonance studies in this field emphasize the effects accompanying with knowledge priming (Jens and Nira, 2007).

Although resonance is an attractive and fundamental phenomenon in human behavior studies, resonance itself has been used as a verbal metaphor much more frequently than an explanatory theory on how an input invokes the output (Tzeng et al., 2005; Cook & O'Brien, 2014; Yeari and Broek, 2016). Instead of using resonance as a verbal metaphor, MINERVA 2 (Hintzman, 1984, 1986) was one of the few successful mechanistic models that incorporates Semon's characterization (Semon, 1921, p. 165) on memory resonance and simulates the resonance process mechanically. One of Hintzman's contributions is that, he assumed the abstract output could be retrieved through the summed responses of those engrams that are most strongly activated by the cue (Hintzman, 1984). This assumption shed light on one of missing pieces of the resonance puzzle: how an author incorporates the words into a document.

Since resonances typically occur in interactions of a specific scenario, the answer can be partially found in the situation model (Graesser et al., 1994). This model described the interaction between text-based propositions and reader's prior knowledge. It suggested that five types of information can trigger the interaction: space, time, causation, protagonist and intentionality (Zwaan and Radvansky, 1998). However, it is believed that the occurrence frequency of such information may also play a salient role in semantic resonance. Moreover, the traditional materials used to study the model are mostly narratives or news with limited length. For a different discourse type, e.g., expository or persuasive essay, or even a longer narrative essay with several stages or sections, words are supposed to be stemmed from an intermediate level between the word level and the document level firstly, and then to the document level. However, such intermediate level of resonance is seldom investigated (Kintsch and van Dijk, 1978; Murdock et al., 2017).

Words or sentences are much commonly used as observed variables to infer human thinking activities, as in lexical decision tasks (Humphreys et al., 1989; Kintsch, 1998, p.4) in most experimental studies in the text comprehension and knowledge activation fields. Recently, one of word level resonance-related studies reported that the cognitive mechanism words (such as think, know, question) are an independent and positive predictor of meaningful connections within the text and background knowledge (Clinton et al., 2015). Another related work is conducted by Franklin and Mewhor (2015). They found that studying or recalling a word alters both the existing representation of that word in hologram (a dynamic distributed representation where a person's vocabulary resides) and all the words associated with it. In comparison with word-level, document level resonance studies are fewer. One finding is that students' reading of multiple documents is influenced by perspective instructions (Cerdána et al., 2013). As seen, neither word level studies nor document level studies can compensate for the lack of intermediate level resonance observations.

In this paper, we contribute our efforts on an intermediate level: topic. In Cognitive Psychology, the topic plays an important role in connecting words and documents (Griffiths et al., 2007; Griffiths, 2015). In general, a topic is the subject of a discourse or the subject of a section of a discourse. The real situation is much more complicated than that. In a document, one topic may cover multiple paragraphs, one paragraph may consist of multiple topics. If a paragraph consists of more than one topic, the topic weights are usually different. A topic weight refers to the proportion of this topic in a paragraph or any other corpus. Multiple topic weights can form a topic distribution. Since a document is the output of its author, its topic distribution may reveal the topic attention bias of its author. Therefore, topic weights and topic distributions are used in this paper as indicators to observe topic resonances.

Before observing topic resonances, a proper tool is necessary to decompose a discourse into several topics. In topic models (Blei et al., 2003; Steyvers and Griffiths, 2011), each word is assumed being generated from a single topic, and a document is regarded as the mixture of different topics. In line with these assumptions, a topic is modelled as a probability distribution over words (Steyvers and Griffiths, 2011). The connotation of one topic is reflected in the words with high occurrence probability (Griffiths et al., 2007).

As one of specific topic modeling approaches, Latent Dirichlet Allocation (LDA) represents each document of a collection as a finite mixture over an underlying set of topics, whose distribution is assumed to have a sparse Dirichlet prior (Blei et al., 2003). Chang et al. (2009) found that the topics extracted by LDA are the closest to human judgments, comparing with the probabilistic latent semantic indexing (Hofmann, 1999) and the correlated topic model (Blei and Lafferty, 2005). Recently, a large-scale study (Ben et al., 2016) recruiting hundreds of participants has found that LDA-based topical semantic similarity measurement agrees with the human judgments to an exciting extent.

Topic model is not the only natural language processing method adopted in human behavior studies. Latent Semantic Analysis (LSA) has facilitated the studies on reading comprehension at the level of words/sentences or documents in the past two decades. LDA can be considered as a probabilistic generalization of LSA whereby each text in a corpus is a set of discourse topics with probabilistic weights (Griffiths, 2015). In fact, LDA and LSA are the Induced Semantic Structure (Hu et al., 2005) instances within the framework of Semantic Representation Analysis (SRA) proposed by Hu et al. (2014). SRA is a general framework underlying most existing semantic extraction/encoding methods, motivated by semantic regularities. In this framework, there are five hierarchical and basic language entities: words, phrases, sentences, paragraphs, and documents. A language entity at higher level (e.g., document) can be represented by aggregating the semantic relationships of entities at lower level (e.g., cosine similarity of words in LSA). LDA and LSA are among numerical and algebraic representations for each language entity, which have been invented in recent decades.

In this paper, we measure the topic fluctuation patterns to uncover topic resonances by LDA based topic model, inspired by the word fluctuation patterns in the Landscape model (van den Broek et al., 1996). More specifically, we analyze the differences between topic distributions of an academic original paper (input paper) and the published commentary (output paper) on the original paper. The changes from the input topic distribution to the output topic distribution would be supposed to indicate the commenter's processing characteristics.

To our best knowledge, our work is one of the first studies to explore individual topic resonance by topic modeling, though there are two topic resonance studies reflecting similar design philosophies.

The first one is Kintsch and van Dijk (1978) when topic modeling approaches were not yet developed. Kintsch and van Dijk (1978) asked the participants to write a short summary after recalling a whole report as much as possible. They found that the macro-operations (a.k.a., topic level operations, e.g., summarization) are under the control of a schema, which reflects the comprehender's goals. Although their model can predict macro-processes (a.k.a., reducing the information by deletion and various types of inferences to its gist), they admitted that a crucial role in macro-processes, general world knowledge (similar to prior knowledge here) is a missing link.

The second similar study (Murdock et al., 2017) figured out a smart way to make up this missing link by assuming the full-text of books listed in Charles Darwin's chronologically-organized reading journals as his prior knowledge. By using an unsupervised Bayesian model, they investigated how this celebrated scientist traded off between exploitation of past discoveries and further exploration when he was searching in an environment with an uncertain resources distribution.

Different from their studies, we observed the topic resonances at a large scale extension from our previous pilot study work (Wang et al., 2016), exploring more general findings in more common cases. Our previous pilot study only involves one sample, which lacks of basic validation, let alone other patterns which can only emerge and be tested in a large scale samples.

Materials & methods

In this section, the details about the commentary-centered materials and the specific version of an LDA-based topic model adopted are presented. Although materials were not the same in all studies, all the studies were conducted in the same quasi-experiment setting, a.k.a., the commentary scenario.

In such scenario, details about related terminology are listed in Table 1.

Table 1

Major terminology used in commentary scenario.

Terms	Descriptions	Examples
Output paper	Main bodya of a commentary.	Blaszczynski, A. (2008). Commentary: a response to “problems with the concept of video game ‘addiction’: some case study examples”. International Journal of Mental Health and Addiction, 6(2): 179–181.
Prior knowledge	Main bodya of references (excluding the input paper) cited by the output paper.	Griffiths, M. D. (1993). Fruit machine gambling: The importance of structural characteristics. Journal of Gambling Studies, 9, 101–120.Griffiths, M. D., & Wood, R. (2000). Risk factors in adolescence: The case of gambling, video-game playing, and the internet. Journal of Gambling Behavior, 16, 199–227.
Input paper	Main bodya of the original paper commented by the output paper.	Wood, R. T. A. (2008). Problems with the concept of video game “addiction”: some case study examples, International Journal of Mental Health and Addiction, 6(2): 169–178.

Main body refers to the full text of a paper excluding its authors' information, graphics and reference lists.

A similar assumption about prior knowledge was also adopted by Murdock et al. (2017). This scenario is illustrated in Fig. 1.

Fig. 1

The quasi-experiment setting.

Materials

The materials for Study 1 and 3 were the same, and they consisted of 93 document sets. Each set was made up of an original academic paper (or hereafter, the input paper), an academic commentary (or hereafter, the output paper) on this input paper and several obtainable references of this commentary. Each set corresponded to one commenter's commenting behavior.

The commentaries were collected randomly from a university digital library portal by searching academic articles whose title contains “commentary” or “comment”. However, not all the commentaries downloaded from digital libraries could be put into the document sets for study, unless they satisfied three rules at the same time. The first rule is that this commentary is written by only one unique author. The second rule is that the commentary is only for one specific paper instead of multiple papers or even a more abstract area, i.e., the input paper is a specific paper rather than a collection of papers. And the third rule is that the commentary has listed at least one reference which is not the input paper in its appendix section.

Challenges also occur in collecting various types of references. Common references include: journal articles, book chapters, institution reports, published proceedings, web pages, web portals, and entire books. Some journal articles are too old to access them on-line for analysis. Some web URLs were invalid. Moreover, most web portals are updated frequently. Entire book contains too much information, so it would not be appropriate for the specific tool we use to analyze. Due to these reasons, several strategies were developed to retrieve the full texts of the references as complete as possible.

For old journal articles:

Their full texts could only be typed into modern electronic documents manually. If scanned versions existed, image processing software can help the character recognition and conversion, but human corrections were still needed.

For invalid web pages:

Full texts of invalid or expired web pages could typically be found by searching their titles or author names over the Internet. Occasionally the content was simply moved to a new website.

For web portals:

It was unusual to list web portals as references because its content was updated frequently. If a web portal was listed, it was most likely used to encourage readers to explore the entire portal. In these situations, the web portals were simply discarded.

For entire books:

The entire book would be inappropriate to include due to its magnitude. If a much shorter overview could be found to introduce this book, it would be used to replace the book, otherwise the book was discarded.

A document set is qualified for our analysis only when the majority (in this paper, 50% was set as the threshold) of the commentary's references were available to extract their full texts. Otherwise, this formation was aborted, and a new round of document set formation was started from the beginning by finding a new commentary.

A total of randomly chosen 93 different commenters' commentaries were collected in this paper. To help us analyze the selected document sets, the following variables are defined based on the input paper and authors of commentaries. The seniority, which refers to the duration during the year when a commenter wrote the commentary and the year when the one got his or her last degree, ranges from 1 to 56 years. There are 24 disciplines, which refers to the name of the commenter's department or the name of the journal where the commentary is published. The institutions of authors are located in 22 different countries from four major continents: America, Europe, Asia and Oceania. More detailed demographic information can be found in Figs. 2 and 3.

Fig. 2

Seniority histogram.

Fig. 3

Disciplines' proportions.

The corpus includes all the materials involved: input papers, commentaries, and available citations of these commentaries. After excluding stop-words from the corpus, we obtained 67,684 unique words. They occurred 1446,228 times totally in the corpus. If one word occurs twice, we count it as 2 tokens. In other words, this corpus consists of 1446,228 tokens.

There are 427 available citations from the 93 commentaries. The number of available references in a commentary ranges from 1 to 50, excluding the input paper. On average, there are 4.60 (SD = 5.67) available references in a commentary.

On average, there are 38.27 (SD = 30.95), 6.94 (SD = 3.79), and 189.41 (SD = 247.20) paragraphs in an input paper, output paper and available references respectively. The average length of paragraphs in an input paper, output paper and available references are 67.00 (SD = 18.34), 71.74 (SD = 49.35) and 66.02 (SD = 14.52) tokens respectively.

The material for Study 2 are the same with Study 1, except that the prior knowledge is synthetically pseudo. The pseudo prior knowledge was faked by switching the references of two commentaries in irrelevant disciplines. In other words, each reference of a commentary in Study 2 was replaced with an unrelated article.

Methods

In this subsection, the major methods for measures and analyses are presented. An evolution of the specific version of LDA adopted for measure is introduced briefly. And several simple analysis methods are enumerated.

Measures

For each of 93 document sets, Stochastic Collapsed Variational Bayesian Inference (SCVB0, Foulds et al., 2013a,b) was extended to decompose the semantic information of the input paper, the output paper and the prior knowledge into a fixed number of topics.

SCVB0 is an LDA-based natural language processing approach, which can learn human-interpretable topics within a corpus efficiently (Foulds et al., 2013a,b). In a general LDA-based topic modeling approach, the topic distribution for document and the word distribution for topic are two crucial variables. There are dozens of parameter estimation algorithms to estimate these two variables and make inferences. These algorithms can be classified into several paradigms. These paradigms include Expectation-Maximization (EM), online version (e.g., Zeng et al., 2016) and parallel version (e.g., Wang et al., 2015). Since the latter two paradigms are concerning about giant data streams or expensive parallel hardware, we focus on the EM framework for a general computing platform in this paper.

The traditional EM (Dempster et al., 1977) was trying to get an exact estimation on these two variables directly. However, this traditional EM suffers from the problem that the number of estimated parameters grows significantly with the amount of input data. To deal with this issue, one idea is to sample values in high dimensional distributions as in Gibbs sampling (e.g., Griffiths and Steyvers, 2004). Another idea is to set tight lower and upper bounds to estimate the posterior topic and word distributions as in variational methods (e.g., Blei et al., 2003).

Recently, a few studies found that algorithms operating in collapsed space (e.g., Collapsed Variational Bayesian inference, CVB), where only latent variables are left and other parameters are marginalized out, can improve the efficiency of LDA than their un-collapsed counterparts. Based on CVB0 (Teh et al., 2007), a stochastic algorithm (SCVB0) was developed to learn human-interpretable topics more accurately and more quickly, both on large and small datasets. SCVB0 has become a standard method whose performance is a benchmark (Zeng et al., 2016).

On the other hand, although collapsed Gibbs sampling is an alternative approach, it needs more human's experiences on parameter estimation (Teh et al., 2007). Therefore, we extend SCVB0 to decompose paragraphs of a document into several topics in our paper. An illustrative example in Section “Example” will show the strategy on this extension.

Analysis

Once we use the extend SCVB0 to measure the topic distributions, the topic-level semantic information can be represented by topic weight vectors. Then we can analyze the topic-level semantic information differences of input, output and prior knowledge by comparing their topic weight vectors. This kind of analysis paradigm is shown in Fig. 4.

Fig. 4

Analysis paradigm.

The topic-level semantic information differences of input, output and prior knowledge can uncover the topic-level semantic processing patterns of people when they write commentaries on input papers. The processing patterns include the changing trends of fluctuations, correlations, and polarizations of input, output and prior knowledge topic distributions. The observation metrics of these processing patterns are listed below.

Metrics

For the topic-level semantic differences of two topic distributions, we use the correlation coefficient of their topic weight vectors to examine how far they are from each other. If the value of correlation coefficient is very close to 1, we say they fluctuate at the same pace. If the value of correlation coefficient is very close to −1, we say they fluctuate at the opposite pace.

For some major components within a couple of topic distributions, we use the overlap degree of the indexes (ODI) of the fundamental topics to indicate whether there is a resonance between the fundamental topics of the two topic distributions. We call the maximal weight topic the first fundamental topic. The two most weighted topics are named as fundamental topics. The ODI of input and output fundamental topics is defined in (1). The ODI of prior knowledge and output fundamental topics is defined in (2). I, O and B is for input, output and prior knowledge respectively. F. is short for fundamental. T is short for topic.

For a single topic distribution, we use coefficient of variation (the ratio of the standard deviation to the mean) to describe how biased it is.

By using these metrics, we explore how individuals combine their inputs with their prior knowledge to deliver outputs on the topic level rather than word/sentence level as Tzeng et al. (2005) or Yeari and Broek (2016).

Example

In this section, we will illustrate our strategy on the extension of SCVB0 by an example. The basic idea of this strategy is to apply the SCVB0 algorithm over all the paragraphs in a document set, and sum up the paragraphs-based topic weights whose paragraphs share the same document. These aggregated topic weights are used as representatives to represent a specific topic weight in a complete discourse (i.e., input paper, output paper, and prior knowledge) under the framework of SRA (Hu et al., 2014). The subsection “Details on SCVB0 extension” gives the Mathematic details on SCVB0 extension. Subsection “Illustration for the SCVB0 Extension Results” and Subsection “Top Words in the Example” present the topic distributions of input, output and prior knowledge provided by the extended SCVB0. Subsection “Measures and Analyses” presents the measurement results and analysis.

Details on SCVB0 extension

As the first step, all the documents in one document set are segmented into several paragraphs. These paragraphs are sorted into a queue and numbered from the 1st paragraph in the input paper to the last paragraph in the output paper. This step was inspired by the ad hoc heuristics processing in Tang et al. (2014). During this segmentation step, if a paragraph was just a short sentence, it would be incorporated into the next paragraph. With this method, we are able to roughly keep the lengths of paragraphs equally for analysis.

After document segmentation, SCVB0 is applied to decompose the semantic information of the paragraphs into several topics. These paragraph-based topic weights are added up to represent the topic distribution of the entire document. The separated paragraphs and documents here are treated as “documents” and “collections” in Foulds et al. (2013a), respectively.

Similar to Foulds et al. (2013a,b), the k-th topic distribution for the j-th paragraph is notated as θ. The estimation of θ is notated as . The expected number of words assigned to the k-th topic for the j-th paragraph is . The EM statistics of is notated as . Suppose there are K topics, the length of the j-th paragraph is C, the Dirichlet prior parameter for θ is α. When SCVB0 is applied to the paragraphs, according to Eq. (12) in Foulds et al. (2013b), we have:

Suppose the input paper is segmented into P paragraphs. These paragraphs are labelled from 1 to P. By adding up each across these paragraphs, we have the estimation of the k-th topic distribution, a.k.a. I, for the whole input paper, as in Eq. (4).

Since we roughly make the paragraph lengths equal, it can be assumed that , (j = 1, 2,…, P). By substituting this approximation to in Eq. (4), we have:

We use to denote the mean of for the P paragraphs in the input paper, i.e.,

Thus, Eq. (5) could be transformed into:

Note that the variable part in the right side of Eq. (7) can be regarded as a ratio, which is over . Then Eq. (7) shows that the estimation probability of k-th input topic distribution is closely related to the average density of words (at the sense of paragraph) that the author spent on the k-th topic in input paper. By using Eq. (7), we extend SCVB0 to represent the input topic distribution.

Suppose there are P paragraphs in output paper. And there are P paragraphs in the bodies of available citations of output paper. For the integrity of our extension, the other two equations similar to Eq. (7) are formed as follows:where O and B (including subscripts) are the indicators of the output paper and prior knowledge respectively.

In SCVB0, the number of topics (the value of K) should be determined a priori before decomposing the document into several topics. For the dataset mentioned in Subsection “Materials”, 47.31% commentaries contain no more than five paragraphs, and 59.14% commentaries contain no more than six paragraphs, we can set the number of topics to 5.

Illustration for the SCVB0 extension result

The empirical document set in this example consists of an input paper Wood (2008), an output paper Blaszczynski (2008), and two extra available references of output paper. The extra references are the references which do not include the input paper.

There are 29, 10, and 92 paragraphs in an input paper, output paper and available references respectively. And there are total of 131 paragraphs in this example. The average length of paragraphs in an input paper, output paper and available references are 85.00, 73.90 and 85.29 tokens respectively. And there are total of 11051 tokens in this example.

After applying the extended SCVB0 algorithm over these 131 paragraphs (11051 tokens), the normalized topic distributions of input paper, output paper, and prior knowledge are presented in Fig. 5. Note that the normalized weight refers to the ratio of original weight over the maximum weight in the same distribution. The top 10 words for each topic are listed in the next subsection.

Fig. 5

Topic distributions for a sample document set.

Top words in the example

The label of each topic consists of two representative words selected manually from the top 5 words, though each topic ought to be latent in LDA-based algorithms. With the help of rest words in Top 10 words of each topic, anyone who have not read the materials, may infer that the content of these five topics as follows. For example, Topic A may be about the relationship between the excessive time and the video gaming addiction criteria. Topic B may be about the worries of adolescents when they enjoy playing fruit machines or other lotteries. Topic C may be about multiple cases on the video game playing addictive behaviours of people. Topic D may be about the current researches on factors, and money may probably be one of these factors. And Topic E may be about the structural characteristics of fruit machines (See Table 2).

Table 2

Top 10 words for each topic in the illustrative example.a

Topic index	A	B	C	D	E
Topic label	excessive time	adolescent lottery	people case	research factors	structural characteristics
The 1st word	video	adolescents	video	internet	characteristics
The 2nd word	time	lottery	problems	factors	structural
The 3rd word	game	machine	people	many	machines
The 4th word	excessive	machines	game	use	machine
The 5th word	criteria	played	case	research	fruit
The 6th word	playing	slot	cause	risk	winning
The 7th word	videogames	found	behaviour	forms	pay
The 8th word	videogame	players	individuals	money	player
The 9th word	behaviour	fruit	playing	slot	near
The 10th word	addictive	adolescent	games	adolescent	psychological

We use TF-IDF approach (Salton et al., 1975) to re-order the top words delivered by the extended SCVB0. TF (Term Frequency) here is replaced by the delivered token weight. The IDF (Inverse Document Frequency) here is replaced by the inverse topic frequency (i.e., the number of documents in IDF is replaced by the number of topics, and the number of documents which a certain term belongs to in IDF is replaced by the number of topics which a certain token belongs to). From a systematical view, this LDA-based topic model is followed by an independent plug-in “IF-IDF” module, rather than incorporated with an embedded IF-IDF formula at the very beginning as in (Wilson and Chew, 2010; Nikolenko et al., 2015).

In fact, the whole material in the illustrative example tells about an academic debate on whether video gaming can be addictive. One side of this debate believes that video gaming can be addictive since people may spend excessive time on playing video games, which obviously matches one of the addictive gambling behavior criteria. And this claim is supported by several self-report cases. However, the opposite side argues this claim lacks of clinical evidences which have to explain that video gaming has a structural characteristics leading to addictive gambling. This side also believes that the worries about adolescents' addictions on video gaming are just media panic.

By comparing the above two paragraphs, it can be convinced that the words and topics in Table 2 are human interpretable and cover the summary of the materials.

Measures and analyses

The correlation coefficient between the input and output topic distributions is 0.96 (p = 0.0095 < 0.05, p indicates the significant level). On the other hand, the correlation coefficient between prior knowledge and output topic distributions is -0.94 (p = 0.016 < 0.05). This implies that the input paper and prior knowledge are both highly correlated with the output paper. More importantly, if we look into the original Hebbian learning model (Hebb, 1949), we may find that the correlation coefficient implies more than just the correlation extent in our experimental setting. The original Hebbian learning model (Hebb, 1949) used the product of two variables to represent the interactions between these two variables. Therefore, the correlation coefficients of topic distributions can be also regarded as a variant of Hebbian learning at the topic level, a.k.a. interactions between input and output topics, since its formula contains the products of input topic weights and output topic weights. The extra implications brought by correlation coefficients between different topic distributions will be discussed in Section “Discussion”.

For the ODI metric, as shown in Fig. 5, Topic C and Topic A are the fundamental input topics. And Topic A is also the 1st fundamental output topic. According to Eq. (1), equals 1, indicating the input paper and the output paper resonate at a fundamental topic, Topic A. Similarly, equals 0, indicating the output paper and the prior knowledge do not resonate at any fundamental topic. In this sample, the 1st fundamental output topic comes from one of the fundamental input topics. In other words, among the topics in prior knowledge spectrum, the topic which dominates the input paper would most likely be the 1st fundamental topic in output paper.

The coefficients of variations of input, output and prior knowledge topic distributions are 1.16, 1.05 and 0.68 respectively. These values show that the input and output topic distributions are much more biased than prior knowledge topic distribution. They also show that the output topic distribution is less biased than the input topic distribution.

This example not only illustrates how to use the extended SCVB0 to decompose the semantic information of a commentary-centered document set into several topics, but also suggests three preliminary hypotheses to answer the three questions proposed at the end of Section “Related Work”. These hypotheses are listed as follows.

Topic resonance may exist.

Among the topics in prior knowledge spectrum, the topic which dominates the input paper would probably be the 1st fundamental topic in output paper.

Input and output topic distributions are probably biased than prior knowledge topic distribution.

The following studies will test if the hypotheses hold in a large dataset. And it is believed that these studies may bring other findings.

Results

Study 1: resonance test on a large scale

To address the first question (Q1), we explore the robustness of the results obtained in the previous section by examining topic resonance in a larger dataset. The raw data for Study 1 are presented in Supplementary Table S1.

Resonance between output and input papers at topic level

In each document set, the correlation coefficient between the input and output topic distributions was calculated. Among these calculated 93 correlation coefficients, 72 (77.42%) correlation coefficients are positive. Moreover, the top 16 (17.20%) correlation coefficients (0.88–1.00) are significant (p < 0.05) as in Fig. 6. This implies that in a document set, the output topic weight distribution tends to fluctuate with the input topic weight distribution at the same pace. In other words, in the statistics sense, if a topic has a high weight in an input paper, this topic also tends to have a high weight in an output paper. The co-occurrence relationship between fundamental input topics and the 1st fundamental output topic is presented below.

Fig. 6

Correlation coefficients of input and output topic distributions vs. p value.

In 61 of 93 (65.59%) document sets, their ODI of the 1st fundamental output topic and the fundamental input topics equal to 1. This means that, to some extent, one of the fundamental input topics evolves into the 1st fundamental output topic in these 61 document sets. More interestingly, we found that the case that the 1st fundamental output topic evolves into the 1st fundamental input topic has appeared 38 times (40.86%).

If the output paper and the input paper were two completely irrelevant academic papers, the theoretical chance that the 1st fundamental output and input topics happen to be the same topic will be (1/K)2= (1/5)2 = 4% (K is the number of topics, see Section “Example”). Obviously, our findings support the hypothesis that the topic resonance happens often between the fundamental input and output topics.

Resonance between output paper and prior knowledge at topic level

In each document set, the correlation coefficient between prior knowledge and output topic distributions was calculated. Among these calculated 93 correlation coefficients, 69 (74.19%) correlation coefficients are negative. Moreover, the bottom 6 (6.45%) correlation coefficients (−0.92 ∼ −1.00) are significant (p < 0.05). This implies that in a comment behavior, the output topic distribution tends to fluctuate with the prior knowledge topic distribution at an opposite pace. In other words, if the weight of a topic is high in prior knowledge, the weight of the same topic might be low in output paper.

In 22 out of 93 (23.66%) document sets, their ODI of the 1st fundamental output topic and the fundamental prior knowledge topic equal to 1. This means that one of the fundamental prior knowledge topics becomes the 1st fundamental output topic in these 22 document sets. In these 22 comment behaviors, there are 7 times that the 1st fundamental output topic is also in the fundamental input topics.

To further explore whether the topic resonance relates to other factors, we design a hypothesis test. In the test, the null hypothesis is that the resonances between output topics and input or prior knowledge topics are irrelevant to gender, discipline, seniority, or commentary length. The test results cannot deny this null hypothesis, as in Table 3.

Table 3

Independent test on fundamental topic resonance.

	Gender		Discipline		Seniority		Commentary length
	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)
Resonance	0.42	χ0.012(1) = 6.63	22.10	χ0.012(23) = 41.64	37.21	χ0.012(41) = 64.95	16.97	χ0.012(15) = 30.58

One way analysis of variance (ANOVA) on gender, discipline, seniority and commentary length also suggest that there are no significant resonance differences between different groups across these four factors, as in Tables 4, 5, 6 and 7 respectively. Note that if topic resonance occurs in a sample, 1 is added into the sum item while otherwise 0 is added.

Table 4

One-way ANOVA on gender.

Summary
Groups	Samples	Sum	Mean	Variance
Male	71	57	0.803	0.16
Female	22	19	0.863	0.12
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.062	1	0.062	0.409	0.524	3.946
Within group	13.830	91	0.152
Total	13.892	92

Table 5

One-way ANOVA on discipline.

Summary
Groups	Samples	Sum	Mean	Variance
Surgery	35	25	0.714	0.21
Medicine	15	13	0.867	0.12
Psychology	14	12	0.857	0.13
etca	29	26	0.897	0.10
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.612	3	0.204	1.37	0.258	2.707
Within group	13.280	89	0.149
Total	13.892	92

The etc group consists of samples from all the other 21 disciplines except the above three.

Table 6

One-way ANOVA on seniority.

Summary
Groupsa	Samples	Sum	Mean	Variance
High	47	39	0.830	0.14
Low	46	37	0.804	0.16
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.015	1	0.015	0.099	0.754	3.946
Within group	13.877	91	0.152
Total	13.892	92

The whole samples are divided into High Seniority group and Low Seniority group by median seniority (23.5 years).

Table 7

One-way ANOVA on commentary length.

Summary
Groupsa	Samples	Sum	Mean	Variance
Long	46	41	0.891	0.099
Short	47	35	0.744	0.194
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.015	1	0.015	0.099	0.754	3.946
Within group	13.877	91	0.152
Total	13.892	92

The whole samples are divided into Long Commentary and Short Commentary group by median commentary length (391 tokens).

Combing the above 2 subsections, we found that the majority (76/93 = 81.72%) of the 1st fundamental output topics are either from fundamental input topics or fundamental prior knowledge topics. In summary, among the topics in prior knowledge spectrum, the topic which dominates the input paper would most likely be the 1st fundamental topic in output paper.

Correlations between input paper and prior knowledge at topic level

In each document set, a correlation coefficient between the input topic distribution and the prior knowledge topic distribution was calculated. Among these calculated 93 correlation coefficients, 87 (93.55%) correlation coefficients are negative (this will be studied further in Study 2). Moreover, the bottom 21 (22.58%) correlation coefficients (-0.88 ∼ -0.98) are significant (p < 0.05).

When the boxplots of correlation coefficients in the above three subsections are put together, a very interesting correlation coefficient shift is emerging in Fig. 7.

Fig. 7

Boxplots of correlation coefficients between topic distributions of input, prior knowledge and output.

The left boxplot entails the correlation coefficients between the input and prior knowledge topic distributions. They are primarily negative. This implies that a commenter's prior knowledge spectrum tends to have a bias towards the topics which are less emphasized in input paper. The right boxplot is for the correlation coefficients between the input and output topic distributions. They are primarily positive. It implies that the output words are re-allocated to echo the salient input topics. In the statistics sense, the correlation coefficients' shifting from primarily negative to primarily positive shows an imperceptible motivation hidden behind the commenter's wording and phrasing: to catch up with the input pace.

Biases in the input paper, output paper, and prior knowledge

The coefficient of variations of input, output, and prior knowledge topic distributions are 0.95 (SD = 0.39), 1.35 (SD = 0.44) and 0.52 (SD = 0.40) respectively. In 69 out of 93 commentaries (74.19%), their coefficients of variations of topic distributions are higher than the ones of input papers. For these 69 commentaries, the coefficients of variations of topic distributions increase 112.14% on average. For the rest 24 commentaries, the coefficients of variations of topic distributions decrease only 18.81% on average. It implies that the topic distributions in commentaries tend to be more biased than those in original papers.

The test suggests that “output topics being more biased than input topics (here after, being more biased)” is independent from gender, discipline, seniority, or commentary length, as in Table 8.

Table 8

Independence test for being more biased on output topics.

	Gender		Discipline		Seniority		Length
	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)	χ02	χ0.012(⋅)
More biased	2.88	χ0.012(1) = 6.63	23.30	χ0.012(23) = 41.64	29.57	χ0.012(41) = 64.95	21.32	χ0.012(15) = 30.58

One-way ANOVA on gender, discipline, seniority and commentary length also suggest that there is no significant “being more biased” difference between different groups across these four factors, as in Tables 9, 10, 11 and 12, respectively. Note that if the output topic distribution is more biased than the input topic distribution in a sample, 1 is added into the sum item, while otherwise 0 is added.

Table 9

One-way analysis of variance (ANOVA) on gender.

Summary
Groups	Samples	Sum	Mean	Variance
Male	71	55	0.775	0.18
Female	22	13	0.591	0.25
ANOVA
Source of Variance	SS	df	MS	F	p-value	F_crit
Between Groups	0.567	1	0.567	2.913	0.091	3.946
Within Group	17.713	91	0.195
Total	18.280	92

Table 10

One-way analysis of variance (ANOVA) on discipline.

Summary
Groups	Samples	Sum	Mean	Variance
Surgery	35	27	0.771	0.18
Medicine	15	11	0.733	0.21
Psychology	14	8	0.571	0.26
etca	29	22	0.759	0.19
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.436	3	0.145	0.725	0.540	2.707
Within group	17.844	89	0.200
Total	18.280	92

The etc group consists of samples from all other 21 disciplines except the above three.

Table 11

One-way analysis of variance (ANOVA) on seniority.

Summary
Groupsa	Samples	Sum	Mean	Variance
High	47	35	0.745	0.19
Low	46	33	0.717	0.21
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.017	1	0.017	0.086	0.770	3.946
Within group	18.262	91	0.200
Total	18.280	92

The whole samples are divided into High Seniority group and Low Seniority group by median seniority (23.5 years).

Table 12

One-way analysis of variance (ANOVA) on commentary length.

Summary
Groupsa	Samples	Sum	Mean	Variance
Long	46	30	0.652	0.232
Short	47	38	0.808	0.158
ANOVA
Source of variance	SS	df	MS	F	p-value	F_crit
Between groups	0.568	1	0.568	2.919	0.091	3.946
Within group	17.711	91	0.194
Total	18.280	92

The whole samples are divided into Long Commentary and Short Commentary group by median commentary length (391 tokens).

In summary, these results suggest that an output paper tends to be more biased than an input paper. In addition, this tendency is similar in terms of gender, discipline, seniority and output length.

Study 2: input & pseudo prior knowledge

In the above subsection, it was found that most (93.55%) of the correlation coefficients between input and prior knowledge topic distributions are negative and some of them (22.58%) are significantly negative (-0.88 ∼ -0.98, p < 0.05). This may seem counterintuitive since input and prior knowledge topic distributions are supposed to be positively related at first glance. To address the second question (Q2), a control experiment was set up to investigate the correlation coefficients between input and pseudo prior knowledge topic distributions. We expected that by comparing the results in the above subsection and in this subsection, more details about the relationships between input and prior knowledge topic distributions will be uncovered.

It is worth to note that Subsection “Materials” has stated the material used in this subsection is the same as in Study 1 except that the prior knowledge is synthetically pseudo. The raw data for Study 2 are presented in Supplementary Table S2. By using the same methods in the above subsection, we found the following facts.

Coincidentally but not identically, 93.55% correlation coefficients between input and pseudo prior knowledge topic distributions are also negative. However, Fig. 8 shows that the boxplot of correlation coefficients between input and pseudo prior knowledge topic distributions is significantly different from the ones between input and real prior knowledge topic distributions. The details are as follows.

Fig. 8

Correlation coefficients between input & pseudo prior knowledge topic distributions vs. correlation coefficients between input & real prior knowledge topic distributions.

The correlation coefficient between input and pseudo prior knowledge topic distributions is −0.542 (SD = 0.296). While the correlation coefficient between input and real prior knowledge topic distributions in Study 1 is −0.666 (SD = 0.331). One side t-test verified that the correlation coefficients between input and pseudo prior knowledge topic distributions are greater than the ones between input and real prior knowledge topic distributions at the 0.5% significant level. Note that this test was conducted under an assumption that both input-prior knowledge correlation coefficients in Study 1 and Study 2 are from normal distributions with unknown and unequal variances.

This shows that compared with the pseudo prior knowledge, the real prior knowledge tends to be more significantly correlated to the input though their correlation coefficients are both negative. Note that we inferred in the above subsection that a commenter's prior knowledge spectrum tends to have a bias towards the topics which are less emphasized in input paper. By comparing the above control experiment result with our observation in the above subsection, it can be inferred that a commenter's prior knowledge topic distribution is an intended supplement to the input topic distribution in general cases. Commenters are more prone to associate references which focus on the non-dominant topics when they are stimulated by the input paper.

Study 3: predicting topic resonance

Study 1 and Study 2 tested topic resonance in a large scale either straightly or conversely. To address the third question (Q3), in this section, we would like to see if the resonance is predictable with some of the findings in Study 1 and 2. These findings are as follows.

Topic resonance exist commonly between fundamental input topics and the 1st fundamental output topics or fundamental prior knowledge topics and the 1st fundamental output topics (the first two subsections in Study 1).

Among the topics in prior knowledge spectrum, the topic which dominates the input paper would most likely be the 1st fundamental topic in output paper (the first two subsections in Study 1).

Commenters are more prone to associate references which focus on the non-dominant topics when they are stimulated by the input paper (Study 2).

Finding (1) infers that the 1st fundamental output topic may be predicted by the fundamental input topics or fundamental prior knowledge topics. Finding (2) implies that prior knowledge can be used as a base for the prediction. Finding (2) also implies that fundamental input topics are important predictors for the 1st fundamental output topic. Finding (3) suggests that input and prior knowledge topics may interact with each other.

In this subsection, we first model the output topic distribution, especially on the dimension that produces the first fundamental input topic. Then we use an inference developed from the model to predict the topic resonance between input and output paper, by estimating the output topic weight on the 1st fundamental input topic dimension. This prediction not only serves as an application of the above findings, but also serves as a third-party check on our topic resonance findings.

Modeling the output topic distribution

We use a Hebbian-like learning model (Munakata and Pfaffly, 2004) to model the relationship between the input, output and prior knowledge topic distributions. A basic Hebbian learning rule takes the following form:where denotes the change in weight from unit i to unit j, a and a denote the activation levels of units i and j respectively, and denotes the learning rate - how quickly the weights change in response to unit activations. Eq. (10) adopts multiplication to represent the interactions between units i and j.

In this study, we assume that one topic is one “unit” in the basic Hebbian learning rule. Note that I, O, and B are estimations of the k-th input, output and prior knowledge topic weight in Section “Example” respectively. Since these variables indicate topic weights, they are equivalent to “activation levels of units” in the basic Hebbian learning rule. Therefore, we propose a Hebbian-like learning model to represent the relationship between I, O, and B, as in Eq. (11). Eq. (11) also matches the implications from the above three findings.

A simple linear model is used as a benchmark for comparison, as in Eq. (12).

We also assume that i_max is the index for the maximum I (the 1st fundamental input topic) for a certain commenter. That is,

We begin with exploring the relationship between the i_max-th input, output and prior knowledge topic weights. The relationships between topic weights on other dimensions are explored in the same way. Since we already know the values of I, O, and B, can be calculated according to Eq. (13). That is,

For a certain commenter c, we have a according to Eq. (14). For the 93 commenters, 93 values can be sorted in an ascending order. The last five values (also the highest five values, whose range is (54.3, 5243.7)) are excluded since they are believed outliers. Among the remaining 88 values, the last three values are also excluded since they are beyond the mean plus 3 times of standard deviation (3 criterion). The final calculated 85 values are plotted in Fig. 9.

Fig. 9

Calculated sorted in ascending order.

These 85 values are divided into 2 parts at first. The first part (36 values) is negative while the second part (49 values) is positive. Then the first part is split into two subgroups evenly, 18 values each. The second part is cut into three subgroups. The numbers of values in these three subgroups are 19, 20 and 10 respectively. The values in each subgroup are thus on a similar order of magnitude: [−1.00, −0.77], [−0.74, −0.085], [0.036, 0.75], [1.00, 3.39] and [3.75, 12.74].

For each subgroup, we use Matlab R2015b's built-in non-linear/linear regression functions (fitnlm/fitlm) to test the above Hebbian-like model (Eq. (11)) and simple linear model (Eq. (12)) respectively. The assessments on Hebbian-like model and simple linear model are presented in Table 13.

Table 13

Detailed model assessments on the maximum weighted input topic dimension.

Subgroup	Assessment indicator	Hebbian-like model	Simple linear model
1	Model parameter	γ_i_max=−0.92747	(bk0,bk1,bk2) = (−0.0068,0,0.082)
	R2	0.282	0.288
	F-statistic vs. zero model	17	6.48
	p	<0.001	0.0216
2	Parameter	γ_i_max=−0.36146	(bk0,bk1,bk2) = (0.0098,0,0.62)
	R2	0.697	0.697
	F-statistic vs. zero model	189	36.9
	p	<0.001	<0.001
3	Parameter	γ_i_max=0.29799	(bk0,bk1,bk2) = (0.12,0,1.12)
	R2	0.795	0.818
	F-statistic vs. zero model	679	76.4
	p	<0.001	<0.001
4	Parameter	γ_i_max=1.5272	(bk0,bk1,bk2) = (0.28,0,1.79)
	R2	0.528	0.662
	F-statistic vs. zero model	328	35.3
	p	<0.001	<0.001
5	Parameter	γ_i_max=5.8898	(bk0,bk1,bk2) = (0.23, 0, 5.29)
	R2	0.662	0.747
	F-statistic vs. zero model	170	23.6
	p	<0.001	0.00126

In Table 13, R2 is the proportion of the total sum of squares explained by the model. In the model, the larger R2 shows the greater variability. F-statistic vs. zero model indicates whether our Hebbian-like model fits the data better than zero model. F-statistic vs. constant model indicates whether the simple linear model fits the data better than constant model.

According to the p value, the simple linear model fails to reject null hypothesis in Subgroup 1 and 5 at the significant level of 0.001. In Subgroup 2, the F-statistic vs. zero model value of Hebbian-like model is much larger than the F-statistic vs. constant model value of simple linear model, indicating that the Hebbian-like model fits the data better. In Subgroup 3 and 4, it seems that it is a tie between the simple linear and Hebbian-like model. However, according to Akaike information criterion (Vrieze, 2012), the Hebbian-like model is better since it has less parameters. More importantly, all b values of simple linear model are zero, indicating that input does not influence output at all, which is against our previous observations.

To summarize, at the maximum I dimension, the Hebbian-like model outperforms simple linear model. The comparison of Hebbian-like model and simple linear model on all the topic dimensions is summarized in Table 14. These dimensions are sorted by topic weights instead of their natural indexes.

Table 14

Model assessment on all five input topic dimensions.

Items		Input topic dimension
		1st weighted	2nd weighted	3rd weighted	4th weighted	5th weighted
Final samples		85	88	86	86	85
Subgroups		5	6	6	7	10
SGNFa subgroups	Hebbian	5	4	4	4	6
	Linear	3	2	1	2	3
Samples in SGNF model	Hebbian	85	66	67	62	47
	Linear	47	37	17	51	25
R2 scope in SGNF model	Hebbian	[0.282, 0.795]	[0.256, 0.537]	[0.290, 0.806]	[0.287, 0.905]	[0.453, 0.950]
	Linear	[0.697, 0.818]	[0.574, 0.690]	0.695	[0.519, 0.754]	[0.748, 0.998]

SGNF is short for significant. A SGNF subgroup refers to the subgroup's model is at the significant level. The significant level is set p < 0.001 for all five dimensions except the fifth. The significant level is set p < 0.01 for the fifth weighted input topic dimension.

Table 14 shows that the Hebbian-like model outperforms the simple linear model on all the dimensions, particularly the former covers more samples or subgroups on the 1st and 2nd weighted input topic dimensions (a.k.a., the 1st and 2nd fundamental input topic dimensions). Here, “outperform” refers to doing better on the number of subgroups whose model is at the significant level. The R2 in Table 13 is only for reference in terms of integrity. A model will be considered valid unless its R2 is far low. It can be also seen that as the input topic goes into less weighted dimensions (4th and 5th), the number of samples covered by the model at the significant level becomes less. This indicates that the generalizabilities of both models degrade when the input topic weight declines, though the Hebbian-like model degrades slower than the simple linear model.

Analyses and prediction

According to Eq. (12), we have

If we bring Eq. (14) into Eq. (15), we have

Note that the input, output and prior knowledge topic weight vectors are normalized by their maximum elements, as shown in Fig. 5. Hence, and . Therefore, on the maximum input topic weight dimension, Eq. (16) can be simplified as

If <0, then (1+) <1, considering , then <1. Therefore, will not have any chance to be the maximum element in the output topic weight vector since the maximum element would equal to 1 strictly due to the normalization approach mentioned above Fig. 5.

On the contrary, if > 0, then (1+) > 1. Although we still have , will have the chance to be the maximum element in the output topic weight vector. The larger is, the larger the chance is. In fact, in Subgroup 5 of Table 13 where = 5.8898, will reach 1 only if is larger than 0.1451. This is a relatively easy condition to satisfy.

Actually, we use the above inference to predict the topic resonance. We follow Eq. (17) to calculate as the predicted value in Subgroup 3–5 since their values are positive. If the predicted value is larger or equal to 1, or its confident interval contains 1 (Matlab prediction function will automatically display it), then a topic resonance is predicted to occur at the dimension of between input and output topic weight vectors. As a result, compared with the observed flag of topic resonance, the prediction accuracy is 16 successful times over 18 predicting times (88.9%).

Considering the definition of topic resonance illustrated in Section “Example”, if the 1st fundamental output topic weight is on the same dimension with either of the 1st or 2nd fundamental (input or prior knowledge) topic weight, it would be accounted as a topic resonance. Therefore, this above prediction only covers a portion of topic resonances, a.k.a, the 1st fundamental output topic weight is on the same dimension with the 1st fundamental input topic weight. Predictions on other complicated topic resonances (the 1st fundamental output topic weight is on the same dimension with the 2nd fundamental input topic weight) will be studied in the future.

Topic resonances may also have cues to observe, such as the titles of references cited by the commenter, or the author names of references cited by the commenter. Our attempts on predicting the topic resonance cues are presented in the next subsection.

Predicting topic resonance cues

References of a commentary and their authors can be seen as two kinds of topic resonance cues. The most exciting application would be to predict the specific references cited by a commentary based on the commenter's publication history. However, it is rather difficult to do so in simple Mathematics. Instead, we managed to predict the authors of references in the commentary in an easy way. In this section, we would like first to explain the reason why it is difficult to predict specific references. Then, we illustrate how to predict the authors of references in a commentary. Either the explanation or the illustration can confirm our topic resonance findings from different angles.

Predicting the references in a commentary

In this subsection, we attempt to predict the references in a commentary based on two kinds of prior knowledge with different scopes.

Based on a broad prior knowledge

In this subsection, we reuse the illustrative sample in Section “Example”, a.k.a., the input paper (Wood, 2008) and the output paper (Blaszczynski, 2008). We name this input paper as W's paper, and this output paper as B's commentary. B's publications can be traced back as early as 2001 on his website. B's commentary was published in the year of 2008. There are 44 journal papers on B's publication list from 2001 to 2008. Five papers are excluded from this study: four unrelated papers and one related paper but submitted after the month when the B's commentary was published. The left 39 papers are in the related domain of W's paper (pathological gambling behavior research). These 39 papers, together with their references, are assumed B's self-reported prior knowledge when he planned to output. However, B's vision may not only be restricted by these papers since he may look for completely new references.

The aforementioned 39 papers list 1294 items in their reference sections, or one paper cites 33.2 (SD = 22.6) references. These 1294 items consist of 915 unique references. There are 185 (20.2%), five (0.55%) and two (0.22%) references appearing in more than one, 10 and 15 (the maximum number) papers respectively. It shows that the citation frequency distribution of references is pretty skewed. It looks easy for us to predict the references if we assume B's previous highly cited references would also be cited in his commentary. Unfortunately, neither references nor the authors of references in B′ commentary (i.e., Griffiths and Wood) are among B's most citing ones (Table 15).

Table 15

Top 12 references cited by B in his 39 journal papers.

Authora (Year)	The times of being cited	Rank
Productivity Commission (1999)	15	1
Lesieur, Blume (1987)	15	1
National Research Council (1999)	14	3
Ladouceur, Walker (1996)	11	4
American Psychiatric Association (1994)	11	4
American Psychiatric Association (2000)	9	6
Sylvain, Ladouceur, Boisvert (1997)	8	7
Blaszczynski, Nower (2002)	8	7
American Psychiatric Association (1987)	8	7
Petry (2004)	7	10
Jacobs (1986)	7	10
Walker (1992)	6	12
Shaffer, Hall (1996)	6	12
McConaghy, Armstrong, Blaszczynski, Allcock (1983)	6	12
Blaszczynski, Steel, McConaghy (1997)	6	12

Only the family names of individual authors are retained, due to the consideration of privacy respect.

This case implies that in such a broad prior knowledge, we cannot predict any references or authors of references in a commentary merely based on the citation frequency distributions of B's all past publications. However, if we narrow all his past publications to the input paper's focus (Internet/video gaming addictive behavior) rather than the above relatively broader area B used to contribute to, the prediction accuracy will be significantly improved, as the following subsection.

Based on a refined prior knowledge

There are six papers among the aforementioned 39 papers on the Internet/video gaming addictive behavior, the same specific subject with input paper. They are selected by keywords. Each of the six papers cites 23.8 (SD = 10.9) references. There are 143 items in the reference sections and 133 unique references. Among these six closely related papers, one paper's references cover both two references of the commentary, and another paper's references cover one reference of the commentary. Comparing with above subsection, this is a significant improvement since we have narrowed the search area down from 915 unique references to 133 unique references, or 14.5% of original search area. We continue to use keywords to filter out 133 unique references. The filtering rule is quite simple. If the title of a reference contains any keywords such as “Internet”, “video”, “gaming”, and “addictive” and so on, this reference will remain in the final search area. Ultimately, 35 references (3.8% of original search area) remain.

However, it is rather difficult to go further because the occurrence times of references are quite even (note that 133 unique references appear 143 times in total). More essentially, although the extended SCVB0 algorithm can efficiently decompose a reference into multiple topics, it is hard to tell which parts of a reference the commenter intends to quote by mathematical weight numbers. The dominant topic of a reference may not always be the most desired part by the commenter. The less emphasized topic of a reference could also be a source that the commenter intends to point to as well, let alone many commenters just cite a sentence or an evidence in a reference.

Nevertheless, we can predict the authors (i.e., Griffiths or Wood) of references in a commentary using simple Statistics, still based on a refined prior knowledge narrowed by keywords, i.e., among the authors of the final 35 references. This will be presented in next subsection.

Predicting the authors of references in a commentary

Following the previous subsection, top four authors of references in B's 35 refined references are listed in Table 16.

Table 16

Top four authors of references in B's 35 refined references.

Authora	The times of appearing in reference sections	Rank
Griffiths, M. D.	8	1
Phillips, J. G.	4	2
Young, K.	2	3
Dickerson, M. G.	2	3

The rest authors only appear once in reference sections.

The highest cited author (Griffiths) in Table 16 is indeed one of the two authors of references in B's commentary. This example implies that the authors of references in commentaries are actually predictable after three steps illustrated above. To summarize, these steps include a) collecting self-report prior knowledge (a commenter's publications and their references); b) refining prior knowledge by keywords; and c) ranking highly cited authors among final references. The higher the rank, the bigger the chance.

Recalling the above three simple steps, we actually assume that the some highly cited authors of references in a commenter's previous closely input-related publications would continue to appear in the one's commentary on the input paper. In this subsection, the highly cited authors are defined top five cited authors. Among the 93 commenters in Subsection “Materials”, 86 commenters' publication lists are found, there are 45 (52.3%) samples supporting the above assumption. Among these 45 samples, 30 (66.7%) top one cited authors of references appearing in a commenter's previous closely input-related publications will continue to appear in the one's commentary on the input paper.

Besides these 45 samples, there are another 25 samples where at least one of the lowly cited (beyond top five cited) authors of references in a commenter's previous closely input-related publications appear in the one's commentary on the input paper. Only the last remaining 16 commenters cite completely new authors' publications in their commentaries.

It is worth to note that though we did not use simple Math to predict the exact references cited by a commentary, we obtained a related byproduct in this subsection. We found that after the two steps (collecting and refining), there are 25 samples (29.1 % of the above 86 commenters) whose final search area contains at least one reference of the commentary. We believe this respectable proportion can be enlightening for the future study.

Discussion

Traditionally, it was assumed that the concepts and propositions in the discourse representation resonate with the ones in reader's prior knowledge. The strength of such resonance can be abstracted into a function, whose independent variables describe how well the concepts and propositions are closely connected to the input (Myers & O'Brien, 1998). This closeness depends on the overlap of both semantic features and contextual features derived from a discourse model (O'Brien et al., 1998). Readers are assumed to construct their comprehensions at two levels of text representations: the text itself, and what the text is about, i.e., the topic level (Humphreys et al., 1989; O'Brien et al., 1998). Gaultney (1995) even observed that boys who are both poor readers and baseball experts perform better reading comprehension when they are trained with baseball stories.

However, a readymade tool was lacking to visualize topic resonance before topic modelling was developed, though academic communities realized such features of semantic resonances very early. Meanwhile, the experiment corpus needs cautious design. If a narrative is too short, it is often difficult for experiment participants to tell apart the text itself and what the text is about. However, if a narrative is too long, it is always easy to activate readers' unknown prior knowledge.

Since prior knowledge is crucial in topic resonance studies, and it is rather difficult to retrieve the prior knowledge of receivers (audience or readers) in an ordinary conversation/discourse, we turn to a special scenario: academic commentaries. The advantage of this quasi-experiment setting is that the academic commenters are required to cite every reference which makes them write so, due to the ethical principles. And these references can be regarded as part of commenters' prior knowledge on the related themes. To our best knowledge, this current work is one of the first efforts which exploit academic commentaries to study topic resonance.

The tactic behind our measures can also be found in Murdock et al. (2017). In their work, Murdock et al. followed Darwin's book reading list to locate 665 full-text available books. They used Kullback-Liebler Divergence, a cognitively-validated, information-theoretic measure of relative surprise to examine this preeminent scientist's reading choices. Their purpose is to observe how a knowledge-seeker balance between exploitation of past discoveries and further exploration.

Different from their studies, we not only concern what a commenter reads (consumption) but also what a commenter writes (production). By observing the interactions between input, output and prior knowledge at the topic level, we tried to address 3 vital questions in our studies: does topic resonance exist commonly? Which components in the prior knowledge spectrum will be most likely activated? And is there a model to quantify the topic resonance pattern?

We found that the topic resonance exists commonly. And it is independent from a commenter's gender, discipline, seniority, and output length. This finding has tight connections with other studies. Cerdna et al. (2013) indicated that students are influenced by perspective instructions when they are reading multiple documents. In their work, participants were required to take different opinions before they started to pick up evidences. Their perspective instructions can be seen as the very short versions of input papers. The influence they exploring can be regarded as some sort of resonance investigated in our work. While we not only confirm their conclusions at the topic level, but also reveal that topics in prior knowledge can also play a heuristic role in shaping the output paper. Anmarkrud et al. (2013) demonstrated that readers tend to connect the knowledge they have known with those they are about to know. They actually observed focus shifts at the document level. Our study used topic correlation to reveal topic shift from prior knowledge to output paper.

We found that the topic in prior knowledge spectrum, which dominates the input paper, will be most likely activated to be the 1st fundamental topic in the output paper. This finding can be summarized as “the topics that resonate together, link (through words) together”, which could be regarded as a mirror image of the “Hebbian plasticity” hypothesis (Hebb, 1949) in semantic domain.

We also found that a Hebbian-like model outperforms a simple linear model significantly on modeling the quantitative relationships between input, output and prior knowledge topic distributions. The explanation is that input and prior knowledge topics may have non-linear interactions as observed while simple linear model ignores this kind of interactions by simply adding their semantic representations together. Recently, Johansena et al. (2014) provided direct evidence which connects the Hebbian plasticity in neuron domain with behavioral associative memory formation in semantic domain. Although it seems plausible to place their finding and our Hebbian-like topic interaction modeling on two ends of a bridge across over biology domain and human behavior domain, we believe there are many middle dots out there to explore and connect.

Another detail about topic resonance we found is that the output topic distribution tends to be more biased than the input topic distribution. This implies that commenters tend to narrow their attentions onto fewer topics. This is another evidence for an empirical law that people tend to filter the input information companied with simplification. At this point, our finding is consistent with Crossley and McNamara (2016) and Peenlen and Kastner (2014). The former work demonstrated that text simplification leads to greater propositional recall. And the latter work believed that attentional templates are shaped by object familiarity to efficiently select relevant objects from cluttered environments.

Conclusions

In this paper, we extended SCVB0 to measure the fluctuation patterns of the latent topics in 93 document sets of original papers, prior knowledge and the corresponding commentaries. Original papers are probes to invoke commenters' feedbacks (commentaries). Topics are in fact the semantic components of these probes, feedbacks and contexts (prior knowledge). Our work actually revealed the relationship between semantic components of probes, feedbacks and contexts. In this conclusion section, semantic components, probes, feedbacks and contexts will be used to broaden implications from our contributions on topic resonances among input, output and prior knowledge.

First of all, we found that semantic resonances commonly exist between fundamental components of a participant's probes and feedbacks. And this resonance is independent of the participant's gender, discipline, seniority and feedback length. We also observed that among the components in context spectrum, the component which dominates the probe would most likely be the 1st fundamental component in the feedback. In addition, the feedback components tend to be more biased than the probe components, and this tendency is also independent of individual gender, discipline, seniority and feedback amount. However, we have not yet found any significant correlations between feedback bias and semantic resonance.

Secondly, we found that the correlation coefficients between the probe and context components are primarily negative. To verify this counter-intuitive observation, pseudo components were synthetized in a control experiment. We found that the correlation coefficients between the probe and context components are significantly less negative. The implication from these two experiments would only be that a participant's context spectrum tends to have a significant bias towards the components which are less emphasized in the probe. Or individuals are more prone to associate context which focus on the non-dominant components when they are invoked by the probe.

Thirdly, we found that the probe-context, context-feedback and probe-feedback component distribution correlation coefficients shift from primarily negative to primarily positive. In the statistical sense, this correlation coefficient shifting indicates an imperceptible pattern hidden in a participant's semantic processing behavior: to follow the lead of a probe. Or the output words are re-allocated by the commenter to echo the salient input topics.

Last but not least, we found that a Hebbian-like model can be used to model the quantitative relationship among the semantic components of probe, feedback and context and to predict the semantic resonance among major components of probe, feedback and context.

To a wider sense, our findings support the metaphor that information exchange between humans is similar to substance exchange between organisms and the environments. Organisms tend to absorb the most suitable component from the stimulus of environments and adapt to the surrounding (e.g., chameleons). Although academic experts may present their own opinions by free will, their discourses seem to share a common pattern. That is, an individual tends to extract the fundamental component from the probe, and activate the corresponding ones in his or her perceived context. This kind of activation mode tends to drive the fundamental probe components to dominate the feedback. The semantic resonance could be regarded as an “adaption” to the “stimulus” in human semantic domain.

One of the differences between information exchange and natural substance exchange is that the former is conscious while the latter is not. However, we found that this consciousness is two-fold. One is a presented a complementary way: the probe-context component distribution correlations are mostly negative. The other is that the probe-feedback component distribution correlations are positive.

Our future work includes a deeper examination on the interaction patterns among semantic representations of probe, feedback and context. More specifically, we may attempt to observe the topic-level patterns on how clients comment on the content of links they forwarded on their social network accounts, or how students interact with each other in MOOC communities. For such smaller text samples, a more appropriate method, e.g. non-negative matrix factorization, may help us to find right semantic components of a message as illustrated by Lee and Seung (1999).

Declarations

Author contribution statement

Tai Wang: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Zongkui Zhou: Conceived and designed the experiments.

Xiangen Hu: Contributed reagents, materials, analysis tools or data; Wrote the paper.

Zhi Liu: Performed the experiments; Wrote the paper.

Yi Ding: Analyzed and interpreted the data; Wrote the paper.

Zhiqiang Cai: Analyzed and interpreted the data.

Funding statement

This work was supported by the National High Technology R & D Program of China (2017YFB1401300, 2017YFB1401303), Natural Science Foundation of China (31600918, 61702207) and Ministry of Education in China Project of Humanities and Social Sciences (15YJC880088).

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.