Differential ability of network and natural language information on social media to predict interpersonal and mental health traits.

OBJECTIVE: Previous studies have shown that digital footprints (mainly Social Networking Services, or SNS) can predict personality traits centered on the Big Five. The present study investigates to what extent different types of SNS information predicts wider traits and attributes.
METHOD: We collected an intensive set of 24 (52 subscales) personality traits and attributes (N = 239) and examined whether machine learning models trained on four different types of SNS (i.e., Twitter) information (network, time, word statistics, and bag of words) predict the traits and attributes.
RESULTS: We found that four types of SNS information can predict 23 subscales collectively. Furthermore, we validated our hypothesis that the network and word statistics information, respectively, exhibit unique strengths for the prediction of inter-personal traits such as autism and mental health traits such as schizophrenia and anxiety. We also found that intelligence is predicted by all four types of SNS information.
CONCLUSIONS: These results reveal that the different types of SNS information can collectivity predict wider human traits and attributes than previously recognized, and also that each information type has unique predictive strengths for specific traits and attributes, suggesting that personality prediction from SNS is a powerful tool for both personality psychology and information technology.

INTRODUCTION

Recent developments in cyberspace, in particular, social networking services (SNS), are rapidly changing our lives including how we communicate and make decisions. Moreover, using SNS information, researchers can access huge digital footprints to study individual differences in behaviors.

Knowing the personality traits and attributes of other people is the core of our communication and decision making (Funder, 2012; Weiner & Greene, 2008). For instance, personality judgment is required for deciding who to make friends with, what to try to sell to consumers, and even whether a person has a specific psychiatric disorder (Vazire & Carlson, 2011). Considering the importance of personality judgments, it would be a big technological advance for both personality psychology and information technology if specific types of SNS information (e.g., network structure and language expressions) are useful for predicting specific personality traits and attributes (e.g., depression tendency and socioeconomic status) of the users.

The extent of the personality traits considered in previous prediction studies is, however, limited and centered on the Big Five inventory (Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness) (Azucar, Marengo, & Settanni, 2018; Park et al., 2015; Youyou, Kosinski, & Stillwell, 2015). Although the Big Five is important for explaining human personality, vast amounts of psychological studies have demonstrated that personality traits outside the Big Five are also important (Roberts et al., 2017; Weiner & Greene, 2008).

Because of its importance to the cognitive and medical sciences and also practical benefits, we are particularly interested in SNS‐based personality predictions in the context of social behavior, decision making, and metal health. For this work, it is important to clarify the extent to which different types of SNS information predict personality traits and attributes. Indeed, several works have demonstrated personality traits and attributes other than Big Five can be predicted from SNS information. One important study based on Facebook data showed that various attributes such as satisfaction with life, intelligence, and drug use can be predicted from a user's history of likes (expressions of positive association with online content) (Kosinski, Stillwell, & Graepel, 2013). These traits are important for measuring life outcomes (Kuncel & Hezlett, 2010; Plomin & Deary, 2015) and drug addictive use (Kotov, Gamez, Schmidt, & Watson, 2010). Other studies also reported that a few psychiatric traits such as depression are predictable from a user's Facebook texts (Eichstaedt et al., 2018; Guntuku, Yaden, Kern, Ungar, & Eichstaedt, 2017). However, no previous study has addressed the predictive link between different types of SNS information and an intensive set of personality traits and attributes.

In the present study, we introduce four key human trait (and attribute) sets that are deeply related to social behavior, decision making, and mental health: mental health, behavioral economics, empathizing–systemizing, and inhibition/activation. Psychiatric disorders such as depression, schizophrenia, and delusion (American Psychiatric Association, 2013) and mental health factors such as stress (Lazarus, 2000) are a pressing priority in modern society. Furthermore, it is important to investigate this set of disorders and factors simultaneously (American Psychiatric Association, 2013), because DSM‐5 defines these diseases as multi‐dimensional symptoms. Behavioral economics indices, such as time discount (Green, Fry, & Myerson, 1994), risk aversion (Charles & Laury, 2002), and socioeconomic status (Krieger, Williams, & Moss, 1997), are known to influence decision making. Empathizing–systemizing theory suggests that individuals can be classified along two dimensions (Baron‐Cohen, Knickmeyer, & Belmonte, 2005): empathy, defined as the ability to recognize and respond to another person's mental state, and systemizing, defined as the drive to analyze or build a rule‐based system. The behavioral inhibition/activation systems (BIS/BAS) are well‐known personality theories derived from a neuropsychologically grounded motivational system (Carver & White, 1994; Gray, 1970). BIS is related to sensitivity to punishment and avoidance motivation, whereas BAS is associated with sensitivity to reward and approach motivation. Gray hypothesized these two distinct, functionally independent systems for behavioral regulation and motivation. Importantly, empathizing–systemizing and behavioral inhibition/activation theories play critical roles in human social behavior and decision making and help explain several deficits in communication and decision making.

To apply SNS information to personality predictions, previous studies have used four types of features: network, time, and two natural language‐based features (Eichstaedt et al., 2018; Guntuku et al., 2017; Kosinski et al., 2013; Park et al., 2015; Youyou et al., 2015). For network information, an early study proved that just three network features (following number, followers number, and listed counts number) on Twitter can predict Big Five personality traits (Quercia, Kosinski, Stillwell, & Crowcroft, 2011). As explained above, Kosinski et al. (2013) conducted a study to demonstrate that various human traits including the Big Five can be predicted from a user's history of likes on Facebook. Regarding time information, Big Five traits were predicted by the time regularity and contact frequency of smartphone usage (de Montjoye, Quoidbach, Robic, & Pentland, 2013), although SNS information was not investigated. The timing of audiovisual non‐verbal cues on YouTube was also able to predict the Big Five personality traits (Biel & Gatica‐Perez, 2013). For natural language information, the frequency of words and phrases (i.e., the frequency of “I,” “you,” “me too,” “do you”) on Facebook showed successful predictions of Big Five personality traits (Park et al., 2015; Schwartz et al., 2013). Eichstaedt et al. (2018) also demonstrated that word usage on Facebook can predict a change in depression symptoms. In addition, meta‐word information such as the average word length and the ratio of emotional words were also predictive of Big Five traits and depression tendency (Golbeck, Robles, Edmondson, & Turner, 2011; Guntuku et al., 2017). Finally, a few research studies have attempted to integrate network, time, and natural language information for personality predictions and demonstrated that the combination of these types of information can predict Big Five personalities (Golbeck et al., 2011), dark triad personality (Sumner, Byers, Boochever, & Park, 2012), and depression (Eichstaedt et al., 2018; Tsugawa et al., 2015). However, these studies did not compare different types of SNS information.

In the current study, we systematically examined four types of SNS information with the following hypothesis in mind: since SNS network information such as followers and likes echoes offline interpersonal friendships (Kim, Natali, Zhu, & Lim, 2016; Komori et al., 2019), it can be used to predict communication‐related and social traits, particularly the empathizing‐systemizing system (Baron‐Cohen et al., 2005). We also hypothesized that time information reflecting a user's lifestyle is linked to the inhibition and activation motivational system (Carver & White, 1994; Gray, 1970), because people with higher motivation for communication or expressing their opinions may post on SNS more often than others and vice versa. Furthermore, natural language information can predict mental health‐related indices, as suggested by a previous study in which depression was found predictable more by language expressions than by other posting activities on Facebook (Eichstaedt et al., 2018). A text analysis of Internet forums also indicated that words used in anxiety, depression, and suicidal ideation forums were different from words used in other forums (Al‐Mosaiwi & Johnstone, 2018). It is also known that mental illness leads to speech‐language deficits (Cohen, McGovern, Dinzeo, & Covington, 2014). Therefore, language information is a strong candidate predictor of various mental health indices. In addition, the frequency of a word and a phrase should be able to predict behavioral economics indices, because previous studies reported that the frequency of words on Twitter predicted political attitude, religion, income (Kosinski et al., 2013; Volkova, Bachrach, & Durme, 2016), and even profession (Kern, McCarthy, Chakrabarty, & Rizoiu, 2019).

To systematically address our hypotheses, we collected 24 self‐reported personality traits (and attributes) (comprising 52 subscales) from 239 participants, which were categorized into eight groups (see Methods): (1) “Mental health,” (2) “Behavioral economics,” (3) “Empathizer‐systemizer,” (4) “Inhibition/activation,” (5) “Big Five personality,” (6) “Intelligence,” (7) “Life satisfaction,” and (8) “Drink & smoke.” We used groups (1) to (4) for a hypothesis‐driven analysis and groups (5) to (8) to conduct a data‐driven analysis and to compare with previous studies (e.g., Kosinski et al., 2013). We evaluated the predictability of these personality traits and attributes from four different SNS information sources: SNS network structure (e.g., number of likes and retweets), bag‐of‐words (BoW; e.g., the appearance of single and consecutive words), within‐tweet word statistics (e.g., length of words and sentences in a tweet and the proportion of positive and negative words), and time information (e.g., the interval between tweets). We illustrate the study design as a diagram in Figure 1. To guarantee the generalization of the predictions, we adopted a repeated‐sampled (10 times) 10‐cross validation of a state‐of‐the‐art machine learning algorithm (i.e., component‐wise gradient boosting (CGB) algorithm (Hofner, Mayr, Robinzonov, & Schmid, 2014)).

FIGURE 1

The design of the present study

METHODS

Participants

We collected participants by advertising to people who had previously registered on our homemade experiment participant system. The system includes only the contact information for hundreds of previous participants. We sent an email invitation to the study for each participant. All participants provided written consent to the study, including anonymous use of their Twitter log and personality traits data for research purposes. The NICT ethics committee approved all study procedures. All 239 participants (156 males and 83 females) were Japanese, completed personality traits questionnaires in an experimental room, and were paid for their participation (three thousand yen each). The average age of the participants was 22.4 years old (SD = 3.70). We strictly checked if the participants met the following criteria by analyzing their twitter logs: (a) posting more than 100 tweets before participating in the study and (b) no more than half the tweets included retweets, links, hashtags, and images to avoid bot or advertisement accounts. If any participant violated one or both conditions, they were disqualified from the study.

Questionnaires for investigating human traits and attributes

Table 1 lists the questionnaires we used in this study (the descriptive statistics and references are shown in Table S1). Motivated by our research interest in social behavior, decision making, and mental health, we selected these questionnaires to examine our hypotheses and to conduct a data‐driven analysis spanning beyond the Big Five Inventory to include nine mental health scores (i.e., schizophrenia [SPQ_C, SPQ_I, SPQ_D], delusion [PDI_DE, PDI_DI, PDI_FR, PDI_CO], psychopathy [PSPS_P, PSPS_S], Machiavellianism [MVS], obsessive‐compulsive disorder [OCIR_WA, OCIR_OB, OCIR_ST, OCIR_OR, OCIR_CH, OCIR_NE], depression [BDI, SDS], anxiety [STAI_S, STAI_T], and stress [PSS]; the subscales are depicted in parentheses as abbreviations). Behavioral economics scores included socioeconomic measures [SES], preferred coworker measures [LPC], and social value orientation [SVO_P, SVO_I, SVO_C]. These scores also deal with things related to money, such as risk aversion and time discounting [RA, TIM]. Empathizer‐systemizer scores include empathy [IRI_F, IRI_PT, IRI_EC, IRI_PD], and autism [AQ_S, AQ_D, AQ_A, AQ_C, AQ_I] scales. Inhibition/activation scores contain the behavioral inhibition system [BIS] and behavioral activation system [BAS_D, BAS_R, BAS_S] scales.

TABLE 1

List of personality tests

Abbreviation	Sub score abbreviation (Item number)	Questionnaire name
AQ
Social skill	AQ_S (10)	Autism‐Spectrum Quotient
Attention shift	AQ_D (10)
Attention detail	AQ_A (10)
Communication	AQ_C (10)
Imagination	AQ_I (10)
AUDIT	(10)	The Alcohol Use Disorders Identification Test
BDI	(21)	Beck Depressions Inventar‐II
Big Five		Big Five personality traits
Extraversion	Big5_E (12)
Agreeableness	Big5_A (12)
Conscientiousness	Big5_C (12)
Neuroticism	Big5_N (12)
Openness	Big5_O (12)
BIS/BAS		Behavioral Inhibition and Approach System
BIS	(7)
BAS_drive	BAS_D (4)
BAS_reward responsiveness	BAS_R (5)
BAS_sensation seeking	BAS_S (4)
FTND	(6)	Fagerstrom Test for Nicotine Dependence
HAP	(14)	Happiness scale
IRI		Interpersonal Reactivity Index
Fantasy	IRI_F (7)
Perspective Taking	IRI_PT (7)
Empathic Concern	IRI_EC (7)
Personal Distress	IRI_PD (7)
JART	Verbal_IQ (100)	Japanese Adult Kanji Reading Test
LPC	(18)	Least Preferred Coworker
MVS	(20)	Machiavellianism Scale
OCI‐R		Obsessive‐Compulsive Inventory‐ Revised
Washing	OCI‐R_WA (3)
Obsession	OCI‐R_OB (3)
Stocking	OCI‐R_ST (3)
Ordering	OCI‐R_OR (3)
Checking	OCI‐R_CH (3)
Neutralization	OCI‐R_NE (3)
PDI		Peters et al. Delusions Inventory
Delusion	PDI_DE (40)
Distress	PDI_DI (40)
Frequency	PDI_FR (40)
Confidence	PDI_CO (40)
PSPS		Primary and Secondary Psychopathy Scales
Primary	PSPS_P (15)
Secondary	PSPS_S (6)
PSS	(10)	Perceived Subjective Stress
RA	(10)	Risk Aversion
Fluid_IQ	(60)	Raven Advanced Progressive Matrices test
RSS	(10)	Rosenberg Self‐Esteem scale
SDS	(20)	Self‐rating Depression Scale
SES	(1)	Socio Economic Status
SVO		Social Value Orientation
Pro‐social	SVO_P (8)
Individualist	SVO_I (8)
Competitor	SVO_C (8)
SPQ		Schizotypal Personality Question
Cognitive/Perceive	SPQ_C (33)
Interpersonal	SPQ_I (33)
Disorganized	SPQ_D (16)
STAI		State‐Trait Anxiety Inventory
State	STAI_S (20)
Trait	STAI_T (20)
TIM	(15)	Time discounting

Big Five personality scores consist of Extraversion, Neuroticism, Agreeableness, Conscientiousness, and Openness [Big5_E, Big5_A, Big5_C, Big5_N, Big5_O], and define the broad and basic personality of a person. Intelligence scores (IQ in Figure 2) contain both fluid and verbal intelligence [Fluid_IQ, Verbal_IQ]. Life satisfaction measures happiness and self‐esteem [HAP, RSS]. Finally, Drink & smoke scores examine people's consumption of alcohol and cigarettes [AUDIT, FTND]. Details of all scales, abbreviations, and basic statistics are summarized in Table S1.

FIGURE 2

(a) Correlations between the predicted human traits scores by the social media network information‐based predictors (out‐of‐sample) and the measured (actual) scores. Error bars denote standard errors for 10 repeated evaluations. Solid, dashed and dotted lines show p = .05/52, p = .01/52, and p = .001/52, respectively. (b) A scatter plot of the measured and predicted Big Five Extraversion scores. Each dot represents an individual participant. (c) Network information associated with personality traits. Standardized regression coefficients are displayed as the strengths of the associations. Red means a positive effect, and blue means a negative effect. All red and blue coefficients are statistically significant coefficients in the glmboost regression, while other coefficients (white) are not significant coefficients

Participants were requested to answer 24 questionnaires (52 subscales) installed on laptop computers in the experimental room. They completed the questionnaires in a group consisting of 1 to 4 participants. To maximize their concentration, the participants answered the questionnaires for 45 minutes and took a 15‐minute rest and repeated this process until they finished. The experimenter explained that the study was about human personality and everyday thinking. The mean duration to complete all questionnaires was 116.0 minutes (SD = 66.5). We used an in‐house system based on the Lime Survey program (LimeSurvey GmbH, Germany) to present the questionnaires on web browsers and to collect choice data. We randomized the presentation order of items within a questionnaire.

SNS behavior

We used Twitter's application programming interface to retrieve all past tweets posted by the target participants (up to a maximum of the 3,200 most recent tweets at the time of collection). This Twitter data collection took place in 2016 and 2017. The minimum number of tweets by all participants was 104. We also collected other Twitter account information from all 239 participants as detailed below.

Feature extraction

We transformed all frequency data into percentages rather than actual values for our analysis, since every participant had a different number of tweets (M = 2,233, SD = 1,128). We calculated the percentage as the frequency of each feature to the total number of tweets.

Network features

From each Twitter account, we extracted a set of 15 network features. We counted the number of pure tweets (tweet by oneself; like a monologue), reply, reply network (how many other accounts interacted with the participant), retweet, hashtag, link, image, users being favorited, favorited intensity (how often favorited), retweeted, and retweeted intensity (how often retweeted) from past tweets extracted by the application programming interface. In addition, we obtained the tweets number, the following number, the follower number, and the favorite number from each account. We could obtain these features since the first time using Twitter and the scores greatly differed among participants and were, therefore, log‐transformed.

Time‐related features

Tweets were individually posted at irregular time intervals, which can contain information about personality (Bethlem et al., 2011). We extracted the frequency of hourly, daily, and monthly tweets at the individual level. Hour time indices were extracted every three hours of the day (0:00–3:00, 3:00–6:00, …, 21:00–24:00). Day time and month time indices represented a day of the week (Monday to Sunday) and a month of the year (January to December), respectively. In addition, we calculated the mean, standard deviation, skewness, and kurtosis of the posted time interval between adjacent tweets and replies, respectively. We examined not only the tweet interval but also the reply interval because how one replies (tweet, retweet, etc.) may be related to personality traits (Muscanell & Guadagno, 2012). Adjacent times were log‐transformed because the interval between adjacent tweets was sometimes long.

Natural language content

To extract language‐related features, we first cleaned the tweet texts using regular expressions. We removed hyperlinks, digits, and punctuations. We also removed stop words in Japanese using the GINZA library. The texts were segmented into words by a Japanese morphological analyzer, MeCab (Kudo, Yamamoto, & Matsumoto, 2004), with MeCab‐ipadic‐NEologd as a Japanese word dictionary (Satou, 2015).

Twitter words statistics

In order to characterize participants from the word usage within tweets, we calculated word statistics. For every participant, we calculated the mean, standard deviation, skewness, and kurtosis length of a tweet in words and the mean, standard deviation, skewness, and kurtosis length of a sentence in characters. We also calculated the proportion of emotional words. We assessed the emotional valence score for each account (the degree of positive and negative emotion strength) from the valence value of the noun, verb, and adjective.

ANEW (Affective Norms for English Words, Warriner, Kuperman, & Brysbaert, 2013) provides lists of emotional words that the research community is widely using. Each word in ANEW is associated with a valence score (from 1: very unpleasant to 9: very pleasant). We selected 150 highly positive words (≥8 points) and 165 highly negative words (≤2 points). Next, we used Japanese WordNet (Kanzaki et al., 2008) to translate these English words into Japanese words. Because each English word had multiple translated words, we obtained 1,331 positive words and 1,643 negative words in Japanese. Next, we asked 10 independent raters to assess the valence of each Japanese word. Specifically, the raters conducted two binary decision tasks with the outputs “positive” or “negative” (whether positive or not, and whether negative or not). In order to keep the consistency of the positive and negative words, we selected the words for which at least nine persons agreed with the emotional value. Thus, we obtained 215 positive words and 650 negative words and extracted these word frequencies to determine the users' relative frequency of positive and negative words. We also calculated the ratio of positive words to negative words.

Bag of words

An effective approach for representing documents is the BoW model (Schwartz et al., 2013; Taira & Haruno, 1999). In this model, word histograms are constructed, in which we counted the frequencies of words in a dictionary within a text document. We determined the relative frequency with which users used words (unigrams), two‐word phrases (bigrams), and three‐word phrases (trigrams). To reduce the number of features, we adopted the words and phrases that were used at least once by 25% of the participants. Next, we created binary representations (0 or 1) of these language features to indicate whether a participant used a particular word or phrase. Finally, we obtained a 239 (number of participants) × 4,585 (number of entries) binary BoW matrix from the total number of 567,596 tweets.

Statistical analysis

Predictions of human traits by machine learning

Before conducting the prediction analysis, we eliminated outliers that exceeded 3 standard deviations from the mean of each variable. For each of the personality traits, we conducted an out‐of‐sample prediction test separately for the network, time, BoW, and word statistics as input features. Generalization was evaluated by repeating the 10‐fold cross‐validation procedure 10 times. In each round, we randomly split the entire data set into 10 groups of participants in which we repeatedly (10 times) trained a regression model by fitting the component‐wise gradient boosting (CGB) algorithm (Hofner et al., 2014) to nine of the ten groups and tested the result with the remaining group. The CGB algorithm is a state‐of‐the‐art adaptive boosting algorithm (Friedman, 2002) and used in real‐world applications (molecular biology (Huang et al., 2011) and genetics (Lin, Futschik, & Li, 2013)). Boosting algorithms are also proven to be effective for natural language applications (Haruno, Shirai, & Ooyama, 1999). Thanks to the high interpretability of CGB, we can know which variable is important for the prediction. We used the glmboost function with the default parameter in mboost R‐package for the analysis (Hofner et al., 2014).

More specifically, the CGB procedure builds a strong predictive model from an ensemble of weak models by reducing the residuals iteratively. Linear predictors in these various functional forms that occurred in the course of the iterative procedure with the resulting coefficients are robust by the use of L2 penalized least squares. Such shrinkage techniques are supposed to stabilize effect estimates (Hofner et al., 2014). Further details of CGB are described elsewhere (Bühlmann & Hothorn, 2008; Hofner et al., 2014).

Using the trained model, we conducted out‐of‐sample predictions for the remaining 10% of the data (i.e., the holdout group). We estimated the predictive accuracy by calculating the Pearson's correlation between the actual and predicted personality‐trait scores. We repeated this procedure 10 times and calculated the average of the correlation coefficient. All correlation coefficients are reported with the Bonferroni multiple corrections for the number of personality predictions. In addition, we confirmed that features highly predictive of a personality trait reasonably reflect the characteristics of the personality trait. Inferences on parameter estimates from CGB were performed via bootstrap resampling with 1,000 replicates by the means and standard errors of the bootstrap distribution to compute p values.

Predictions of personality clusters by machine learning

To provide additional evidence that each of the four SNS information types is a good predictor of specific human traits and attributes, we tested whether these sources can also predict human traits clusters. In order to group similar human traits, we calculated the pairwise inter‐correlations for each of the 52 subscales and averaged them into a 52 × 52 matrix. We then conducted a hierarchical clustering analysis of this matrix by Ward's method. The analysis generated a dendrogram to estimate the number of likely clusters. We tested the predictive accuracy in a four‐way regression of human trait clusters by the same out‐of‐sample approach.

Data availability

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy restrictions.

RESULTS

To evaluate the prediction accuracies for personality traits and attributes and also each SNS feature, we computed correlation coefficients between a trait score and an estimated value averaged over all participants (Figures 2a, 3a, 4a, 5a). For about half of the personality traits (23/52), we found reliable correlations between SNS‐based personality predictions and actual trait values (all p values <.05/52; with Bonferroni correction). Importantly, successful traits existed mainly in (1) “Mental health,” (3) “Empathizer‐systemizer,” (5) “Big Five personality,” (6) “Intelligence,” (7) “Life satisfaction,” and (8) “Drink & smoke,” but not in (2) “Behavioral economics” or (4) “Inhibition/activation.” Furthermore, to understand how SNS information serves as a personality trait marker, we extracted SNS markers for each personality trait (Figures 2c, 3b, 4b, 5b) by selecting the significant coefficients obtained by the CGB algorithm (see also Methods section).

FIGURE 3

(a) Correlations between the predicted human traits scores by the social media time information‐based predictors (out‐of‐sample) and the measured scores. Error bars denote standard errors for 10 repeated evaluations. Solid, dashed and dotted lines show p = .05/52, p = .01/52, and p = .001/52, respectively. (b) Temporal information associated with personality traits. Standardized regression coefficients are displayed as the strengths of the associations. Red means a positive effect, and blue means a negative effect. All red and blue coefficients are statistically significant coefficients in the glmboost regression, while other coefficients (white) are not significant coefficients

FIGURE 4

(a) Correlations between the predicted human traits scores by the social media word statistics information‐based predictors (out‐of‐sample) and the measured scores. Error bars denote standard errors for 10 repeated evaluations. Solid, dashed and dotted lines show p = .05/52, p = .01/52, and p = .001/52, respectively. (b) Word statistics information associated with personality traits. Standardized regression coefficients are displayed as the strengths of the associations. Red means a positive effect, and blue means a negative effect. All red and blue coefficients are statistically significant coefficients in the glmboost regression, while other coefficients (white) are not significant coefficients

FIGURE 5

(a) Correlations between the predicted human traits scores by BoW (Bag of Words) information‐based predictors (out‐of‐sample) and the measured scores. Error bars denote standard errors for 10 repeated evaluations. Solid, dashed and dotted lines show p = .05/52, p = .01/52, and p = .001/52, respectively. (b) Words and phrases with the strongest correlations to OCIR_OB, SES, and AUDIT, as predicted by binary BoW (N = 4,585). Word clouds contain the 30 positive (red) and negative (blue) words and phrases with the highest correlations with OCIR_OB, SES, and AUDIT. Word size is proportional to the correlation size. Dots (.) in the word clouds connect words within phrases

Predictions of personality traits by network information

We found that the highest correlation in the network information (Figure 2a) was with Extraversion (Big Five_E) (r = .44), followed in order by social skill (AQ_S) (r = .41), communication difficulty (AQ_C) (r = .35), verbal intelligence (Verbal_IQ) (r = .33), schizotypal interpersonal (SPQ_I) (r = .25), empathic concern (IRI_EC) (r = .23), happiness (HAP) (r = .22), detailed attention (AQ_A) (r = .22), and Neuroticism (Big Five_N) (r = .21). These results demonstrate that network information performs well on “empathizer‐systemizer” and inter‐personal relationships. In Figure 2b, we exemplify the measured and predicted Big Five Extraveersion scores that exhibited the highest prediction performance.

Figure 2c summarizes the network information markers of the predicted personality traits. We see that “Favorited” and “Reply network” are key factors. Specifically, “Favorited” is positively correlated with the social aspects of autism and with verbal intelligence, but negatively correlated with Extraversion, empathic concern, and happiness. In contrast, “Reply network” correlated positively with Extraversion and empathic concern and negatively with schizotypal interpersonal and the communication, attention detail, and social aspects of autism. Similarly, “Tweet” correlated negatively with happiness. “Link” correlated positively with empathic concern and negatively with attention detail. These observations, which show that communication‐related features such as “Reply network” contribute to Extraversion (Big_5) and empathic concern (IRI_EC) positively and schizophrenia interpersonal (SPQ_I) and autism (AQ_C, AQ_S, AQ_A) negatively, indicate that on‐line communication is deeply associated with users' mental states. Importantly, network information is the best predictor for autism traits, in which the most contributing feature was “Favorited,” which deeply relates to social communication and interaction. In addition, “Favorited” and “Tweet” also predicted verbal intelligence (Verbal_IQ).

Predictions of personality traits by time information

Time information (Figure 3a), in contrast, showed the highest correlation with verbal intelligence (Verbal_IQ) (r = .25), and strong correlation with pro‐social (SVO_P) (r = .22) orientation measures.

Figure 3b illustrates that less tweets in March as well as a smaller SD for “Reply time” provide clues for predicting verbal intelligence (Verbal_IQ). These features may reflect the busy lifestyle and punctual communication of people with high verbal intelligence. In contrast, the scores of social value orientation (SVO_P) correlated with the Period of use at time PM6‐9 may reflect the characteristics of prosocials. Reply time SD was also negatively correlated with SVO_P.

Predictions of personality traits by word statistics information

Word statistics and BoW were superior to network information and time information in terms of the average accuracy over all personality traits. In more detail, we found the highest correlation by word statistics (Figure 4a) for verbal intelligence (Verbal_IQ)' (r = .36), followed by four delusion scores (PDI_FR: r = .32, PDI_DI: r = .31, PDI_CO: r = .29, PDI_DE: r = .29), fluid intelligence (Fluid_IQ) (r = .29), social skill (AQ_S) (r = .24), schizotypal disorganized (SPQ_D) (r = .23), communication difficulty (AQ_C) (r = .22), anxiety (STAI_T) (r = .22), and depression (BDI) (r = .21). Overall, word statistics exhibited remarkable performances for (1) “Mental health” and (6) “Intelligence.”

Figure 4b illustrates the results for word statistics markers for predictable personality traits. The sentence length SD and the proportion of positive/negative words were the most prominent predictors. Specifically, the sentence length SD and the proportion of negative words predicted schizophrenia disorganized (SPQ_D) and subcategories of delusion (PDI_DE, PDI_FR, PDI_CO). The proportion of negative words also predicted delusion (PDI_DI, PDI_FR, PDI_CO) and anxiety (STAI_T). The proportion of positive words negatively contributed to intelligence (Fluid_IQ, Verbal_IQ), anxiety (STAI_T), depression (BDI), and autism (AQ_C, AQ_S). It is remarkable to see that variance in tweet sentence length and the proportion of emotional words reflect mental health and intelligence.

Predictions of personality traits by BoW information

As seen in Figure 5a, BoW showed a high correlation with verbal intelligence (Verbal_IQ) (r = .37), socioeconomic state (SES) (r = .36), imagination deficit (AQ_I) (r = .31), schizotypal disorganized (SPQ_D) (r = .28), fluid intelligence (Fluid_IQ) (r = .27), obsession (OCIR_OB) (r = .27), distress of delusion (PDI_DI) (r = .26), social skill (AQ_S) (r = .25), alcohol use (AUDIT) (r = .23), and cigarette use (FTND) (r = .23). We found a negative correlation (r = −.25) only for obsession neutralization (OCIR_NE), probably due to the limited number of participants. In comparison with word statistics, BoW was superior in drink & smoke, but inferior in mental health such as BDI (depression) and STAI (anxiety).

We examined whether the words and phrases that were most effective for the BoW‐based prediction capture the characteristics of each personality well. Figure 5b exemplifies the markers of obsessive‐compulsive disorder (OCIR_OB) (top), socioeconomic status (SES) (center), and alcohol (AUDIT) (bottom). We see that OCIR_OB markers include many tense and imminent words, and SES markers reflect study‐related words (e.g., press, paper, laboratory, and presentation). In contrast, entries in AUDIT contained many words reminding us of drinking and smoking, such as “drunk,” “eat,” “last train,” and “one hour.” These results demonstrate that BoW‐based markers captured target personalities well.

Specific and common human traits predicted by the four types of social media information

Figure 6 summarizes the percentage of significant correlations within each human trait category for different social media information types. We confirmed that each SNS information source has different compatibilities with human traits and attributes. That is, network information achieved good performance for empathizer‐systemizer and Big Five scores, which may represent interpersonal‐related personalities. Time information, in contrast, performed well on social value orientation. Word statistics showed high prediction accuracy for mental health, such as delusion, schizophrenia, and depression tendency. Only BoW could predict daily habits (i.e., drink & smoke). Notably, all four SNS information types were predictive of intelligence.

FIGURE 6

A summary of the prediction results for the different SNS information types and different human traits. Significant correlations (in percentages) for the eight personality categories and the four types of social media information are shown

Cluster‐based personality traits predictions by four types of SNS information

To increase the reliability of the results reported so far, we conducted the same human traits prediction analysis based on the four types of SNS information for the set of hierarchically clustered human traits and attributes. We first searched the dendrogram constructed by clustering the 52 personality subscales and determined the optimal number of clusters, where each cluster was significantly discriminable from others. We identified 13 discriminable categories (Figure 7a) with a confusion pattern that was often seen in the original questionnaire groups (e.g., BAS, PDI, and OCIR), as psychological intuitions (e.g., Extraversion (Big5_E) was indistinguishable from happiness (HAP) and self‐esteem (RSS), and depression (BDI, SDS) was indistinguishable from anxiety (STAI_T, STAI_S). Fluid intelligence was close to verbal intelligence. The interpretation was not clear only in a few cases, such as Machiavellianism (MVS) grouped with detailed attention (AQ_D) and socioeconomic state (SES).

FIGURE 7

(a) A dendrogram of personality clustered variables. The dotted rectangle lines indicate the 13 clusters used in this analysis based on the interpretability. (b) Correlations between the four types of social media information‐based personality prediction scores (out of sample) and actual personality clusters. Error bars denote standard errors for 10 repeated evaluations. Solid, dashed and dotted lines show p = .05/13, p = .01/13, and p = .001/13, respectively

Figure 7b summarizes the prediction accuracies averaged in the human traits clusters for each SNS feature. We found reliable correlations between the human traits prediction scores and the actual cluster scores (all p values < .05/13; with onferroni correction). In the predictions by the network information, the highest correlation was observed for Cluster 1 (r = .37), Cluster 3 (r = .39), and Cluster 12 (r = .39). For the time information, significant correlations between actual and predicted scores were identified for Cluster 3 and Cluster 4 (r = .22 and r = .24, respectively). Significant results by word statistics were seen for Cluster 3 (r = .40), Cluster 5 (r = .31), Cluster 11 (r = .21), and Cluster 12 (r = .20). Finally, BoW produced significant correlation coefficients for Cluster 1 (r = .31), Cluster 3 (r = .31), Cluster 5 (r = .19), Cluster 10 (r = .21), and Cluster 12 (r = .23).

We noticed that each SNS information source has different compatibilities with human traits clusters in a highly consistent manner with the single traits prediction results. That is, network information performed well for interpersonal‐related human traits clusters (1 and 12, including Extraversion, communication skill, and schizotypal interpersonal); time information predicted prosocial‐related Cluster 4 well; word statistics showed high prediction accuracy for the delusion cluster and depression‐related cluster (5 and 11, including all PDI subcategories, BDI, and SDS); and only BoW could predict the alcohol‐related Cluster 10. Importantly, all four SNS information sources were able to predict the intelligence‐related Cluster 3. These data suggested that our human traits prediction results based on four SNS information sources are robust.

DISCUSSION

In this study, we examined the predictability of wide human traits (and attributes) from four different SNS information sources by collecting an intensive set of 24 (52 subscales) human traits categorized as (1) “Mental health,” (2) “Behavioral economics,” (3) “Empathizer‐systemizer,” (4) “Inhibition/activation,” (5) “Big Five personality,” (6) “Intelligence,” (7) “Life satisfaction,” and (8) “Drink & smoke.” We demonstrated that SNS information collectively predicts very broad (23/52) traits and that each SNS information source has different compatibilities with the traits. We also confirmed that these results were robust even when we conducted cluster‐based predictions.

The present study is unique in several ways. First, we studied an abundant set of traits measured beyond the Big Five and used machine learning to evaluate the predictive ability of four different types of SNS information for these traits. The present study also showcased that this type of methodology could be a useful tool for personality psychology (Hinds & Joinson, 2019). Among the four different SNS information types, word statistics showed the highest prediction accuracy for mental health (e.g., delusion, depression, and anxiety), while network information performed well on interpersonal traits (Extraversion and empathizer‐systemizer), consistent with our working hypothesis. Only time information predicted social value orientation (i.e., prosocials and individualists), and BoW was the only predictor of socioeconomic status and drink & smoke. Thus, this study extended previous research that reported personality predictions based on digital footprints including SNS (Biel & Gatica‐Perez, 2013; de Montjoye et al., 2013; Guntuku et al., 2017; Kosinski et al., 2013; Quercia et al., 2011) in terms of both the range of personality traits and the types of SNS information examined.

Network information predicted personality traits for interpersonal relationships. We found that key variables for those predictions are “Reply network” and “Favorited.” People with high Extraversion and happiness showed broad communication networks and frequent communication with other users, as suggested in a previous study (Kosinski, Bachrach, Kohli, Stillwell, & Graepel, 2014). Moreover, people with high autism, schizophrenia, and Neuroticism tendency did not reply often. These results suggest the communication style of people who score high in Extraversion and autism scales. A previous study reported that friendship is a robust predictor of subjective well‐being (Demir, 2015). Our results indicate that friendship on the Internet is also associated with well‐being. Related to this observation, several studies claim that the frequent use of SNS (Twitter in particular) is related to lower subjective well‐being (Kross et al., 2013). However, it is important to note that these previous studies relied only on questionnaires about the frequency of Twitter, Facebook, and Instagram use, and did not consider behaviors on the SNS. The present results suggest that the use of SNS may have a positive effect on users' subjective well‐being when the users establish broad social networks and communicate frequently with others on it, although we cannot rule out the possibility that happier users simply have more friends on SNS. In addition, tweets by people with high autism scores were frequently favorited by other users. Although it is difficult to pin down the precise reason, one plausible possibility is that autistic people have a strong and special interest in specific topics like movies, music, and cartoons (Jordan & Caldwell‐Harris, 2012), and their tweets may attract people who have the same interest.

Word statistics predicted well mental health traits including depression and anxiety. Importantly, emotional words had strong predictive power, with the use of more negative words and less positive words predicting delusion, depression, and anxiety. We also found that the variability of the sentence length is associated with delusion and schizophrenia tendencies. A previous study reported that people with delusion and schizophrenia show a general reasoning bias called ‘jumping to conclusions’ bias (Garety & Freeman, 1999), in which they misattribute the causality of events too easily and too quickly. Although our participants were not patients, the variability in sentence length may be related to unstable thinking. These observations, to our knowledge, are the first demonstration that SNS information including word statistics can predict multiple dimensions of mental health traits beyond depression tendency (Eichstaedt et al., 2018; Guntuku et al., 2017).

It is also noteworthy that the weights for the best predictors of mental health traits were different for each trait, which may contribute to the development of SNS‐based methods that detect subtle changes in specific mental health traits and alert users for caution. Such an automatic assessment method would be beneficial since it is currently difficult to detect mental health problems at their very early stage. These results also suggested that the tendency of schizophrenia, delusion, depression, and anxiety emerge on everyday activity on SNS. Individual mental health traits may have unique language usage markers; if so, everyday language could reflect mental state differences more than previously thought possible.

The results for BoW and time information were not consistent with our hypotheses, and they were not effective at predicting traits in behavioral economics and BIS/BAS. However, BoW information could be used to predict socioeconomic status and alcohol and cigarette consumption. It may have captured users' interest and language use, which echoes socioeconomic status (Preoţiuc‐Pietro, Lampos, & Aletras, 2015) and lifestyle (Ding, Bickel, & Pan, 2017), although it did not reflect the broad range of behavioral economics traits. Notably, only time information predicted social value orientation (Haruno & Frith, 2010; Van Lange, 1999). We found people with prosocial orientation replied to others within a short time, consistent with the well‐established observation that prosocials prefer behavior that benefits others and society (Millon, Lerner, & Weiner, 2003). These people may be adjusting the timing of their communication for the sake of others even in the online environment. Prosocials also tended to have created their accounts recently and use Twitter often between 6 and 9 PM, although the reasons are unknown. Further research is necessary to explain why time information exhibited a correlation with social value orientation, but not with behavioral inhibition/activation (Carver & White, 1994; Gray, 1970).

Finally, we showed that all four SNS information sources predicted intelligence, particularly verbal intelligence. Our analysis showed that people with high verbal intelligence tended to tweet frequently and were more favorited. These people also showed stable timing in their replies. In addition, people with high fluid and verbal intelligence did not use long or positive words frequently. A closer look at the four types of SNS information would help us understand the everyday activities of individuals with high intelligence. These people tend to use simple and non‐emotional words and are likely to play an opinion‐leader role in SNS (Zhang et al., 2016). This strong link between intelligence and SNS behavior is consistent with previous reports that showed intelligence is a deciding factor of various human activities such as educational achievement (Plomin & Deary, 2015), job performance (Kuncel & Hezlett, 2010), and health (Gottfredson, 2004).

One may argue that the findings of this study are specific to Twitter and not generalizable to other SNS. Most representative and meta‐analysis studies (Azucar et al., 2018) on the Big Five personality traits established that the average prediction performance by SNS information is best for Extraversion. This knowledge is consistent with our results (Figure 2) and suggests that the prediction of personality traits based on SNS information may be comparable among different SNS. Nevertheless, generalizability among different SNS is an important topic for future study.

There are limitations to the present study. First, although our method could predict a wide range of personality traits and attributes, it did not cope with BIS/BAS, time discount, risk aversion, psychopath, Machiavellianism, or self‐esteem. A majority of these traits are concerned with economics or reward, which may not be revealed in SNS behaviors. Further research is needed to address this possibility. Second, there was a bias in the participant population. The current participants were composed of more males than females (156 males and 83 females). Since the gender ratio was uneven, we did not perform a specific gender analysis. Future study needs to examine the gender effect on the accuracy of the predictions from SNS information. In addition, the current results are difficult to apply to people who do not use SNS; future study should compare the personality of SNS users and non‐users. Third, our sample size is relatively small compared with previous research of SNS personality predictions (Kosinski et al., 2013; Youyou et al., 2015). It is possible that our prediction performances were limited due to the small number of participants (Cortes, Jackel, Solla, Vapnik, & Denker, 1994). Future research may be able to achieve a higher prediction performance when we collect more data from more participants (Ruopp, 2016).

Finally, with the increasing availability and integrity of digital footprints including not only SNS but also smartphone use (Chittaranjan & Blom, 2013) and the spread of IoT (Internet Of Things) (Prayoga & Abraham, 2016), the results of this study strongly foresee that digital footprints from various modalities can be integrated for fully automated personality (and attributes) predictions in the near future. To make the best use of these methods, it is time for us to start thinking of the pros and cons of each modality and to discuss how far we can apply such personality identification technologies to real‐world problems.

CONFLICT OF INTERESTS

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Supplementary Material

Click here for additional data file.