Voters' view of leaders during the Covid-19 crisis: Quantitative analysis of keyword descriptions provides strength and direction of evaluations.

Objectives: Previous research suggests that governments usually gain support during crises such as the Covid-19. However, these findings are based on rating scales that only allow us to measure the strength of this support. This article proposes a new measure of how voters evaluate Prime Ministers (PM) by asking for descriptive keywords that are analyzed by natural language processing.
Methods: By collecting a representative sample of citizens' own key words describing their PM in 15 countries in Europe during the outbreak of Covid-19, and analyzing these by latent semantic analysis and a multiple OLS regression, we could quantify the strength and direction of voters' view.
Results: The strength analysis supported previous studies that describing the PM with positive words was strongly associated with vote intention. Furthermore, a change in the direction of the attitudes from "good" to "honest" was found. A new finding was that the pandemic was associated with an increase in polarization. Conclusions: The keyword evaluation analysis provides opportunities of evaluating both strength and direction of voters' view of their PM, where we show new results related to increased polarization and shift in the direction of attitudes.

Previous research suggests that crises and disasters instigate increased support for the governing party (Bechtel and Hainmueller, 2011; Healy and Malhotra, 2009). So far, a number studies of the Covid‐19 outbreak support this view: for example, Bol et al. (2021) find increased support for democracy as well as for the incumbent party during the beginning of the crisis, Esaiasson et al. (2020) find a strengthened support for democratic institutions, and Kudzko and Markowitz (2020) report increasing support for political leaders. This can be associated with a “rally‐round‐the‐flag”‐effect, which is when the leader or governing party benefits from leading the nation during an international crisis (Mueller, 1970). Bol et al. (2021), focusing on countries that implemented strict lookdowns, argue that one explanatory factor for the increase in government support during the first month of the crisis is related to the public's support of implementing harder measures. However, Grossman et al. (2020) show that citizen compliance with local government communication can be contingent on party‐ID and depend on expectations. The same study shows that, in the case of the Unite States, signals can have greater impact if they contradict the general leadership but confirm the citizen's position.

This suggests that party sympathy can play a crucial role in compliance with measures, and that citizens’ reaction can be more diverse than some previous research indicates. Whereas the general effect of lockdowns and crisis was positive on people's trust in institutions and government, some data from PR contexts indicate that the impact of the crisis affected citizens heterogeneously depending on party sympathy (Esaiasson et al., 2020).

Although previous research on how crises influence voters’ view of leadership has produced several interesting findings, a shortcoming with them is that the data on government and leadership support often comes from respondents from non‐representative samples who evaluate leaders from standard rating scales. This is problematic as the hallmark of crises is that they are unexpected events, and it is therefore not typically possible to plan studies with rating scales that are specifically tailored to study the aspect that is relevant to specific issues that are related to the crisis. A solution to this methodological problem is to use an open response format where respondents type their answer and where these responses are quantified with natural language processing (NLP) algorithms. We call this approach the question‐based computer language assessments (QCLA).

In this study, we implement QCLA by letting respondents describe their current Prime Minister (PM) in two keywords. From this approach, we develop an understanding of what citizens mean when their support for a leader is rising or falling. This collection of text data proved to be particularly fruitful during the Covid‐19 outbreak, when the national leaders became the center of attention, and the contextual setting shifted rapidly. The expectation was that collecting voters’ own descriptions of their current political leadership would give a richer understanding of how citizens evaluate their head of government, which nuance and complement current findings about increased support for government during crises such as Covid‐19.

The aim of this article is to show how the keyword approach can be used to both measure the strength and the direction of voters’ evaluations of their leadership following the Covid‐19 outbreak. The overarching hypothesis is that the Covid‐19 crisis influenced both the strength and the direction of the evaluation leadership, where we argue that the pandemic not only results in a rally‐round‐the‐flag effect, but also increase the polarization and changes the voters’ attitudes of their leaders from being good to be being honest as the magnitude of the pandemic becomes apparent.

The empirical study was embedded in a larger data collection including more than 15,000 European citizens in fifteen countries, selected on a representative basis (Bol et al., 2021). At the time of planning the study, the authors were unaware that the data collection would coincide with the outbreak of Covid‐19 and could therefore not plan specific questions related to this issue. All countries and respondents in the survey were, to some extent, affected by the crisis. The survey included the unique items where the voter described their PM in two keywords, which has previously been tested only in national election contexts (Fredén and Sikström, 2021). The survey also contained more standard variables such as vote intention, and socio‐economic variables. Since the data was collected before and after some of the countries introduced strict lockdown measures, it is possible to compare qualitative shifts in attitudes based on quantified word descriptions during the first phase of the crisis.

Using NLP, which associates the text descriptions produced by the respondents with how words usually co‐occur with other words, brings new evidence to the dynamics of government support during the Covid‐19 outbreak. While previous studies identify falling or rising support, this type of analysis identifies shifts in central keywords, as well as shifts in positive and negative attitudes. From the latent semantic analysis (LSA), words can be visualized in word clouds in relation to their centrality and occurrence. Furthermore, more advanced estimation techniques can translate the words into negative versus positive descriptions, and include this valence measure in multivariate models. In line with previous studies, we find that support for the leader of government increased over time. In addition, we find that this increase in support is associated with citizens describing their leader as “honest”. There is also a tendency that whereas supporters of the PM party became more united in their descriptions of the PM over time and under implementation of stricter measures, supporters of other parties became more dispersed relative to PM party supporters. This suggests that citizens polarized their view of the leader at the covid‐19 crisis outbreak.

Introducing a key‐word‐based measure of leader support and analyzing the words by NLP allowed us to quantify the direction of the attitude changes of public opinion during the first phase of the covid‐19 crisis.

EVALUATING LEADERS AND SUPPORTING PARTIES DURING AN INTERNATIONAL CRISIS

The general wisdom from research looking at the relationship between voters’ evaluations of government and their vote choice, suggest that crises tend to have a major impact on how citizens’ vote. For example, Mueller (1970) has shown that citizens during an international crisis tend to support the leading party largely, and natural disasters and financial crises appear to have similar impact, as long as they are international in their character. One characteristic with such a crisis is that people become (even) more dependent on their national leadership and coordination. So far, the Covid‐19 seems to follow similar patterns. Bol et al. (2021) show that people's satisfaction with democracy and tendency of supporting the governing party increased as countries implemented stricter lockdowns, and evaluations of leaders appear to have soared. In the United States, where political campaigns and parties center more around party identification and candidates than in Europe, the system has polarized even further, dividing the two main parties—Democrats and Republicans—on opposite sides, as well as an increased tendency for populism (Lieberman et al., 2019). Recent research from the Covid‐19 outbreak conducted in spring 2020 suggests that some Republican state governors took a different stance than their party's official strategy and in this manner gained compliance from Democrat supporters (Grossman et al., 2020). This suggests that in the current situation, party identification and perception of party leadership are important cues to understanding how people behave.

A recent cross‐country comparative study suggests that the association between ideology and behavior during the first phase of the covid‐19 was stronger in the United States than in some countries in Europe (Becher et al., 2021). However, their study focuses on the relationship between ideology and compliance with general health guidelines. Since citizens in Europe experienced the same international crisis, and given previous findings of short‐term (positive) effects on citizens’ government evaluations during crises, we expect patterns that are more general in the attitudes toward leaders (Mueller, 1970; Stimson, 1976).

For example, in the party oriented context of Sweden, the tendency to comply with Covid‐19 health guidelines is weakly associated with ideology (Becher et al., 2021), and the outbreak of the Covid‐19 crisis increased support for the governing institutions (Esaiasson et al., 2020). However, it also rapidly divided the elite intellectual debate along two sides: pro and contra the relatively weak government measures (Kudo, 2020). One observation is that polarization between political camps potentially became stronger during the outbreak of the Covid‐19, also in more moderately polarized contexts, during a moment where individual experiences potentially played a greater role than in regular election campaigns.

MEASURING LEADER SUPPORT

The typical operationalization of leader or party support is some kind of rating scale. For example, in the long‐tradition surveys of presidential support in the United States, that the previous “rally‐round‐the‐flag”‐literature base their conclusions in, approval ratings is the standard formula: “Do you approve or disapprove of the way (the incumbent) is handling his job as President?” (Mueller, 1970; Stimson, 1976), or, in a more recent version: “Do you approve or disapprove of the job Donald Trump is doing as President?” (Monmouth University Polling Institute, 2020). In the European context, polling institutes have collected similar survey items of leader support or confidence ratings during the Covid‐19 pandemic (see, e.g., Kudzko and Markowitz, 2020).

This study introduces a new measure of leader evaluation in a political context. The proposal is that we can learn something new from analyzing leader sympathies based on voters’ free text words, which is basically absent in previous studies on the impact of leaders on political attitudes. Whereas rating scales can measure citizens’ overall evaluations, words can more deeply illustrate the way the voter actually perceives how the leader handles and appears in the current context. In other words, the keyword analysis gives meaning to what a “seven” stands for: it could be honesty and competence, and/or strength and courage. In that manner, we also take into account some of the potential variation on what a number stands for in a certain context, and let the respondents formulate their evaluations in words in their first language, which is rare in comparative studies.

Moreover, the words can be quantified into a negative‐positive‐scale based on how the words the citizen indicates are usually used in context. This is thus a richer representation of the voter's overall evaluation of leadership, than a number on a scale. In addition, analyzing the text materials via NLP is a step forward compared to studies that tend to count words or make more simple associations in relation to dictionaries, such as for example linguistic inquiry and word count (LIWC) (compare Crabtree et al., 2020). NLP maps the content of words based on their co‐occurrence with other words. In this manner, each significant word becomes more meaningful, and it is possible to obtain latent meaning of word descriptions.

In addition, collecting free text materials during the outbreak of the Covid‐19 crisis has an additional value, as this crisis came very unexpectedly and shifted character over a very short period, when the survey was in the field. It is thus a unique opportunity to detect shifts during the first phase of a pandemic, and how the people looked at their leaders.

From the combination between keyword descriptions and standard party choice variables, the aim is to evaluate and discuss the association between leader support (as described and measured from text) and vote intention (categorical variable) over the very first phase of the crisis . This study thus contributes by asking voters how they perceived their leadership at the time when the crisis broke out, and analyzing their answers in a manner that is a complement to previous approval rating‐based studies.

HYPOTHESES

This study uses LSA to analyze words that participants generate to describe their PM during the Covid‐19 outbreak. We obtain word cloud patterns and estimate valence (i.e., the degree of positive‐negative description) based on the quantitative analysis of keywords. Following the introduction above, we make the following hypotheses regarding voters’ descriptions of their leaders:

Changes in the direction of attitudes

The implementation of stricter measures will produce qualitative changes in citizen attitudes toward PMs, which will be visible from latent semantic word clouds.

Rally‐round‐the‐flag

Through the keywords, we can obtain a numeric valence measure, which can identify negative and positive attitudes directly. Doing this, we relate the textual material and analysis to standard rating scales and evaluate whether the attitude has become more or less positive. Drawing on previous crisis‐oriented research, we expect that the general attitude will be more favorable of leadership during the crisis:

The PM descriptions become more positive following the crisis set out by the Covid‐19 outbreak.

Party support effect

We expect that the positive attitude will be more prevalent among citizens supporting the leader's party than those that do not support this party, i.e. that the “rallying” will be contingent on party sympathy.

Participants that support the PM's party will describe their leader with higher valence than those preferring other parties.

Polarization of PM‐ versus non‐PM party supporters

From the valence score, we can also obtain classic statistics such as standard deviations. We argue that the standard deviation of the valence score of the PM should be an indicator of how polarized participants’ views are of the PM, so that a low standard deviation means a lack of polarization, whereas a high standard deviation means a polarized view. Following the previous hypotheses, we expect that polarization—differences between supporters and non‐supporters – will intensify over the studied period, i.e. over the implementation of stricter measures:

Participants supporting other parties will polarize their descriptions of the leader (measured as the standard deviation of valence) more than citizens supporting the Prime Minister's party.

METHODS

We collected data from March 2, 2020 until April 3, 2020, when many countries introduced strict measures in order to combat the virus. The data collection was part of a larger study managed by King's College London and Dynata (Bol et al., 2021). One thousand citizens in each of the fifteen countries (except for Belgium, where two groups of 1000 were recruited from different regions divided by language), making up 15,000 participants, took an online survey about their attitudes toward electoral systems as well as some other democratic issues. Our keyword items were thus embedded in a general survey, asking respondents to give their opinion on four free text items, consisting of free generation of two keywords. Previous studies show that two keywords usually contain sufficient information for co‐occurrence analyses, and that the first word usually contains most information (Kjell et al., 2019). Since our study was part of a very large cross‐country survey, we had to limit the number of words to two for each of the four‐keyword items that were included in the survey.

The question that is the focus in this study is how the participants describe their Prime Minister (PM). The question was: “How would you describe your current PM? Please indicate two descriptive keywords or answer “don't know” if you do not know who he/she is” (see also the online Appendix). This item was the third out of four keyword‐based items in the survey. The other three, that are not the focus in this study, were formulated as follows; “How would you describe a good political leader?” (item 1), “Which are the two most important political issues for you personally?” (item 2), and “Describe how you feel right now with two words” (item 4). It is possible that the preceding item on issues instigated the voter to describe the PM in more ideological words; however, the comparison between time points in the study is still valid as the survey looked exactly the same over the studied period.

The second focus is on the following standard vote intention item: “If there was an election today, which party would you vote for?”, which was included at the beginning of the survey. Based on the responses on this item, we created a variable indicating if the citizen intended to vote for the PM's party, or for some other party (for a list of PM party coding, see the online Appendix). The respondents received and answered the survey in their first language, and an expert with knowledge of that particular country had validated the formulation of the keyword items referring to the PM. The answers were translated into English using Google translate for those participants/countries that used other languages than English. The translation was facilitated using full stops after every keyword, so that the translation was performed word by word, rather than being interpreted as completed sentences by Google translate. All respondents who had entered at least one keyword describing their PM were included in the analysis (n = 9276). Thus, about one third of the 15,000 respondents did not insert a keyword to describe their PM. The Prime Minister keyword module was included as the third extra module toward the end of an extensive survey and included the options ``Don't know” and “Refusal”, which should explain some of the dropout (for a screenshot of the item, see the online Appendix). The main difference between respondents who inserted descriptive keywords of their PM versus those who skipped it is that the keyword respondents tend to be higher educated.1 More importantly, the share who inserted keywords was consistent over time. Compared with studies using self‐selected samples, the present sample is still large and representative in terms of educational level and other socio‐economic characteristics. For an overview of the characteristics of respondents versus non‐respondents of the keyword item, see Table A1 in the online Appendix.

The text material was analyzed using the Matlab version of the data program Semanticexcel.com, which is a software implementing the NLP method LSA (Kjell et al., 2019). LSA is a quantitative text analytical approach that quantifies and systematizes text responses (Landauer and Dumais, 1997). This data‐driven (unsupervised) method is suitable for measuring meaning in word expressions by quantifying how similar the words are to each other. The method resembles factor analysis, since words that are similar in meaning receive similar semantic representations. In this study, we use a semantic representation that was generated by applying the LSA method on text taken from the Google n‐grams (n = 5) database that is fully described in Kjell et al. (2019). Here we provide a short summary of the method.

N‐grams are a set of N consecutive words in texts, where we use the 5‐grams occurring at least twenty times. First, a co‐occurrence matrix of the frequency of words occurring together in the n‐gram was generated, where each cell represents the number of times, the words in the rows and columns have co‐occurred in the 5‐grams. The rows in the co‐occurrence matrix represent the 120,000 most common words in the text, which roughly matches the size of the vocabulary of humans in a language. To make the computational load of creating the singular value decomposition (SVD) less demanding, we limited the number of columns to 50,000 common words. Each cell was normalized by taking the logarithm of the frequency count plus one. Finally, a data compression algorithm called SVD was applied, that maintains as much information as possible in an ordered set of dimensions. The number of dimensions maintained in the SVD was optimized to 512 dimensions, where the quality of the semantic representation was measured by a synonym test. Less, or more, number of dimensions yielded poorer performance on this test. This resulted in a vector describing each word. This vector was normalized to the length of one. The two words generated from each participant were aggregated, the final vector was again normalized to the length of one.

We used the semantic representation to analyze the data in two ways. First, word clouds were generated to visualize the qualitative shifts in the use of words over different periods (before March 16, or March 16 or after) and between supporters and non‐supporters of the PM party. In this analysis, the independent variables are thus time and party support, and the dependent variable is the keywords. To avoid over‐fitting, the number of dimensions used in the representation is a hyper‐parameter that is optimized in this algorithm. The result is evaluated with tenfold leave‐out cross‐validation procedure, where the 90 percent of the data material is used to generate the regression coefficients that are applied to make a prediction on the 10 percent left out data. This procedure is repeated ten times, so all data get a predicted value. Based on the predicted value, we plot the 100 most indicative words in the respective group (at different time points, or PM party supporter vs. non‐PM party supporter). Thus, the word clouds illustrate significant words generated by the survey participants in relation to these semantic spaces. In the plots, the most indicative words appear in the center of the cloud, where the font size represents the frequency or occurrence of the words in the data set. From this method, we can visualize and evaluate significant differences between the word descriptions.

In the next step, we transformed the words into numeric values of positive and negative contents, i.e. valence. This was done using the same regression method as described above to train a model to estimate the valence of the words. In this case, the outcome variable fitted in the regression came from the Affective Norms for English Words (ANEW) database, consisting of 1000 words that are manually rated for valence on a scale from 1 to 10. This analysis starts from the Google n‐gram space, because this space creates more accurate valence scores. The text descriptions for the valence analyses, on the other hand, were based on a semantic space created on the data of PM descriptions. We ran distribution analyses that indicated that the valence scores that were based on the current data semantic space generated more variance than valence scores based on the Google n‐gram semantic space, and chose the former valence score for the standard deviation and OLS regression analyses.

The central explanatory variables in the multiple analyses were, in line with the descriptive analyses and the hypotheses, PM party support and time. A variable indicating whether the respondent took the survey under strict lockdown (1) or under more voluntary restrictions (0) was also created, based on the dates of countries in strict lockdowns in the University of Oxford Government Response Tracker and Bol et al. (2021).

As controls, smoothed daily death rate based on a data source from IHME/University of Washington adjusted by the day the respondent took the survey and the number of inhabitants in the country is included.2 In addition, the socio‐economic variables gender, age, education (coded on a three point scale based on the Dynata text classifications, where 0 is the lowest level—secondary school or lower, 1 is the mid‐level—vocational training, and 2 is the highest level, university education) are controls in the multivariate analysis. In addition, the regression models contain country‐dummies to control for country‐specific effects.

RESULTS

The direction of attitude change

The words voters use to describe their PM provide evidence of how citizens describe their leaders during the outbreak of Covid‐19. Hypothesis 1 stated that the quantitative assessments of the keywords could generate the direction of the attitude changes of the voters. We find that the emphasis shifted over the studied period: “good” is more significant in the descriptions that came before March 16 (left in the figure), whereas “honest” stands out as significant after that date (right in the figure), giving support for a qualitative attitude shift (Figure 1).

FIGURE 1

Rally round the flag hypothesis. European citizens describe their leaders during the Covid‐19 crisis. The x‐axis represents time. On the left are words associated with descriptions by voters who described their PM before March 16, when many countries started implementing stricter regulations, and on the right are those who described their PM from March 16, onward. The font size of the word represents frequency, and central words are more representative for the co‐occurrence patterns in this particular group. Yellow words are more representative than red words. Semantic space: Google n‐grams. n = 9276

Systematizing voters’ word descriptions of their PMs along the same timeline shows that citizens described their PM with positive as well as negative words before many countries implemented lockdowns as well as after.

Hypothesis 2 investigated the rally‐round‐the flag effect, or that support for the PM increases during the crises. First, in line with previous studies (Bol et al., 2021), the finding is that the average share of respondents who said that they intended to vote for the governing party increased from 18 percent before March 16 (n = 12591), to 23 percent from March 16 onward (n = 2823), when many of the countries had implemented lockdowns. This development is also visible in the valence analyses (see below). The data thus indicates that the governing party gained support as the crisis evolved.

Hypothesis 3 was that party sympathy plays a crucial role in the evaluation of leaders during a crisis. In line with this expectation, Figure 2 shows that the positive word descriptions are most significant among supporters of the PM party (on the right side of the figure), where descriptions such as “good” and “honest” are central. On the other hand, among non‐PM party supporters (on the left side of the figure), “incompetent” and “liar” are characteristics that stand out as significant.

FIGURE 2

Party support effect hypothesis. European citizens describe their leaders during the outbreak of the Covid‐19 crisis. On the left are voters who said that they intended to vote for other parties than the PM's; on the right are those who intended to vote for the PM party. The font size of the word represents frequency, and central words are more representative for the co‐occurrence patterns in this particular group. Yellow words are more representative than red words. Semantic space: Google n‐grams. n = 9276

The strength of the party support effect over time (Hypothesis 3) can be quantified by estimating the valence of the words from supporters of the PM party versus non‐supporters. Based on the ANEW data set, we made a multiple regression model for predicting the valence of the words (ranging from 1 to 9, with a mean value of 5.8, where higher value indicates descriptions that are more positive). Looking at this indicator divided by party support, we find that, translated into valence, the mean valence value is 6.6 for supporters of the PM party during the early period, increasing to 7.1 in the latter, whereas non‐PM party supporters’ increase in support is more modest: from 5.5 in the early period to 5.8 in the latter (Figure 3).

FIGURE 3

Rally round the flag by party support. Semantic valence score Prime Minister evaluations across time and party support. Higher value indicates a more positive score. The difference in level of valence between PM party supporters and others, and the difference between period 1 and 2 versus period 3 is significant at the 0.05‐level. Semantic space of key words in this analysis: created on current data. n = 9276

Polarization of PM‐ versus non‐PM party supporters (Hypothesis 4)

One interpretation is that the outbreak of the crisis amplified the voter's perception of sympathy for the governing party and the PM's competence (or incompetence), that is, the descriptions of the PM by PM party supporters versus others became more dispersed. This can be further investigated comparing standard deviations for PM party supporters versus others during across periods. Looking at the standard deviation measures divided by party preference gives an indication of this (Figure 4). Citizens who supported other parties than the governing party evaluated the leader with greater differentiation during lockdown—standard deviation increased slightly from a level of 1.92 before lockdowns, to 1.95 during. On the other hand, PM party supporters’ support stabilized considerably—from standard deviation of descriptions 1.70 before lockdowns, to 1.67 and then 1.38 during lockdowns. The difference in standard deviations between supporters of the PM party and supporters of other parties is statistically significant at the 0.05‐level, and the magnitude of the difference increases over the studied period.

FIGURE 4

Polarization of PM‐ versus non‐PM‐party voters ‐hypothesis. Standard deviation of semantic valence score Prime Minister evaluations by party support and across time. Semantic space of key words in this analysis: created on current data. n = 9276

Looking at the overview statistics over time and groups, it is also evident that the most negative evaluations became more negative over time in the group of voters choosing other parties than the PM's: from value −1.97 in the beginning of the outbreak, to −3.30 in late March (online Appendix Tables 3–4).

In this kind of pooled analysis, variation between countries should be a feature that affects the results to a greater or less extent (see Becher et al., 2021; Kritzinger et al., 2021). Therefore, we conducted separate analyses of valence and standard deviations in the two PM party supporters versus non‐PM party supporters groups of respondents in all the 15 countries in the survey (compare online Appendix Table 1). These show that a majority of the countries follow the patterns described above, that is, standard deviation of descriptions of the PM decreases over time for government party supporters, and relative to non‐government party supporters.3 However, in Finland, France, and Germany, as well as in the Netherlands and in Spain, these patterns are less clear. Finland and France have semi‐presidential systems, which should affect the tendency to associate the PM with crisis leadership. Germany has the long‐term leader Angela Merkel who may unite supporters from different political sides relative to other countries. Netherlands and Spain are systems with a great number of parties and local differences, which could blur citizen association with political leadership. In the analysis below, we take potential country‐specific differences further into account.

MULTIVARIATE ANALYSIS

Based on the predicted valence score from the free key words, we conducted a regression analysis on the relationship between party support, time, and lockdowns, including country‐dummies and robust standard errors clustered by country, which should mitigate country‐specific heteroscedasticity (Colin and Miller, 2015). The regression analyses support the previous semantic analyses and shows that time has a small positive effect on overall evaluations of the PM, under control for socio‐economic factors, providing further support of the rally‐round‐the‐flag effect (Table 1). Under control for death rates (Model 2), the relationship between time and positive evaluations becomes weaker. However, including death rate in the model does not improve the general fit of it (r‐squared remains stable), which would indicate that time in crisis was a more decisive factor for increasing leader support than the actual number of deaths during this period. Moreover, vote intention has a strong relationship to positive evaluations of the PM in free text also under the multivariate controls. The overall interpretation of the descriptive statistics and the multivariate models is that the European citizenry, in general, looked at their leader more positively, but with different (party sympathy) lenses when the crisis intensified.

TABLE 1

OLS regression of the impact of time and party sympathy on citizens’ PM evaluations

Explanatory factor	Model 1	Model 2, controlled for death rate
Government party vote intention (0–1)	+1.080* (0.141)	+1.080* (0.141)
Days (−11.6–16.04)	+0.015* (0.007)	+0.016 (0.008)
Locked down (0–1)	−0.097 (0.056)	−0.096 (0.058)
Age (18–99)	+0.004* (0.002)	+0.004* (0.002)
Female (0–1)	+0.135* (0.051)	+0.135* (0.051)
Vocational education
(0–1)	−0.001 (0.046)	−0.001 (0.046)
Higher education
(0–1)	+0.061 (0.049)	+0.061 (0.049)
Death rate smoothed value
(−3.48–8.13)		−0.000 (0.034)
Constant	+4.889 (0.091)	+4.889 (0.099)
R‐squared	0.105	0.105
Number of observations	9257	9257

Note: Dependent value is valence score based on citizens’ free text keywords, where higher value indicates evaluations that are more positive. Country dummies are included as controls in the models (with the United Kingdom as baseline) in order to control for country‐fixed effects.

Standard errors are clustered at the country level.

*indicates statistical significance at the 0.05‐level.

The socio‐economic characteristics that influence positive PM descriptions are older age and being female, which corresponds to the relationships between these characteristics and general trust in government during the Covid‐19 outbreak found in Bol et al. (2021). However, being locked down “per se” is unrelated to more positive text descriptions of the PM.

To summarize, the analysis suggests that the rally‐round‐the‐flag effect for PMs is contingent on party sympathy. Moreover, the analysis shows that citizens’ evaluations of their leaders dispersed at the time of lockdowns. Analyzing citizens’ free text answers thus reveals that the citizens’ picture of their PM was heterogeneous, rather than steady.

CONCLUSIONS

This study has used a keyword based approach to analyze the strength and direction of attitude changes toward leadership during the Covid‐19 outbreak. Whereas earlier studies based on rating scales have been limited to study the strength of these attitudes, our approach also allows us to study the direction of this change. Our findings replicate earlier studies that west European citizens’ view of their leaders, in general, became more positive at the time of lockdowns. Moreover, the quantitative textual analysis allowed us to show that the direction of the attitudes move from good to honest as the magnitude for the pandemic became more apparent, a finding that would have been difficult to study with standardized rating scales as it was not possible to foresee the outbreak at the time of the planning of the study. Thus, these nuances in the view of leadership became revealed by analyzing the data through NLP, which identifies the particular characteristics citizens associated with leadership over time, and between party sympathy groups.

Another finding is that a supportive view of the PM is highly contingent on a tendency to support the PM's party. One interpretation is that the crisis outbreak accelerated a development into two “camps”: those who encouraged and wanted to see more of the same leadership and party traditions (the party in government), and those who distinguished themselves with a more critical view and supported alternative strategies more intensely than before. Voters who had the governing party as a viable option became more likely to support that party, and one reason for that can be that their view of the intense crisis phase leadership was positive. However, many citizens took distance from its PM during lockdowns, and the most negative descriptions became more negative over the studied period. This nuances some previous studies during the same period, showing that polarization along party lines was not very prevalent in Europe, concerning the relationship between ideology and compliance with recommendations and restrictions (Becher et al., 2021). We argue that leader support is of more short‐lived character than such compliance. This is consistent with previous findings from the American system, indicating a short‐lived increase in leader evaluations at international crises (Mueller, 1970).

Future studies should further elaborate the link between leadership evaluations and party perceptions more in depth. In this analysis, we used leader evaluations as the dependent variable, but it is very possible that citizens reevaluate their party sympathy depending on party leaders (Fredén and Sikström, 2021). The free text based quantitative semantic analysis suggests that support for parties and leaders are intertwined and potentially even more so during an intense crisis outbreak. Moreover, to what extent the keyword measurement strategy is sensitive to the local context would be worth further elaboration (Adcock and Collier, 2001). For even more in‐depth knowledge, it would be fruitful to follow up the panel respondents with their view on PM leadership during different phases of the crisis.

Table A1 Drop out Analysis/Sample characteristics

Table A3‐4 Predicted Prime Minister valence score and dispersion across time and party support

Click here for additional data file.