Neural mechanisms mediating degrees of strategic uncertainty.

In social interactions, strategic uncertainty arises when the outcome of one's choice depends on the choices of others. An important question is whether strategic uncertainty can be resolved by assessing subjective probabilities to the counterparts' behavior, as if playing against nature, and thus transforming the strategic interaction into a risky (individual) situation. By means of functional magnetic resonance imaging with human participants we tested the hypothesis that choices under strategic uncertainty are supported by the neural circuits mediating choices under individual risk and deliberation in social settings (i.e. strategic thinking). Participants were confronted with risky lotteries and two types of coordination games requiring different degrees of strategic thinking of the kind 'I think that you think that I think etc.' We found that the brain network mediating risk during lotteries (anterior insula, dorsomedial prefrontal cortex and parietal cortex) is also engaged in the processing of strategic uncertainty in games. In social settings, activity in this network is modulated by the level of strategic thinking that is reflected in the activity of the dorsomedial and dorsolateral prefrontal cortex. These results suggest that strategic uncertainty is resolved by the interplay between the neural circuits mediating risk and higher order beliefs (i.e. beliefs about others' beliefs).

Introduction

Since the foundation of game theory (von Neumann and Morgenstern, 1947), there has been a growing quest for common theoretical and behavioral approaches for two seemingly distinct situations: playing against nature or against other players. A main focus has been on modeling uncertainty in such situations (Knight, 1921; Luce and Raiffa, 1957; Roth, 1977; Aumann and Dreze, 2009). In social interactions, strategic uncertainty arises when the outcome of one’s choice depends on the choices of others. An important question is whether strategic uncertainty can be represented by subjective probabilities (Savage, 1954) to the counterparts’ behavior as in games against nature, therefore reducing situations of strategic uncertainty to individual risky choices (Heinemann ). In social interaction, people might form beliefs about others’ beliefs and behave according to different levels of strategic thinking, modeled as level-k (Nagel, 1995; Stahl and Wilson, 1995) or cognitive hierarchies (Camerer ). Thus, an additional question is how deliberation (i.e. degrees of strategic thinking) interacts with strategic uncertainty.

Behavioral models have tackled the distinction between low vs high levels of strategic thinking, however, without connecting it with risk (Camerer and Fehr, 2006; Crawford ; Hargreaves Heap ). Similarly, neuroeconomics has separately investigated risk (Knutson ; Preuschoff ; Rushworth and Behrens, 2008) and strategic uncertainty (Coricelli and Nagel, 2009; Yoshida ). This is the first neuroeconomics study in which behavioral and neural processes underlying strategic uncertainty are related to the processes arising from risky choices.

In our study, we compare the behavior and brain activity measured with functional magnetic resonance imaging (fMRI) during choices under risk (i.e. individual lottery decisions) and strategic uncertainty in social interactive games: stag-hunt and entry games. Our original paradigm allows us to measure strategic uncertainty by eliciting certainty equivalents (i.e. a measure of risk attitude) in games analogous to measuring risk attitudes in lotteries. The two games differ in deliberation demands. In stag-hunt games, players have incentives to coordinate on the same choices and require low degrees of strategic reasoning. Entry games encourage coordination on opposite choices, which requires higher degrees of (recursive) strategic reasoning of the kind ‘I think that you think that I think etc.’ Our experimental design therefore allowed us to compare behavior and brain activity between risky and strategic uncertainty conditions and to investigate their modulation with varying levels of deliberation.

At the behavioral level, we confirmed that stag-hunt games generate behavioral patterns similar to those observed in lotteries (Heinemann ). At a neural level, we found that these similarities are associated with a common neural substrate composed by the anterior insula, dorsomedial prefrontal cortex and parietal cortex, previously associated only with the processing of individual risk (Knutson ; Preuschoff ; Christopoulos ; Mohr ). In parallel, we found that entry games induce higher levels of uncertainty, as reflected in higher activity in the network mediating risk, and recruit brain regions (dorsomedial and dorsolateral prefrontal cortex) mediating high levels of strategic reasoning (Coricelli and Nagel, 2009; Bhatt ; Yoshida ). Our results therefore provide a common neural framework for previously unrelated decision-making situations such as risk and coordination problems that induce strategic uncertainty.

Materials and methods

Participants

Twenty healthy right-handed subjects (nine females) were recruited to take part in a study at the Center for Mind/Brain Sciences (CIMeC, University of Trento, Italy). Mean age of participants was 26 years (6.28, s.d.). These volunteers gave fully informed consent for the project, approved by the local (University of Trento) Ethical Committee. Individuals with a history of psychiatric or neurological problems were not included in the study. Two participants were excluded from the analysis due to excessive movement (>3 mm) during the image acquisition. We considered only 18 participants for the analyses of behavioral and fMRI data. Subjects knew that they were playing with nine other subjects who were under exactly the same conditions, but at a different scheduled time. Participants were told that the experiment aimed at studying decision making in social context, that they would receive a compensation of €30 show-up fees and that the money gained in three (one for each condition: lottery, stag-hunt and entry games) randomly extracted trials would be added to their compensation. Once 10 subjects had been scanned, we sent them an e-mail message with a table of their own choices and the choices of their co-players (preserving anonymity). We repeated the same procedure for the second group of 10 participants. For each subject, we summed up the winning amount of three trials randomly selected (one trial for each condition). Participants earned on average €27 (plus the show-up fee). A transfer of the money they won was sent to their bank account.

Experimental design and task

Before entering the scanner, the task was explained to subjects and they were allowed a 5 min familiarization with the lottery and games. Each participant underwent fMRI scanning while performing 210 trials (see Figure 1 and Supplementary Table S1). During scanning, subjects viewed a projection of a computer screen and chose one of two options (A or B) in each trial. We used lotteries and two types of strategic games: stag-hunt and entry games. Both have been studied extensively in theory and experimental settings (reviewed by Camerer, 2003). In strategic games, participants (either in groups of N = 10 or N = 2) had to choose between two options A and B, where Option A was paying a sure payoff X that was varied between situations (X{1, 2, 3, 4, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10, 10.5, 11, 12, 13, 14, 15} Euros). In stag-hunt games, Option B was rewarded with €15, provided that at least K subjects from the same group decided for B, and 0 otherwise. In groups of N = 10 subjects, K was varied between 4, 7 and 10; in groups with N = 2, K = 2. In entry games, Option B was rewarded with €15, provided that at most K subjects from the same group decided for B, and 0 otherwise. In groups of N = 10 subjects, K was varied between 4 and 7; in groups with N = 2, K = 1. In lottery choices, participants chose between the exact same options A and B, where Option A was paying a sure payoff X ≤ €15 that was varied between situations as in strategic games. Option B was paying €15 with a fixed known probability . The wide range of sure payoffs allowed us to measure attitudes towards risk and strategic uncertainty by eliciting certainty equivalents for lotteries and analogue measures for strategic games. Participants were instructed that their choices in the social conditions (stag-hunt and entry games) would be matched with the choices of the other nine participants. In games with group size N = 2, the participants were informed that their choice would be matched with the one of another randomly selected participant (within the group of 10 participants).

Fig. 1.

Experimental design. Each trial had the same structure: a subject had to choose between a known sure payoff (say A-choice; with values = {1, 2, 3, 4, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10, 10.5, 11, 12, 13, 14, 15}) and an uncertain payoff (say B-choice). The figure shows three examples of different screens with a typical trial of each condition and intertrial intervals (i.e. fixation cross). The screen on the top is a lottery situation (i.e. risk). The subject chooses between the sure payoff 9.50 (A-choice) and a lottery (B-choice) in which he gets €15 with probability of 2/3 (if die is 1, 2, 3 and 4) and zero otherwise. The middle screen shows a stag-hunt game situation (i.e. strategic uncertainty that requires low level of strategic thinking) with a sure payoff choice of 9.50 (Choice A) and an uncertain game choice (B-choice), in which player i who chooses B gets 15 if at least 4 out of all 10 players (including i) choose B and otherwise he gets 0. In the lower screen, the entry game situation (i.e. strategic uncertainty that requires high level of strategic thinking), with a certain payoff choice of 9.50 (A-choice) and an uncertain game choice (B-choice), in which player i who chooses B receives 15 if at most 4 out of all 10 players (including i) choose B and zero otherwise. Option A was paying a sure payoff that was varied between situations (as shown on the top).

Time course of the experimental task

(see Figure 1) At each trial, subjects viewed an information screen indicating the payoffs of the A-choice and the type of condition (lottery, stag hunt and entry) indicated by the text under the €15 payoff of the B-choice (e.g. ‘if die is 1, 2, 3, 4’, ‘if at least 4 of 10’ and ‘if at most 4 of 10’, for the lottery, stag-hunt and entry games, respectively). The type of condition varied randomly across trials. Participants could choose (self-paced within 6 s) between pressing the right or left button of an MRI compatible button box. After each choice, the next trial was presented after a variable delay ranging from 10 to 14 s (randomly drawn from a lognormal distribution) so to dissociate fMRI responses from different trials. We did not provide any feedback between trials in order to eliminate learning through payoff experience.

Statistical analysis of behavioral data

Regression analyses

Choice behavior was analyzed based on panel data analysis using the statistical software package Stata (StataCorp, College Station, TX). We ran panel logit regressions, which took each participant as the unit, and estimated both random and conditional fixed effects. We used B-choice for each condition as dependent variable and the value of the sure payoff (X) and the probability P (for the lotteries) or the K parameter (for the games) as independent variables. We report the results for the random effects analysis (see Table 1).

Table 1.

Regression analyses

	(1)	(2)	(3)
	Lotteries	Stag hunt	Entry
X	−0.769	−0.567	−0.249
	(0.052)***	(0.039)***	(0.029)***
P	2.915
	(0.205)***
K		−1.944	0.415
		(0.151)***	(0.099)***
Constant	−0.455	13.694	−2.740
	(0.457)	(1.053)***	(0.978)**

Notes: Panel logit regressions for choice for the three conditions. The dependent variable ‘choice B’ is equal to 1 if subject chose B and 0 if subject chose A. X indicates values of sure payoffs, P refers to the winning probabilities (P = 1/3, 1/2, 2/3) and K to the minimum number of participants that should choose B in order to win in the stag-hunt games (K = 4, 7, 10), and the maximum number of participants that could choose B in the entry game (K = 4, 7). Regression 1: Log likelihood = −331.17384, Wald chi2(3) = 241.08, P > chi2 = 0.000. Regression 2: Log likelihood = −391.56465, Wald chi2(3) = 242.18, P > chi2 = 0.000. Regression 3: Log likelihood = −353.13122, Wald chi2(3) = 83.93, P > chi2 = 0.000. Numbers indicate Coeff. (SE), ***P < 0.001, **P < 0.01.

Measures of risk and strategic uncertainty

Estimating risk preference by means of certainty equivalents in lotteries and games

We estimated the probability of the participant choosing the risky option (B-choice) for each condition (lottery, stag hunt and entry), as a function of the value of the sure payoff (X), and the probability P (for the lotteries) or a relative threshold (for strategic games), using logit regressions. Parameter k is the proportion of other subjects needed for success (i.e. obtaining the €15 of Choice B) in the games. We then computed for each participant the estimated threshold (i.e. certainty equivalent) for a lottery with a winning probability of P = 0.5, and for games with a relative threshold of k = 0.5: ; where b0 is the coefficient of the constant, b1 is the coefficient of P (for the lotteries) or k (for the games) and b2 is the coefficient of the variable X. Probabilities of B-choices were estimated as: . In order to determine risk averse (RA) and risk loving (RL) types, we ranked participants in terms of certainty equivalents for the lottery choices (X*-lottery). Participants with certainty equivalent higher than the expected value of the risky options were categorized as RLs (n = 6); we then considered for comparison purpose as (highly) RA a group of participants (n = 6) with lowest certainty equivalents. Note that median split of the data generated similar results. We used median split of the certainty equivalent for the lottery choices to define RA and RL participants for the fMRI analysis [general linear model (GLM) 1].

Measures of threshold strategies

Previous experiments have shown that a vast majority of subjects use perfect threshold strategies in ordered (for increasing sure payoffs) lottery choices and stag-hunt games (Holt and Laury, 2002; Heinemann ). We confronted subjects with 21 different X-values for sure payoffs in an unordered fashion. Ordering their choices for any given Option B and increasing payoffs for A, a perfect threshold strategy would be a sequence starting with B and ending with A with only one switching point. A sequence without switching would also be considered as a perfect threshold strategy, with the threshold being out of range. In order to measure how close actual behavior was to a perfect threshold strategy, we count the minimum number of decisions that have to be turned around to generate a perfect threshold. For example, a strategy with several switches between options, such as BBBABABBBBAAAAAAAAAAA requires dropping two A-choices (underlined) to generate a perfect threshold. We define (almost perfect) threshold strategies as strategies that require at most two such drops while any strategy requiring at least three or more drops is classified as non-threshold strategy.

Threshold strategies and levels of strategic reasoning in games

The cognitive hierarchy theory defines heterogeneity of players by their strategic sophistication, measured by levels of reasoning: level-0 players play randomly, level-1 players best respond to random play and more sophisticated players (level-2) best respond to a distribution of lower level players (Camerer ). Depending on the game, the level of strategic sophistication may lead to threshold or non-threshold strategies. In stag-hunt games, higher levels of reasoning do not change one’s decision, nor the number of switching points one should exhibit. For instance, a level-1 player should best respond to the chance behavior of level-0 players and should thus exhibit a perfect threshold at some sure payoff value; a level-2 player (who best responds to a combination of level-0 and level-1 players) should then ‘match’ the behavior of the level-1 player; and so forth, for higher levels of reasoning. Consequently, in our stag-hunt games, an increased number of switching points (i.e. non-threshold strategy) is unlikely to be a signature of increased reasoning, but rather indicate inconsistent decisions. In entry games, however, the best response to a mixture of different levels may be a non-threshold strategy. Consider, for example, a level-2 player who believes that 50% of subjects are Level 0 and 50% are Level 1 so that an arbitrarily chosen subject will enter with 75% for X < X* = Bin(K−1, N−1, 0.5)15. Take N = 10, K = 7, then X*=13.65. Thus, the player should enter for X < Bin(K−1, N−1, 0.75)15 = 5.99 and not enter for larger X until X*. For X > X*, the player expects that another subject enters with 25% probability. Thus, the player should enter for X* < X < Bin(K−1, N−1, 0.25)15 = 14.98, and for 14.98 < X < 15, should again not enter. Thus, in entry games, non-threshold strategies can be taken as a sign of increased reasoning (Camerer ). Since each player profits from thinking one level further than the others, and because the game has no symmetric equilibrium in pure strategies, non-threshold strategies in the entry games can be expected to yield higher payoffs than any threshold strategy. Thus, higher levels of reasoning (Level 2 or higher) should not affect behavior and performance in stag-hunt games, but may lead to non-threshold strategies and higher expected payoffs in entry games.

Computing expected payoffs

We computed the expected payoff for each subject in each trial. If subjects chose A, the payoff was the sure payoff X [i.e. ]; if they chose B, the expected payoff was: , for the lotteries condition, where ; for the stag-hunt games; and for the entry games. The terms in parenthesis indicate the success probability for an agent playing B, considering the observed relative frequency of B-choices for the group of participants. The best response to the observed distribution and the cumulated expected payoffs for each participant given his choice were computed.

fMRI method

fMRI data acquisition

A 4T Bruker MedSpec Biospin MR scanner (CIMeC, Trento, Italy) and an eight-channel birdcage head coil were used to acquire both high-resolution T1-weighted anatomical MRI using a 3D MPRAGE with a resolution of 1 mm3 voxels and T2*-weighted Echo planar imaging (EPI). The parameters of the acquisition were the following: 34 slices, acquired in ascending interleaved order, the in-plane resolution was 3 mm3 voxels, the repetition time 2 s and the echo time was 33 ms. For the main experiment, each participant completed three runs, each composed of 70 trials. An additional scan was performed in between two different runs in order to determine the point-spread function that was then used to correct the known distortion in a high-field MR system.

Pre-processing

The pre-processing and statistical analysis of the fMRI data was performed using SPM8 (Wellcome Trust Centre for Neuroimaging, London, UK; http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). The first five EPI volumes were discarded from the analyses to allow for stabilization of the MR signal. The remaining EPI images from all sessions were slice-time corrected and aligned to the first volume of each session to correct for head movement between scans. A mean image was created using the realigned volumes. T1-weighted structural images were first co-registered to the mean EPI image of each participant. Normalization parameters between the co-registered T1 and the standard MNI T1 template were then calculated and applied to the anatomy and all EPI volumes. Data were then smoothed using an 8 mm full-width-at-half-maximum isotropic Gaussian kernel to accommodate inter-subject differences in anatomy. Finally, intensity normalization and high-pass filtering (128 s) were applied to the data. We did not include movement-related regressors at the first level, because head displacements were negligible. Indeed, we checked that each subject did not move more than 3 mm in each fMRI run. To control for potential differences in head movement across conditions and groups, we performed a one-way analysis of variance on mean displacements of the head during fMRI, by taking as a factor the group type. We did not find any difference in head movement neither between RA and RL participants (P = 0.2974), nor between threshold (Th) and non-threshold (nTh) participants (P = 0.5881).

fMRI data analysis

First-level analyses

The statistical analysis of the pre-processed event-related fMRI signals was performed using a GLM approach. Each trial consisted of one event aligned to the presentation of the two options. To dissociate the fMRI activity from different trials, regressors were constructed by convolving the canonical hemodynamic response with boxcar functions of constant duration (2 s) aligned to the time of option presentation (similar results are found when using boxcar functions with variable duration, i.e. equal to participant’s reactions times at each trial). At the single-participant level, the GLM design matrix was composed of six regressors containing trials associated to different combinations of three game types (lotteries, stag hunt and entry games) and two choices (uncertain option, B-choice, or sure payoffs, A-choice).

Group-level analyses

The goal of the fMRI analyses was 2-fold. In the first analysis (GLM 1), we investigated brain regions involved in making risky decisions (i.e. B-choices) during lotteries and games as a function of risk preference. In accordance with behavioral data, we also investigated brain regions whose neural activity during B-choices was similar for lotteries and stag-hunt games, and whether these brain regions displayed selectivity for subjective risk preference. In the second analysis (GLM 2), we studied how deliberation (i.e. the degree of strategic thinking) interacts with strategic uncertainty in stag-hunt and entry games as a function of threshold strategies. As mentioned in the previous section, the level of participants’ strategic sophistication is reflected in threshold or non-threshold strategies. Thus, we searched for brain regions displaying selectivity for strategy type in the two games.

GLM 1

In the first model we searched for brain regions with higher responses to B-choices in the entry games with respect to lotteries and stag hunt, and displaying differences for subjects being classified as RA vs RL (i.e. selectivity for risk preference) in the lotteries and stag-hunt games only. To investigate neural responses when participants chose the uncertain option (B) in the three conditions or the sure payoffs (A) as a function of risk preference (RA and RL participants, respectively), we created a design matrix containing 12 regressors: First, we searched for brain regions displaying an increase in BOLD response during B-choices in the entry games with respect to lotteries and stag-hunt games, we tested a specific effect using a contrast vector λ = [0, 0, −1, −1, 0, 0, −1, −1, 0, 0, 2, 2]. The constants in λ are the coefficients of a function that ‘contrasts’ the parameter estimates b. The vector λ is referred to as the contrast vector. Then, to search for brain regions selective for risk preference within the previously identified network, we masked (inclusively) the previous contrast with a contrast vector testing for a significant increase in BOLD activity for RA with respect to RL participants during B-choices in lotteries and stag-hunt games: λ = [0, 0, 1, −1, 0, 0, 1, −1, 0, 0, 0, 0]. The resulting network displayed higher neural activity for B-choices in the entry games, irrespective of risk preferences, and higher activity for RA types compared to RL types for B-choices in lotteries and stag-hunt games. To control for similar effects for A-choices, we ran an additional analysis with a contrast vector testing for a significant increase in BOLD activity for RA with respect to RL participants during A-choices in the three conditions (λ = [1, −1, 0, 0, 1, −1, 0, 0, 1, −1, 0, 0]).

GLM 2

In a second linear model, we studied how deliberation interacts with strategic uncertainty in stag-hunt and entry games. To do so, we investigated neural responses when participants chose the uncertain option (B) or the sure payoffs (A) in the three conditions as a function of threshold strategies (see Measures of risk and strategic uncertainty: Measures of threshold strategies). We thus created a design matrix containing 12 regressors: where Th and nTh refer to participants using a threshold or a non-threshold strategy, respectively. The goal of this second model was to study differences among threshold and non-threshold strategies in entry games in brain areas displaying a significant increase for social vs individual context. We thus created a contrast searching for a significant increase for games with respect to lotteries λ = [−2, −2, −2, −2, 1, 1, 1, 1, 1, 1, 1, 1] and we performed an inclusive mask with a contrast testing for a significant increase for non-threshold vs threshold participants in the entry game λ = [0, 0, 0, 0, 0, 0, 0, 0, −1, 1, −1, 1].

All fMRI statistics and P values arise from group random-effects analyses. We considered as activated (after height thresholding at P < 0.001, whole brain analysis) all clusters surviving P < 0.05 corrected for multiple comparisons (family-wise error, FWE). Plots representing parameter estimates (beta values) were realized by extracting BOLD data for areas of interest. Areas of interest were functionally defined. Parameter estimates from the fitted model were extracted and averaged across all voxels in the cluster for each subject. These analyses were performed with the MarsBaR 0.41 SPM toolbox (http://marsbar.sourceforge.net/). Reported coordinates conform to the Montreal Neurological Institute (MNI) space.

Results

Behavioral results

Choice times differed between the three experimental conditions, between entry games and the lottery (Wilcoxon signed-rank test, z = 3.06, P = 0.002), between entry and stag-hunt games (z = 2.67, P = 0.008) and between lottery and stag hunt (z = −2.35, P = 0.02). Reaction times (RT) in all three types of situations were shorter for safe (Option A) than uncertain (Option B) choices. Over all conditions, A-choices had a mean RT of 2.57 s (standard error, SE = 0.17) whereas B-choices had a mean RT of 3 s (SE = 0.16) (z = −3.059, P = 0.002). There were no differences in RT for A-choices in the three conditions (Kruskal–Wallis, chi-square with 2 d.f. = 0.0839, P = 0.66); while, RT for B-choices significantly differed among conditions (Kruskal–Wallis, chi-square with 2 d.f. = 6.61, P = 0.037); with longer RT for B-choices in the entry games compared to lotteries and stag-hunt games (Wilcoxon signed-rank test, B-choices lottery vs stag hunt: z = −1.96, P = 0.05; B-choices lottery vs entry: z = −3.59, P = 0.002; B-choices stag hunt vs entry: z = −2.98, P = 0.0029).

Figure 2 shows the relative frequencies of B-choices separately for lotteries, stag-hunt and entry games conditional on the (21 different) sure payoffs. The frequency of B-choices decreased with increasing sure payoffs keeping other parameters constant [see Figure 2: all logit functions are decreasing; regression analysis in Table 1 shows that the coefficient of the values of the sure payoffs (X) is negative, P < 0.001] (see Materials and methods: Regression analyses). Similar pattern of behavior is found for N = 2 conditions (see Supplementary Material and Supplementary Figure S1). Furthermore, in lottery conditions, the frequencies of B-choices increased with increasing probability (P) of winning €15 (the frequency curves or logit functions shift to the right with higher P). Similarly, in stag-hunt games, a rising threshold for the number of B-players required for success (K = 4 vs 7 vs 10) reduced the number of B-choices (the curves shift to the left). While in entry games, as expected, the curves shifted to the right when more players were allowed to enter (K = 4 vs 7) (Table 1).

Fig. 2.

Behavioral results. Relative frequency of B-choices in lotteries, stag-hunt and entry games, depending on (21 different) sure payoffs (x-axis); probabilities [P = 1/3 (red), 1/2 (green), 2/3 (blue), in lotteries]; number of players (K) 4 (blue), 7 (green), 10 (red), in stag-hunt games that at least have to choose B to win; 4 (red) or 7 (green) in entry games that at most can choose B to win. The colored solid lines indicate logit estimations. The vertical black lines indicate certainty equivalences.

Thus, in all conditions, a higher sure payoff reduced the proportion of B-choices, and the easier it seemed to obtain the high payoff (keeping the sure payoff constant), the more likely B was chosen.

The parametric modulation of sure payoffs for Option A for any given Option B allows the estimation of certainty equivalents for the respective uncertain payoff associated with choosing B, and therefore the quantification of risk preferences. The level of strategic reasoning was also assessed through the analysis of threshold vs non-threshold strategies (see Materials and methods).

Risk preferences

In order to characterize risk preferences, we estimated a certainty equivalent X* for each subject for a lottery with a winning probability of 0.5. Similar certainty equivalents were estimated for the stag-hunt and the entry games (see Materials and methods: Estimating risk preference by means of certainty equivalents in lotteries and games). Higher certainty equivalents for lotteries are associated with higher degrees of risk aversion. Notably, in stag-hunt games RA subjects were more likely to choose the sure payoff (Option A) in comparison to RL subjects (Kruskal–Wallis, chi-square with 1 d.f. = 13.095, P = 0.0003). We found a high correlation between certainty equivalents for the lottery and the stag-hunt games (X*-lottery and X*-stag hunt: Pearson correlation r = 0.69, P = 0.0019, Bonferroni-adjusted significance level). This did not hold for entry games, where certainty equivalents had no significant correlation to certainty equivalents of either the lottery (X*-lottery and X*-entry: r = 0.27, P = 0.33) or the stag-hunt games (X*-stag hunt and X*-entry: r = 0.22, P = 0.41). Moreover, we did not find any behavioral differences between RA and RL subjects in the entry games (i.e. both types of participants chose the same proportion of A-choices in the entry games, Kruskal–Wallis, chi-square with 1 d.f. = 0.4, P = 0.53).

Levels of strategic reasoning

Ideally, when a participant chooses a threshold strategy, he/she selects B if the payoff for A is low, and A if its payoff is high with at most one switching point, indicating the certainty equivalence for the uncertain Option B. In lotteries, we found that 50% of strategies are such perfect threshold strategies (see Materials and methods: Measures of threshold strategies). In 28% of lottery choices, there was only one decision deviating from a perfect threshold, in 15% there were two deviations. We counted the minimum number of decisions that had to be dropped for getting a perfect threshold strategy within each block of 21 decisions that could be ordered. We called strategies with one or two of these drops ‘almost perfect threshold strategies’. The remaining 7% were classified as non-threshold strategies as they required more than two drops (see Table 2 for the entire classification).

Table 2.

Proportion of threshold strategies for each condition

No. of drops	Lottery	Stag hunt	Entry
0	0.5	0.319	0.148
1	0.278	0.278	0.296
2	0.148	0.236	0.167
3	0.056	0.097	0.093
>4	0.019	0.069	0.296

Notes: The ‘drops’ represent the minimum number of decisions that had to be changed for getting a perfect threshold strategy within each block of 21 decisions that could be ordered for increasing sure payoffs. There is a significant difference in the frequencies of almost threshold strategies (i.e. at most two drops, highlighted in grey) among conditions [Pearson chi-square (8 d.f.) = 32.39, P < 0.001].

We used the same definition of threshold strategies for strategic games. We found no significant difference in the frequencies of threshold strategies between lotteries and stag-hunt games [Pearson chi-square (4 d.f.) = 6.09, Fisher exact P = 0.21], while there were less threshold strategies in the entry games than both in lotteries [Pearson chi-square (4 d.f.) = 24.14, Fisher exact P < 0.001; see Table 2] and stag-hunt games [Pearson chi-square (4 d.f.) = 13.97, Fisher exact P = 0.007]. As described in Materials and methods section, higher levels of strategic reasoning in entry games can lead to non-threshold strategies. Moreover, non-threshold strategy participants (n = 8, participants with more than two drops) in the entry games showed higher expected payoffs compared to threshold strategy players [n =10 participants with at most two drops; two-sample Wilcoxon rank-sum (Mann–Whitney) test, z = 2.71, P = 0.0067]. Thus, the observed high proportion of non-threshold strategies in the entry games is consistent with higher levels of strategic thinking. Notably, we did not find any significant correlation across subjects between certainty equivalents for the lotteries (i.e. risk preferences) and frequencies of threshold strategies (i.e. level of strategic sophistication) in the entry games (Pearson correlation r = −0.21, P = 0.39; see also additional analysis in the Supplementary Material).

To conclude the behavioral section, we found similar behavioral patterns in lotteries and stag-hunt games, suggesting that the strategic situation is translated into a problem of risk by a majority of players measured through correlated certainty equivalences and (almost) perfect threshold strategies. In entry games, behavior was characterized by lower frequencies of B-choices and non-threshold strategies, thus reflecting higher strategic uncertainty and higher levels of strategic reasoning.

fMRI results

Common neural circuit involved in making risky decisions in lotteries and games

Behavioral results showed a general reduction of B-choices in entry games and a lack of correlation with lottery choices. Thus, we searched for brain regions displaying more activity during B-choices (i.e. reflecting higher riskiness in choice), independent of risk preferences, in entry games compared to lotteries and stag hunt. The behavioral data suggested also that strategic uncertainty in stag-hunt games can be reduced to situations of individual risky choices. To test this hypothesis at the neural level, we searched for brain regions displaying similar activity for B-choices during lotteries and stag-hunt games, and a stronger activation for RA than RL participants (with risk preferences measured through lottery choices; see fMRI method: GLM 1).

The statistical analyses revealed a brain network involving the bilateral anterior insula (aINS; peak MNI coordinates: right anterior insula: x = 36, y = 21, z = −15, Z-value = 4.37; and left anterior insula: x = −30, y = 18, z = −9, Z-value = 4.17; Figure 3), the dorsomedial prefrontal cortex (dmPFC; peak MNI coordinates x = 3, y = 24, z = 42, Z-value = 4.66; Figure 3) and the parietal cortex (angular gyrus; peak MNI coordinates x = 54, y = -54, z = 33, Z-value = 4.54). Consistent with behavioral responses, brain regions of this network showed: (i) higher activity during B-choices in the entry games compared to lotteries and stag-hunt games; (ii) same level of activity for RA and RL in the entry games and (iii) identical pattern of activity in the stag hunt and lotteries, with higher activity for RA compared to RL (i.e. selectivity for risk preferences) during B-choices for these two conditions (see Figure 3). Notably, we did not find any differential activity (with the exception of the occipital cortex, peak MNI coordinates x = 24, y = −78, z = 21, Z-value = 4.44) between RA and RL participants for A-choices in the three conditions (results from an additional analysis using GLM 1 with a contrast vector testing for a significant increase in BOLD activity for RA with respect to RL participants during A-choices; see fMRI method).

Fig. 3.

Neural circuit involved in making risky decisions in lotteries and games. (A) Activity in the anterior insula (aINS; MNI: 36, 21, −15 and −30, 18, −9) and dorsomedial prefrontal cortex (dmPFC; MNI: 3, 24, 42) correlates with riskiness of choice in lotteries and games (GLM 1). The bar graphs (B and C) indicate the brain activity (beta values SEM) related with B-choices (in each condition) of RA vs RL participants. The graph B plots the average activity of the left and right anterior insulae.

Neural circuit associated with degrees of strategic thinking in games

In the second analysis (see fMRI method: GLM 2), we studied how the degree of strategic thinking, reflected in threshold (Th) or non-threshold (nTh) strategies, interacts with strategic uncertainty in stag-hunt and entry games. We searched for brain regions displaying selectivity for strategy type (nTh vs Th) during choices in entry games. Interestingly, the dorsolateral prefrontal cortex (dlPFC; peak MNI coordinates x = 39, y = 18, z = 39, Z-value = 6.25) and the dorsomedial prefrontal cortex (dmPFC; peak MNI coordinates x = −6, y = 30, z = 42, Z-value = 5.22), in addition to the parietal cortex (angular gyrus; peak MNI coordinates x = −42, y = −69, z = 45, Z-value = 5.19) showed an increase for games with respect to lotteries and a selectivity for the strategy type in the entry game (Figure 4). Notably the region of interest (ROI) in the dmPFC shown in Figure 3 has a peak activity at MNI coordinates (x = 3, y = 24, z = 42) and contains 377 voxels, whereas the ROI in the dmPFC in Figure 4 has peak activity at MNI coordinates (x = −6, y = 30, z = 42) and contains 112 voxels. The distance between peaks is approximately 11 mm. The two ROIs overlap by 88 voxels. This represents approximately 23% of the first ROI and 79% for the second. Thus, the second ROI is largely included into the first one even though more than 20% of the voxels are separated. Therefore, the two ROIs do not completely overlap, but share a large portion of cortical surface (see Supplementary Figure S2). We thus found (i) enhanced activity of the dlPFC and the dmPFC for non-threshold compared to threshold strategy players in the entry game; (ii) lower level of activity in those regions for non-threshold strategy players in the stag-hunt games compared to the entry games (i.e. reflecting lower level of strategic reasoning in the stag hunt) and (iii) no differential activity for threshold players between the entry and stag-hunt games.

Fig. 4.

Neural circuit associated with degrees of strategic thinking in games. (A) Activity in the dorsolateral prefrontal cortex (dlPFC; MNI: 39, 18, 39) and dorsomedial prefrontal cortex (dmPFC; MNI: −6, 30, 42) from GLM 2 for the contrast non-threshold (nTh; i.e. high-level strategic thinking) vs threshold strategy players (Th; i.e. low-level strategic thinking) in the entry games. The bar graphs (B and C) indicate the brain activity (beta values SEM) related with choices (average activity of A-choice and B-choice in each condition) of non-threshold (nTh) vs threshold strategy (Th) participants. The categorization of threshold and non-threshold players was based on choice data from the entry games.

Discussion

We measured strategic uncertainty by eliciting certainty equivalents in two different games, the stag-hunt game and the entry game, analogue to measuring risk attitudes in lotteries, and we related observed behavior to neural activity measured through fMRI. We used lottery choices to estimate individual degrees of risk aversion and we used the frequency of threshold strategies as a measure distinguishing low from higher levels of reasoning in games.

A neural and behavioral measure of strategic uncertainty

We found similar behavioral patterns emerging from choices during lotteries and the stag-hunt games, while choices in the entry games reflected no correlation with risk attitudes. Similarly, the fMRI results demonstrated that a common brain network composed of the anterior insula, dorsomedial prefrontal cortex and parietal cortex (commonly associated with individual risk processing, see Knutson ; Preuschoff , 2008; Christopoulos ; Mohr ) is activated when choosing the uncertain option in individual and social contexts. Within this network, the role of the anterior insula is consistent with the integration of the representation/resolution of uncertainty and individual preferences towards risk (see Singer ). Our results suggest an extension of the role of the anterior insula in computing the riskiness of the choice option (risk prediction) to the social domain, when individuals try to reduce the complexity underlying social interaction to individual risky choices (as we observed in the stag-hunt game). The anterior insula thus reflects risk preferences and guides choice selection both in individual and social settings.

A common neural network for the resolution of social and environmental uncertainty was also found in the work of Behrens . In their study, they found two adjacent areas of the anterior cingulate cortex for action–outcome associative learning from personal experience (anterior cingulate sulcus) and social information (vicarious learning, reflected in the activity of the anterior cingulate gyrus).

In our study, the activity in this network was similar for B-choices in lotteries and stag-hunt games, while it was higher for B-choices in entry games, thus evidencing a neural correlate of the riskiness of payoffs and indicating higher strategic uncertainty. In addition, participants who were more reluctant to engage in risky choices (i.e. RA subjects) displayed stronger activity in the dorsomedial prefrontal cortex, and anterior insula with respect to RL players when choosing B in lotteries and stag-hunt games.

In entry games, we did not find dissociations at a neural level related to risk types. Overall, neural evidence suggests that strategic uncertainty in stag-hunt games reduces to risk and leads to behavioral patterns consistent with assigning subjective probabilities to the behavior of other players. In entry games, however, RA and RL participants had the same level of activity in this network when selecting the B-choice. This matches the lack of correlation between risk types and entry behavior. Thus, our results suggest that strategic uncertainty in entry games cannot be resolved by the neural machinery processing risk, and requires additional neural computations, most probably based on (strategic) deliberation.

Interplay between degrees of strategic thinking and strategic uncertainty

Whereas risk is typically studied using lotteries, strategic uncertainty is present in games with multiple equilibria, when even common knowledge of rationality and deductive reasoning do not provide a unique equilibrium (Van Huyck ). The coordination games, used in our study, are paradigmatic examples of games with multiple equilibria and strategic uncertainty (Ochs, 1995; Camerer 2003; Heinemann ). The two games differ in their strategic nature. Stag-hunt games are games of strategic complements, also called games with positive feedback (Woodford, 2003); for example an investment pays off if and only if a sufficient number of agents invest in the same industry, such that there are two Nash equilibria: ‘all invest’ (payoff-dominant equilibrium) and ‘nobody invests’ (risk-dominant equilibrium). Entry games are games of strategic substitutes, also called games with negative feedback; for example if too many agents invest in a new market, all get nothing, which leads to a mixed, or population, equilibrium where a certain number of players invest and the others stay out.

The two games differ in deliberation (i.e. degrees of strategic thinking) demands. In the stag-hunt game, a single and intuitive guess has to be made: ‘how many agents will choose to invest (i.e. B-choice)’? A typical line of reasoning in stag-hunt games is: ‘If I think that everybody invests then I should invest, and if I think that everybody thinks like me then I should still invest’; thus, in this game low level of strategic reasoning who have a high (low)-coordination belief and choose to invest (not to invest) and all higher-level players do the same choice. So low and high levels of reasoning correspond; put differently, further deliberation does not produce a different choice. This is an ideal condition where beliefs on others’ behavior can be based on subjective probabilities assessment.

In entry games, however, players may alternate between enter (i.e. B-choice) and no-enter (i.e. A-choice), etc.: ‘If I think that everybody enters then I should not enter, but if I think that everybody thinks the same then I should enter, etc.’ Thus, more deliberation in terms of iterated reasoning may affect behavior and generate, as observed in our study, higher expected payoffs in entry games. This suggests that the stag-hunt games require low degrees of strategic reasoning, while the entry games encourage higher degrees of (recursive) strategic reasoning (Camerer, 2003).

We found neural evidence supporting these hypotheses showing stronger activity during choices in entry games with respect to lotteries and stag-hunt games, for non-threshold (i.e. high level of strategic reasoning) compared to threshold (i.e. low level of reasoning) strategy players in the dorsolateral prefrontal cortex and the dorsomedial prefrontal cortex—areas associated with high level of strategic reasoning (Coricelli and Nagel, 2009; Bhatt ; Yoshida ). Overall behavioral patterns and brain activity in the entry games are associated with higher uncertainty and higher reasoning about beliefs of others’ beliefs and behavior, as reflected in dlPFC and dmPFC activity. This pattern of brain activity reflects the interaction between strategic thinking and uncertainty, thus more deliberation correlates with higher strategic uncertainty and with more sophisticated behavior in games. These findings suggest how the interplay between the neural substrates associated with risk and strategic thinking modulates choices in social settings.

More general, these results are consistent with the proposed key role of the dorsomedial and dorsolateral prefrontal cortex in performance monitoring and cognitive control in complex cognitive processes (Koechlin and Summerfield, 2007; Shenhav ). As described by Rushworth a reward-related activity is found in the anterior cingulate sulcus (which corresponds to the more dorsal portion of our dorsomedial prefrontal cortex) in choice under uncertainty. A recent study (Kolling ) shows the role of the dorsal anterior cingulate cortex (MNI coordinates, x = −2, y = 28, z = 36) in tracking the value of alternative course of actions in a foraging task involving risk. Activity of this area is associated with the relative value of riskier and non-default choices. Consistent with the findings of Kolling we found activity in the dorsomedial prefrontal cortex associated with the riskier choice (B-choice made by RA participants) both in lotteries and stag-hunt game (MNI coordinates x = 3, y = 24, z = 42, see Figure 3), and with increasing strategic uncertainty (reflected in non-threshold strategies) in the entry game (MNI coordinates x = −6, y = 30, z = 42, see Figure 4).

Thus, the comparison between entry games and stag-hunt games was crucial in our study to investigate the relationship between higher levels of reasoning and strategic uncertainty. This important feature was not present in a recent study by Ekins , where they compared only stag-hunt games and lottery choices and therefore could not identify brain regions related to strategic reasoning.

Notably, we found both areas commonly associated with ‘intuition’ (Kuo ), such as the insula, and areas associated with ‘deliberation’ (MacDonald ), such as the dorsolateral prefrontal cortex, in situations that require more deliberation and strategic uncertainty (i.e. the entry games). Our findings thus provide evidence in favor of an integrative view vs a dual system theory of decision making.

As a final remark, the B-choices in the stag-hunt game could be interpreted as altruistic behavior, because a B-choice raises expected payoffs for others, and participants might do so, even if the action is very risky. Similarly, subjects might abstain from entering in the entry game to give others a higher chance of winning. Thus, other-regarding preferences (Ruff and Fehr, 2014) can justify a higher number of B-choices in stag hunt and a lower number of B-choices in the entry game. Contrary to this hypothesis, we found no correlation between behavior (proportion of B-choices) in stag-hunt and entry games (Pearson correlation r = 0.0771, P = 0.76). Therefore, we have no behavioral support for a brain analysis on the role of other-regarding preferences.

Conclusions

Our study provides evidence about the notion that the brain network mediating risk during lotteries is also engaged in processing strategic uncertainty. Moreover, we showed how risk attitudes modulate the activity in relevant parts of this network both in lotteries (as previously shown by Rudorf ) and in the stag-hunt games. We thus showed for the first time how individual risk attitudes are reflected in the neural correlates of social/strategic uncertainty, and consequently how they affect behavior in social interaction. We conclude that games of strategic substitutes (entry games) create more strategic uncertainty as predicted by the nature of the theoretical equilibrium, which also involves higher order beliefs (i.e. beliefs about beliefs); whereas the strategic uncertainty of games of strategic complements (stag hunt) can be ‘reduced’ to standard risk. Our study thus supports a cognitive hierarchy theory of brain and behavior, according to which different levels of strategic thinking modulate the uncertainty underlying social interactions.

Click here for additional data file.