Can People Judge the Veracity of Their Intuitions?

People differ in the belief that their intuitions produce good decision outcomes. In the present research, we sought to test the validity of these beliefs by comparing individuals' self-reports with measures of actual intuition performance in a standard implicit learning task, exposing participants to seemingly random letter strings (Studies 1a-b) and social media profile pictures (Study 2) that conformed to an underlying rule or grammar. A meta-analysis synthesizing the present data (N = 400) and secondary data by Pretz, Totz, and Kaufman found that people's enduring beliefs in their intuitions were not reflective of actual performance in the implicit learning task. Meanwhile, task-specific confidence in intuition bore no sizable relation with implicit learning performance, but the observed data favoured neither the null hypothesis nor the alternative hypothesis. Together, the present findings suggest that people's ability to judge the veracity of their intuitions may be limited.

The British TV series Luther centers on DCI John Luther, a troubled detective working in a serious crime unit, who thanks to his superior intuition prowess, solves difficult cases, often finding himself operating at the fringes of the law (or beyond). Luther’s defiance and brute confidence is a defining feature of the character, but his religious reliance on instincts is reminiscent of many detective stories. For viewers or readers of such stories, it seems plausible enough that some individuals endowed with superior instincts know to trust their intuitions, perhaps encouraged by the steady recurrence of confirmatory outcomes—murderers getting caught and cases getting solved. However, moving beyond the realm of fiction and super instincts, the question arises whether in the absence of performance feedback individuals have the introspective insight to judge how good their intuitions are. In other words, do individuals’ metacognitive representations of intuition performance—how much certainty and confidence individuals express in their intuitions—predict actual performance outcomes? The present research addresses this question focusing on implicit learning performance.

Dual process models presume that automatic processes give rise to intuitive hunches or gut feelings (Chaiken & Trope, 1999; Epstein, Pacini, Denes-Raj, & Heier, 1996; Evans & Frankish, 2009). Quick, intuitive judgments are often contrasted with slow, elaborate processes, with the latter producing superior performance outcomes (e.g., Brewer, 1988; Devine, 1989; Fiske & Neuberg, 1990). Even when making decisions under imperfect conditions that favor intuitive processes (see Gigerenzer & Goldstein, 1996), people can lack the ability to adequately scrutinize their judgments (Fiedler, 2000; see also Nisbett & Wilson, 1977). Thus, individuals may lack an understanding of the, perhaps limited, veracity of intuitions (see Griffin, Gonzalez, & Varey, 2001).

However, some researchers have argued that people’s intuitive capabilities are often misrepresented in the literature and less erroneous than one might assume (Bowers, Regehr, Balthazard, & Parker, 1990). For example, studies on impression formation show that people’s impressions of others are often surprisingly accurate when formed on the basis of relatively sparse information (Ambady & Rosenthal, 1992). There is also evidence that individuals can monitor their performance and correct for biases. For example, when asked to judge whether a coin is biased in favor of heads or tails, people focus on the proportion of heads in the sample and then correct for the overall sample size (Slovic & Lichtenstein, 1971; Tversky & Kahneman, 1974). Other work demonstrates that people can readily adjust inferences drawn from intuitive feelings through associative learning (Unkelbach, 2007) or logic (Winkielman, Schwarz, & Belli, 1998).

People are relatively apt at extracting information from their gut feelings (e.g., Kelley & Jacoby, 1990), including information on the goodness or fit of mental processes (Whittlesea & Dorken, 1997). People can also readily express their preference for, and endorsement of, intuitions in self-report measures (Betsch, 2008; Epstein et al., 1996; Pacini & Epstein, 1999). One finding that emerges from studies using these measures is that people who consider themselves to be more intuitive (as opposed to rational) also tend to believe that their intuitions lead to good decision outcomes and have predictive validity (Pacini & Epstein, 1999). However, thus far the validity of these claims has mainly been examined in tasks geared toward producing biases or erroneous decisions (Epstein et al., 1996; Pacini & Epstein, 1999; Shiloh, Salton, & Sharabi, 2002). Arguably, such tasks are ill-suited to validate individuals’ explicit beliefs of intuition performance.

Some clues as to the link between individuals’ explicit beliefs and actual intuition performance may derive from work on the feeling of knowing, first studied in the context of episodic memory (Blake, 1973; Hart, 1967; Schacter, 1983). Metcalfe (1986) found that individuals were unable to judge their ability to solve “insight” problems in the future. Metcalfe reasoned these results could be due to the fact that the tasks at hand required sudden illumination, and it may be particularly difficult for individuals to predict Eureka-type phenomena.

Implicit learning tasks may provide a better outlet to probe individuals’ intuition performance and associated beliefs. In an implicit learning task, participants witness systematic associations between stimuli, often under disguise, and participants’ knowledge of the associative pattern is tested at a later stage. The tasks are designed so as to render the extraction of patterns or rules very difficult and too burdensome for slow, conscious processes. Knowledge of the rules is therefore assumed to be acquired implicitly (via System 1, see Kahneman, 2003), and intuitions are thought to give individuals “vague feelings” of the acquired knowledge (Reber, 1989, p. 233). Using the artificial grammar (AG) task (Reber, 1967), a standard implicit learning task that involves learning rules related to nonsensical letter strings, Dienes and colleagues demonstrated that individuals can judge their own test performance ex post facto (Dienes, Altmann, Kwan, & Goode, 1995; Dienes & Seth, 2010). This suggests that individuals may be able to judge the veracity of their intuitions (i.e., whether their feelings translate into correct answers). For example, stronger or more concrete feelings may give individuals greater confidence that their intuitions are correct.

However, responses in the AG task are not solely based on intuitions and instead derive from a mix of implicit and explicit knowledge (see Dienes, Broadbent, & Berry, 1991). Thus, explicit knowledge of some of the letter strings, but not feelings associated with implicit processes, may have underpinned individuals’ performance judgments. Furthermore, evidence for successful implicit learning can also be found even when individuals are tossing a mental coin (Dienes & Seth, 2010), presumably oblivious to the veracity of their intuitions. This suggests some dissociation between individuals’ metacognitive representations and intuition performance. In addition, asking participants to judge the correctness of their answers in an implicit learning task only provides an indirect measure of individuals’ explicit beliefs about their intuition performance. In particular, people’s mental representations of task performance may not encompass any “intuition” concepts.

Pretz, Totz, and Kaufman (2010) administered the AG task and a probabilistic serial reaction time task (Schvaneveldt & Gomez, 1998)—the two paradigms of choice to study implicit learning (Shanks, 2005)—to a sample of participants who also completed the rational–experiential inventory measuring individual differences in the (self-proclaimed) propensity to trust one’s intuition (see Pacini & Epstein, 1999). The authors found that individuals who expressed greater trust in their intuitions did not perform any better in the implicit learning tasks, r Combined = .13, 95% confidence interval (CI) = [−.08, .31].[1] This casts some doubt on people’s ability to judge how good their intuitions are. However, the absence of an association in a single sample (N adj = 100) is not sufficient to establish a null effect. Relatedly, the CI for the population parameter encompasses a wide range of values, calling for further studies.

The Present Research

In the current research, we sought to shed light on the question whether metacognitive representations of intuition performance—how much certainty and confidence individuals express in their intuitions—predict actual performance outcomes in an implicit learning task. We extend previous work in at least three ways. First, previous studies focused on implicit learning of nonsensical stimuli (e.g., letter strings). People seem to be more insightful, however, when relying on intuitions in social domains (e.g., Ambady & Rosenthal, 1992). Consequently, in the present research, we probed the validity of individuals’ beliefs regarding their intuition performance in tasks involving nonsocial (Studies 1a–b) and social stimuli (Study 2), thus providing a stronger test of the validity of individuals’ beliefs.

Second, we employed both generalized trait measures and task-specific state measures of how much faith individuals placed in their intuitions. This is a critical addition because the absence of an association between individuals’ beliefs and actual performance can be readily attributed to a lack of specificity in the trait measures employed to predict performance (see Ajzen & Fishbein, 1973; Pachur & Spaar, 2015). We sought to overcome this limitation by probing individuals’ confidence in their intuitions vis-à-vis the specific task at hand. In addition, in order to obtain a more complete picture of participants’ performance and corresponding mental representations, we included auxiliary self-report measures (detailed below) to help elucidate participants’ involvement with, and approach to, the task.

Finally, we extend the body of evidence currently available by carrying out a meta-analytic synthesis of all primary and secondary data to provide more precise estimates of the population parameters. Where applicable, we supplement our analysis with Bayes factor (BF01) calculations (e.g., Dienes, 2014). Following Rouder and Morey (2012), our null hypothesis corresponds to a nil point (ρ = 0) and the alternative hypothesis to a full range of positive and negative effects that follow a Cauchy distribution (see also Rouder, Speckman, Sun, Morey, & Iverson, 2009). We report BF01 for all nonsignificant effects (yielded using a frequentist approach) to specify the chances that the observed data occurred under the null hypothesis rather than the alternative hypothesis; values larger than 3 provide evidence in favor of the null hypothesis (e.g., Dienes, 2014). Note that a decision to favor the null hypothesis implies that a nil effect provides a better approximation than the alternative hypothesis, but it should not be taken as evidence that an effect is strictly zero in the population, which is implausible (Cohen, 1994). Also note that relative to a null hypothesis with an interval around zero, a null hypothesis with a nil point favors the alternative hypothesis and is therefore a more conservative approach to provide evidence for invariance (see Morey & Rouder, 2011; see also Lakens, in press, for a frequentist approach).

Method

Participants

One hundred and one students participated in Study 1a (87 female; M Age = 20.40, SD Age = 5.05), 77 students in Study 1b (61 female; M Age = 20.32, SD Age = 4.34), and 222 students and members of the public in Study 2 (170 female; M Age = 24.48, SD Age = 9.36). We only considered responses from participants who completed all sessions and all parts of the study (see Procedure). Students (n Total = 325) were recruited from a British University and took part in exchange for course credits. Members of the public (n Total = 75) were recruited from the United States and the United kingdom via Amazon Mechanical Turk (MTurk) and took part for a monetary incentive. The sample sizes of the individual studies provided 80% power at α = .05 to detect effects ranging from |ρ| = .19 (Study 2) to |ρ| = .31 (Study 1b).

Procedure and Materials

Studies 1a and 1b were conducted longitudinally in two consecutive sessions. At Time 1, participants completed the individual difference measures online. The implicit learning task was administered at Time 2 and carried out online (Study 1a) or in the laboratory under the supervision of an experimenter (Study 1b). Time 1 and Time 2 took place at least 1 day apart. Study 2 was conducted online within the same session, and the individual difference measures preceded the implicit learning task. Table S1 in the Supplementary Material provides an overview of all key variations between Studies 1a–b and 2.

Self-reported intuition

All three studies contained the English version of the Preference for Intuition Scale (Betsch, 2008), a 9-item questionnaire that measures endorsement of intuitive reasoning (e.g., “With most decisions, it makes sense to completely rely on your feelings”). Participants indicated their answers on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree). In addition, participants in Study 1b completed the Faith in Intuition Scale (Epstein et al., 1996), a 12-item measure of preference for intuitive thought and behavior (e.g., “I believe in trusting my hunches”). Participants responded on a 5-point scale ranging from 1 (completely false) to 5 (completely true). Finally, participants in Study 2 completed the experiential inventory (Pacini & Epstein, 1999), a 20-item measure of the extent to which individuals engage in, and excel at, experiential thinking (e.g., “I like to rely on my intuitive impressions”; “I hardly ever go wrong when I listen to my deepest gut feelings to find an answer”). Participants responded to all items on a 5-point scale from 1 (definitely not true of myself) to 5 (definitely true of myself).

Implicit AG learning task

Participants in Studies 1a and 1b completed a standard version of the AG learning task described in Abrams and Reber (1988) as a measure of implicit learning performance. The task involved learning associations between (nonsensical) letter strings that followed a rule or “grammar.” Participants in Study 2 completed a new, adapted version of the task that required participants to learn associations between profile pictures that are commonly used in online social media. The same finite-state grammar was used to dictate the order of the letter strings and the profile pictures (see Figure 1). In other words, the two versions of the AG learning task were almost identical, except that one version employed nonsocial stimuli (letter strings) and another version employed social stimuli (profile pictures). Performance was measured by the total number of correct responses, bound at 0 (minimum) and 46 (maximum), with scores above 23 indicating above-chance performance. A more detailed description of the task can be found in the Supplementary Material.

Figure 1.

Finite-state grammar underpinning the nonsocial (top panel) and social (bottom panel) version of the artificial grammar learning task (adapted from Abrams & Reber, 1988). Rule conforming and rule nonconforming example stimuli are shown on the right.

Task-related self-report measures

After the implicit learning task, participants’ beliefs were probed with regard to (a) their confidence in their intuitions (e.g., “To what extent did your gut feeling give you certainty in your answers?”), (b) their self-assessed performance (e.g., “How correct do you think are your answers?”), (c) their use of intuition as a basis for their decisions (e.g., “To what extent did you rely on your gut feelings to decide whether a string [Online Chat post] was grammatically correct or incorrect?”), and (d) the perceived benefits of further information about the underlying grammatical rule (e.g., “To what extent would reading further information on the rules that governed the order of posts been beneficial to perform well on the task?”). All constructs were measured with 2 items, except for the confidence in intuition measure in Study 2, where a third item was added as a safeguard to avoid a (hypothetical) situation where this key measure had low reliability. In addition, some minor changes were made to the wording of other auxiliary measures that yielded low reliability in Study 1b (see Supplementary Material—Data Preparation). In Study 1a, participants also indicated how much effort they invested in the learning task (“How much effort did you invest throughout the questionnaire to provide correct answers?”; single item measure). A similar, 2-item measure was used in Study 2 (“How much effort did you invest in the Online Chat task?”; “How much did you try to perform well in the Online Chat task?”). Study 1b was conducted in a controlled laboratory environment, which enabled us to use the amount of time participants spent on the practice and test phase of the AG task as an objective indicator of effort. All self-report responses were made on 9-point scales (see Supplementary Material).

Results

In our presentation of the results below, we take a cumulative approach (see Cumming, 2014) and use meta-analytic procedures to identify trends that generalize across studies, drawing on both primary (Studies 1a, 1b, and 2) and secondary (Pretz, Totz, & Kaufman, 2010) data wherever applicable. Since some variations in population parameters are to be expected between the different versions of the AG learning task (nonsocial vs. social) and the different modes of assessment (laboratory vs. online; longitudinal vs. single session), we employ random-effects meta-analysis models to derive the estimates of central tendencies (see Higgins & Green, 2011, Chapter 9).

Data Preparation

A description of the data preparation is provided in the Supplementary Material. Tables 1–3 display the reliabilities, descriptive statistics, and single-order correlations for all measures. CIs are provided in Tables S1–S3.

Table 1.

Internal Consistency, Means, Standard Deviations (SDs), and Zero-Order Correlations for All Measures in Study 1a.a

	Measure	1	2a	2b	2c	2d	3	4
1	Preference for intuition	(.78)	.13	−.07	.06	.06	.02	−.24*
2a	Confidence in intuition	—	(.78)	.59**	.31**	.03	.24*	.23*
2b	Self-assessed performance	—	—	(.91)	.08	−.04	.14	.28*
2c	Use of intuition	—	—	—	(.73)	.06	.22*	.13
2d	Benefits of further information	—	—	—	—	(.59)	.25*	−.01
3	Subjective effort	—	—	—	—	—	—	.05
4	AG learning performance	—	—	—	—	—	—	—
	Mean	4.82	4.41	4.29	7.33	6.88	7.02	27.50
	SD	0.77	2.14	1.89	1.63	1.74	1.40	4.32

Note. AG = artificial grammar.

aValues in parentheses are reliability estimates (so standardized and equivalent to effect sizes) and do not necessitate p values.

*p < .05. **p < .01.

Table 2.

Internal Consistency, Means, Standard Deviations (SDs), and Zero-Order Correlations for All Measures in Study 1b.a

	Measure	1a	1b	2a	2b	2c	2d	3	4
1a	Preference for intuition	(.74)	.58**	.23*	.11	−.12	−.04	−.12	−.06
1b	Faith in intuition	—	(.74)	.27*	.12	.01	.13	−.15	−.23*
2a	Confidence in intuition	—	—	(.70)	.32**	.21	−.05	.15	−.10
2b	Self-assessed performance	—	—	—	(.70)	.17	−.02	−.03	.09
2c	Use of intuition	—	—	—	—	(.38)	.09	−.04	−.15
2d	Benefits of further information	—	—	—	—	—	(.44)	.05	.04
3	Objective effort	—	—	—	—	—	—	—	−.08
4	AG learning performance	—	—	—	—	—	—	—	—
	Mean	4.72	3.50	4.66	5.00	7.03	7.37	3.63	27.36
	SD	0.72	0.48	1.84	1.72	1.52	1.55	1.09	4.29

Note. AG = artificial grammar.

aValues in parentheses are reliability estimates (so standardized and equivalent to effect sizes) and do not necessitate p values.

*p < .05. **p < .01.

Table 3.

Internal Consistency, Means, Standard Deviations (SDs), and Zero-Order Correlations for All Measures in Study 2.a

	Measure	1a	1b	2a	2b	2c	2d	3	4
1a	Preference for intuition	(.79)	.71**	.32**	.04	.19**	−.13	.02	−.01
1b	Experiential inventory	—	(.88)	.36**	.09	.25**	.09	.12	.05
2a	Confidence in intuition	—	—	(.90)	.52**	.32**	−.10	.16*	.10
2b	Self-assessed performance	—	—	—	(.84)	.14*	.01	.23**	.27**
2c	Use of intuition	—	—	—	—	(.62)	.00	.25**	.06
2d	Benefits of further information	—	—	—	—	—	(.73)	.05	−.02
3	Subjective effort	—	—	—	—	—	—	(.77)	.11
4	Social AG learning performance	—	—	—	—	—	—	—	—
	Mean	4.71	3.38	5.33	5.46	6.78	5.56	7.86	31.50
	SD	0.84	0.52	1.84	1.70	1.49	2.41	1.30	4.36

Note. AG = artificial grammar.

aValues in parentheses are reliability estimates (so standardized and equivalent to effect sizes) and do not necessitate p values.

*p < .05. **p < .01.

Implicit Learning Performance

Overall performance in the AG learning task was above chance level in all studies, suggesting that participants learned the underlying rule, M CorrectStudy1a = 27.50, SD = 4.32, t(100) = 10.45, p < .001, d = 2.09; M CorrectStudy1b = 27.36, SD = 4.29, t(76) = 8.93, p < .001, d = 2.05; M CorrectStudy2 = 31.50, SD = 4.36, t(221) = 29.07, p < .001, d = 3.91.[2] As anticipated, performance was enhanced in the social version of the AG learning task compared to the nonsocial version of task, Zs ≥ 4.20, ps < .001. This also held when restricting the samples to participants drawn from the same population, Zs ≥ 3.96, ps < .001. As can be seen in Tables 1–3, implicit learning performance shared no sizable relationship with subjective (Studies 1a and 2) and objective (Study 1b) effort, suggesting that more deliberative approaches to the task were not beneficial, r Combined = .04, n = 400, 95% CI [−.10, .18], Q = 2.06, df = 2, BF01 = 6.64. The BF01 suggests that the observed data are 6.64 times more likely to occur under the null hypothesis than under the alternative hypothesis, thus supporting the assumption of invariance (e.g., Dienes, 2014).

Beliefs in intuition

As shown in Figure 2, across studies we found little evidence that dispositional trust in intuition translated into superior learning performance. A meta-analysis of all primary (Studies 1a–b and 2) and secondary (Pretz et al., 2010) data indicates that there was no sizable association between dispositional intuition endorsement (i.e., Preference for Intuition, Faith in Intuition, Experiential Inventory) and implicit learning performance, r Combined = −.05, n = 500, 95% CI [−.17, .08], Q = 9.37, df = 5, BF01 = 6.02. Interestingly, the more participants placed trust in their intuitions, the worse they performed in the nonsocial version of the AG learning task (Studies 1a–b), r Combined = −.20, n = 178, 95% CI [−.34, −.06], Q = .94, df = 2. Meanwhile, performance in the social version of the AG learning task (Study 2) bore no sizable relation with intuition endorsement, r Combined = .02, n = 222, 95% CI [−.11, .15], BF01 = 6.55. These variations between studies (i.e., Studies 1a–b vs. Study 2) employing different versions of the same AG learning task (i.e., social vs. nonsocial) were significant, Z = 2.26, p = .024.

Figure 2.

Forest plot and random effects meta-analysis of the correlations between dispositional intuition endorsement and implicit learning performance. Observations denote pairwise comparisons. N is adjusted by dividing the number of participants in a given study by the number of measures included in the meta-analysis that were administered to the same group of participants (see Higgins & Green, 2011, Chapter 16).

Self-assessed performance correlated with actual performance on the AG learning task, r Combined = .24, n = 400, 95% CI [.14, .33], Q = 2.17, df = 2, replicating Dienes and colleagues’ findings (Dienes et al., 1995; Dienes & Seth, 2010; see Tables 1–3). However, as shown in Figure 3, there was no evidence that task-specific confidence in intuition was related to actual performance across studies, r Combined = .09, n = 400, 95% CI [−.07, .24], Q = 4.64, df = 2, BF01 = 2.17. The BF01 is below 3, suggesting that there is insufficient evidence to favor the null hypothesis over the alternative hypothesis. Consequently, we must entertain the possibility that confidence in one’s intuitions vis-à-vis the specific task provides an indication of actual intuition performance. However, as shown by the CI (r = .09 [−.07, .24]), in a given sample, the predictive validity can be expected to be low (r ≈ .10) and is unlikely to exceed r ≈ .25. We will return to the implications of this finding in the Discussion.

Figure 3.

Forest plot and random effects meta-analysis of the correlations between task-specific confidence in intuition and implicit learning performance.

Exploratory analyses

We conducted a series of exploratory analyses in the hope to shed some further light onto the lack of an association between people’s beliefs about their intuitions and actual intuition performance. First, as indicated earlier, participants’ judgments of how well they did on the task may reflect, at least in part, participants’ explicit knowledge related to the conscious acquisition of some underlying rules, which accounts for some (but not all) of the variation in performance in the AG task. Consequently, controlling for participants’ self-assessed performance statistically could be a way to discern variations in task performance that can be attributed to implicit processes. With this in mind, we regressed the AG learning task scores on the measures of dispositional intuition endorsement as well as participants’ self-assessed performance ratings. We conducted three separate ordinary least squares (OLS) regressions, one for each study (Study 1a, Study 1b, and Study 2), and then meta-analyzed the resulting semipartial correlation coefficients. In addition, we repeated the same procedure, substituting the dispositional intuition endorsement variables with the task-specific confidence in intuition measure. Controlling for participants’ self-assessed performance, the association between dispositional intuition endorsement and implicit learning performance remained unaltered, sr Combined = −.08, n = 400, 95% CI [−.20, .05], Q = 6.27, df = 4, BF01 = 2.63. Similarly, task-specific confidence in intuition had no sizable relationship with actual performance, sr Combined = −.03, n = 400, 95% CI [−.18, .11], Q = 2.04, df = 2, BF01 = 7.60, this time providing sufficient evidence to favor the null hypothesis.

People who were dispositionally inclined to place greater faith in their intuitions did not invest different levels of effort in the task, r Combined = .01, n = 400, 95% CI [−.12, .13], Q = 2.82, df = 4, BF01 = 8.86, but they expressed greater confidence in their intuitions in the AG learning task, r Combined = .27, n = 400, 95% CI [.14, .38], Q = 3.69, df = 4. To separate the contributions of the more general, dispositional measures and the more task-specific, state measures of intuition confidence to implicit learning performance, we regressed the AG learning task scores on the measures of dispositional intuition endorsement (i.e., Preference for Intuition, Faith in Intuition, Experiential Inventory) and the measure of task-specific confidence in intuition. We conducted three OLS regressions, one for each study, and then meta-analyzed the resulting partial correlation coefficients.[3] Neither dispositional nor task-specific manifestations of intuition confidence bore a significant relation with performance on the AG learning task; dispositional: pr Combined = −.09, n = 400, 95% CI [−.22, .04], Q = 8.41, df = 4, BF01 = 1.89; task-specific: pr Combined = .11, n = 400, 95% CI [−.03, .26], Q = 4.59, df = 2, BF01 = 0.87. However, once again, the existence of a small positive association between task-specific confidence in intuition and intuition performance could not be ruled out.

We also sought to rule out that nongrammatical stimuli misled participants’ intuitive judgments by examining participants’ performance on trials involving grammatical stimuli only. Zero-order correlations revealed that correct responses on grammatical trials were neither related to dispositional intuition endorsement (i.e., Preference for Intuition, Faith in Intuition, Experiential Inventory), r Combined = −.07, n = 400, 95% CI [−.20, .06], Q = 3.60, df = 4, BF01 = 3.51, nor to task-specific confidence in intuition, r Combined = .09, n = 400, 95% CI [−.05, .23], Q = 4.68, df = 2, BF01 = 1.89, although the latter measure provided insufficient evidence to favor the null hypothesis.

Discussion

The current research examined how people’s beliefs about their intuitions—how certain and confident people are in their intuitions—relate to actual intuition performance in a standard implicit learning task. To this end, we exposed participants to seemingly random letter strings (Studies 1a–b) or social media profile pictures (Study 2) that conformed to an underlying rule or grammar. Participants’ incidental learning of the rule was then probed in a subsequent test phase. Pooling data from three new studies (N = 400) and, where applicable, previously published work (Pretz et al., 2010), we found that people’s enduring beliefs in their intuitions were a poor guide to actual performance. In particular, people who were dispositionally inclined to place greater trust in their intuitions did not perform any better in the test phase of the implicit learning task than people who did not place such great trust in their intuitions. Further analyses suggested that this result cannot be attributed to variations in explicit knowledge nor to the fact that participants were misled by nongrammatical/rule-nonconforming stimuli, nor to differences in (subjective and objective) effort.

It may not come as a complete surprise that measures that tap into general dispositions fail to predict performance on a specific task (cf. Ajzen & Fishbein, 1973; Pachur & Spaar, 2015). To address the issue of domain-specificity, we also assessed task-specific confidence in intuition—a state measure of how confident and certain participants were in their intuitions during the test phase of the implicit learning tasks. The measure was administered after the implicit learning tasks, thus ensuring a high degree of familiarity with the task. In spite of these “favorable” conditions, pooled across studies task-specific confidence in intuition did not predict actual intuition performance. We also used statistical techniques to separate the unique contributions of general and task-specific beliefs in intuition to task performance. Once again, no reliable associations were found. However, using a Bayesian approach to test for invariance (Rouder & Morey, 2012), we could not affirm the null hypothesis.

It remains possible, and perhaps the most likely scenario, that individuals’ confidence in their intuitions vis-à-vis a task at hand provides some indication of actual learning performance. However, as outlined above in a given sample, the predictive validity can be expected to be low (r ≈ .10) and is unlikely to exceed r ≈ .25. A relatively weak relationship with actual performance implies that individuals’ task-specific beliefs in their intuitions can be frequently misleading. This can be illustrated with Rosenthal and Rubin’s (1982) binomial effect size display: Assuming, for the sake of illustration, a dichotomous outcome in the implicit learning task (correct judgments vs. incorrect judgments) and a clear separation of participants into those with high and low confidence, the difference in success between people with high and low confidence in a given sample is unlikely to exceed 25% (and equally likely to be zero). Put differently, 1 of the 4 times individuals with high confidence in their intuitions may perform better than individuals with low confidence in their intuitions. Conversely, 3 of the 4 times high (vs. low) confidence may not provide an indication of better performance (correct vs. incorrect). Note that these figures illustrate a “best-case” scenario; in a given sample, we would expect the difference between people with high and low confidence to be ∼10%, which would imply that 9 of the 10 times high (vs. low) confidence does not translate into superior performance.

Limitations

The present findings are limited to implicit learning of novel stimuli and should not be generalized to other facets of intuition, such as the intuitive decision-making of experts (e.g., Chase & Simon, 1973). Simon (1992) likened the expert intuitions to recognition, invoking memory-based processes of matching and information retrieval. In contrast, the present work placed an emphasis on information acquisition. It is interesting to note, however, that studies on expert intuitions often arrive at similar conclusions as the present research (cf. Einhorn & Hogarth, 1978). Reviewing research on expert intuitions, Kahneman and Klein (2009) concluded that subjective confidence provides an “unreliable indication of the validity of intuitive judgments and decisions” (p. 524).

On a related note, it is possible that individuals can calibrate their mental representations of intuition performance through feedback, and this may contribute to align individuals’ beliefs with actual performance. However, external feedback can also exacerbate biases and foster overconfidence and false beliefs (Arkes, 2001; Fiedler, 2000; Hogarth, 2001). Further studies are needed to gain a fuller and more complete understanding if and under what conditions individuals are able to judge the veracity of their intuitions.

Conclusions

People readily express their preference for, and endorsement of, intuitions in self-report measures (Betsch, 2008; Epstein et al., 1996; Pacini & Epstein, 1999).[4] The more individuals perceive themselves to be intuitive (as opposed to rational), the more they tend to believe that their intuitions have predictive validity and lead to good decision outcomes (Pacini & Epstein, 1999). When putting these assumptions to a test in an implicit learning task, we found no evidence that individuals’ dispositional trust in their intuitions was warranted. Furthermore, while confidence in one’s intuitions vis-à-vis a particular task at hand may bear some relation to actual intuition performance, the predictive validity is likely low and may lead to frequent misjudgments.