Great Minds Think Alike? Spatial Search Processes Can Be More Idiosyncratic When Guided by More Accurate Information.

Existing research demonstrates that pre-decisional information sampling strategies are often stable within a given person while varying greatly across people. However, it remains largely unknown what drives these individual differences, that is, why in some circumstances we collect information more idiosyncratically. In this brief report, we present a pre-registered online study of spatial search. Using a novel technique that combines machine-learning dimension reduction and sequence alignment algorithms, we quantify the extent to which the shape and temporal properties of a search trajectory are idiosyncratic. We show that this metric increases (trajectories become more idiosyncratic) when a person is better informed about the likely location of the search target, while poorly informed individuals seem more likely to resort to default search routines determined bottom-up by the properties of the search field. This shows that when many people independently attempt to solve a task in a similar way, they are not necessarily "onto something."

Introduction

A vast body of research on human decision‐making demonstrates that, when lacking sufficient information as to which choice option is best, we tend to look to others for guidance. For instance, in information and availability cascades (Anderson & Holt, 1997; Kuran & Sunstein, 1998), people see the fact that the same choice is made by a large number of their peers as a sign that they must “know something” and are likely to copy that choice (e.g., to buy the most popular product in a category on Amazon). Similarly, as seen in crowd attention studies (Sun, Yu, Zhou, & Shen, 2017; Sweeny & Whitney, 2014), people are likely to look at whatever a large portion of the crowd pays attention to. In other words, we tend to copy those patterns of behavior or attention that are commonly seen in others, rather than the unusual, idiosyncratic ones, deeming the latter less likely to lead to success.

However, research suggests that, in a variety of contexts, idiosyncratic behavior is not associated with poor performance. For example, when making judgments based on viewing others' faces, people scan the face images with their eyes in a highly idiosyncratic manner, but all these fingerprint‐like unique information search patterns lead to similarly good task completion (Mehoudar, Arizpe, Baker, & Yovel, 2014; Peterson & Eckstein, 2012). In fact, the increased similarity between the visual search patterns of people who collaborate on a task may be related to decreased performance (Coco, Dale, & Keller, 2018). Even more strikingly, in the context of research on the processing of various tasks by experts as opposed to novices (see, e.g., Richstone et al., 2010; Wolff, Jarodzka, van den Bogert, & Boshuizen, 2016), it has been noted that information search patterns of different experts are less similar to each other than those of novices (Jarodzka, Scheiter, Gerjets, & van Gog, 2010). Recently, Krol and Krol (2020) demonstrated that those decision‐makers who were better informed beforehand searched for further information with their eyes in a more idiosyncratic manner. These outcomes were explained in terms of prior information (or expertise) triggering a greater degree of top‐down control of subsequent search, driven by the person's individual characteristics and experiences rather than steered bottom‐up by the features of the stimuli, which are the same for all people viewing them. Relatedly, other recent work demonstrated that computational noise is a core feature of human learning in volatile environments and that an increase in that noise accompanying a high learning rate is a significant source of behavioral variability observed in reward‐guided decisions (Findling, Skvortsova, Dromnelle, Palminteri, & Wyart, 2019).

In our opinion, the potential association between the acquisition of knowledge and idiosyncratic processing could also explain the apparent difficulty of researchers to pinpoint the features of task processing that distinguish experts from novices (Brams et al., 2019). To put it simply, the source of the difficulty might be that the main thing common to experts is that they have relatively little in common. If so, then our tendency to copy commonly seen rather than idiosyncratic patterns of behavior could in some situations be counterproductive, leading us to follow poorly informed individuals.

A well‐known problem, where the relationship between knowledge, experience, and idiosyncratic behavior is particularly interesting, is spatial search, a real‐world example of which is digging for gold within a land area divided into smaller plots. Humans faced with this kind of task in experimental settings are known to display foraging‐like behavior, where their decisions on which plot to focus on next are influenced by knowledge of previously found targets' locations. Such experience‐driven processes were observed by researchers regardless of whether the search was carried out via mouse clicks (Kerster, Rhodes, & Kello, 2016), eye movements (Cain, Vul, Clark, & Mitroff, 2012; Najemnik & Geisler, 2005), or even inside a person's own mind (Todd & Hills, 2020).

But suppose that some individuals have better prior knowledge than others about where targets (in the example, gold) have recently been found. In what way would their search trajectories (i.e., which plots they explored and in what order), differ from those of their less well informed counterparts? If the prior knowledge is highly accurate—for example, gold has been found several times in a specific plot and has never been found elsewhere—we would expect the informed individuals' efforts to converge on this high‐reward location (in foraging models, this is known as local exploitation). This, in turn, would make their search trajectories similar to each other and less idiosyncratic in terms of their overall placement when compared to people with no information on where to look, who would likely rely on a less focused, global exploration of the search space.

Nevertheless, in light of the aforementioned research linking idiosyncratic behavior with expertise, we hypothesize that search trajectory features other than their overall placement, like their shape and temporal ordering, are still going to be more idiosyncratic in well‐informed individuals. If true, this would indicate that those individuals may be distinguished from their less well informed counterparts even when the environment is volatile and prior information imprecise. In our example, if gold has been found in several different plots across the search area, people who are aware of these locations will not necessarily converge on the same spot at the same time, but might still be identified based on the fact that they will move between different plots in an idiosyncratic manner.

To test this hypothesis, we conducted an online experiment and recorded the mouse click sequences that participants used to explore a rectangular grid in search of a target, after receiving information about its likely location. We manipulated the quality and quantity of that information and used a novel technique to measure the associated changes in the extent to which the shape of the search trajectory was idiosyncratic. By statistically testing the effect of prior information features on the proposed measure, the paper presents an approach that turns idiosyncratic behavior from a source of noise obstructing research into its subject and instrument.

Methods

Participants

A power analysis of a pilot study (with 20 participants, but otherwise an identical design as the main study) indicated a main effect (information absent vs. present) of sufficient size for the required sample size to be less than 100 given a two‐tailed alpha = 0.05 and power = 0.90. Due to the computational load of the data analysis pipeline increasing exponentially with sample size (see below), this was also our maximum capacity in this respect.

Accordingly, for the main study, we used the Prolific platform to recruit a sample of 100 adults, English‐speaking participants with a 100% platform approval rating (mean age 33.2, SD = 12.2, 54% female participants). Of these, eight participants did not meet the data quality criteria (their answer accuracy rate was below chance), leaving a final sample of N = 92. The study took approximately 80 min to complete.

Stimuli and design

In a repeated measures design, participants each attempted a number of visual search trials, looking for a single target—a red circle—among identically shaped black distractors in a 5 9 grid. Initially, all circles were shown as gray, but with a click the participant could flip each circle for 1 s to see if it was black or red underneath (this could be done any number of times, and one could flip the same circle more than once). At any time, the participant could answer whether they thought there was a single red circle somewhere in the grid (participants were informed that this would happen in a randomly chosen 50% of the trials and that otherwise all circles would be black). Each correct answer was worth a fixed monetary amount, adding up to a maximum of 6 GBP over all trials, in addition to a 6 GBP “show‐up fee.”

Before initiating each search, the participant had to click through 12 “prior information” slides, some of them blank, and others each containing a hint, that is, an example location of the target in the subsequent search grid.

In addition to the presence of the target, we manipulated the number of prior information slides that contained hints (“information quantity”). This was either 0 (“none”), 4 (“low”), 8 (“medium”), or 12 (“high”), with the blank slides inserted at randomly chosen slots in the sequence. We also manipulated the “quality” of the information, either “low” or “high,” which determined how well the distribution of the example locations matched that of the actual target.

Specifically, for each participant and each combination of information quality, quantity, and “target‐present,” the 9 5 grid was randomly partitioned into nine plots of five cells each using the K‐Medoids clustering algorithm with random seeding (Kaufman & Rousseeuw, 2009). (This clustering algorithm was chosen as it made it possible to specify the exact number of clusters and ensured that the plots were approximately square‐shaped.). Each of the nine plots was then used to generate a separate trial, where the target—if present—was drawn from the five cells that comprised the plot with equal probability (thus, regardless of the condition, the target was equally likely to be found in any part of the grid).

The resulting total number of trials was

In the “low‐quality” condition, half of the hints (i.e., two, four, or six hints, depending on information quantity) would be drawn in the same way as the corresponding target, that is, from the cells comprising the plot used to draw the target, while the remaining hints were drawn from all the remaining cells (with equal probability). The proportion of hints that matched the distribution of the target in this way would increase from half to three quarters in the “high‐quality” condition. (See Fig. 1 for an illustration of how this was presented to the participants.)

Fig. 1

An example single‐trial display sequence under high information quality—medium quantity condition. The participant first sees a total of 12 prior information slides, four of them blank, and two of the remaining eight (slides 3 and 10) pointing away from the true target location. The participant then clicks in the search panel (bottom) to flip individual circles (in the example, the clicked circle turns black, as the target is at a different location, marked with a red cross).

Data processing

The procedure outlined here is an adaptation of the one in Krol and Krol (2020) for the purpose of analyzing mouse clicks instead of eye‐movement sequence data. The pre‐registered data analysis code is available on the OSF page.

For each target search, we first partitioned the sequence of cells clicked in the search panel into subsequences of length = 3 (the highest computationally feasible number—see below). To separate their shape and temporal properties from their overall location, we recorded the cell coordinates relative to the spatial median of each subsequence. Thus, for example, any two clicks aimed at one cell to the right of the median were deemed equivalent regardless of the respective medians' locations.

All recorded subsequences were then compared pairwise using the Needleman‐Wunsch (1970) algorithm (requiring close to 370 million runs of the algorithm, a figure that would rise to several billion if longer subsequences were used). The algorithm aligns the compared sequences (by deleting or shifting their elements) to maximize the number of matching elements and outputs the resulting similarity score. Simply put, two subsequences would receive a high similarity score if, in each case, cells that were similarly placed relative to their respective spatial medians were visited in a similar order. Using a different similarity metric did not seem to affect our findings. The results of all pairwise comparisons were stored in a similarity matrix for further use.

In particular, we then used the t‐distributed stochastic neighbor embedding algorithm (t‐SNE; see van der Maaten & Hinton, 2008) to map each subsequence to a point in a two‐dimensional space in a way that minimizes the Kullback–Leibler divergence between the distributions modeling the original versus dimension‐reduced data (the former represented by the similarity matrix obtained as described above). That is, similar subsequences were mapped to neighboring points on the plane. We then estimated the distribution of the obtained low‐dimensional sequence representation points. This was achieved using non‐parametric smooth kernel density estimation with spherical Gaussian kernels of fixed, automatically determined size (set to a value such that a kernel centered around a training example contains an average of 10 other examples).

Finally, we used the “RarerProbability” algorithm, embedded in the Wolfram Mathematica software package, to compute the probability of generating (from the estimated distribution) a sample with a lower density than each of the obtained subsequence representation points. In simple terms, this algorithm is a multivariate generalization of a two‐tailed p‐value, giving a score between 0 (an extremely unlikely multivariate outcome) and 1 (a likely one). Due to the non‐normal distribution of rarer probability within trials, for each trial, we calculated its median value over all subsequences that comprised the corresponding search trajectory. We subtracted this probability from one to obtain a measure of the extent to which the search trajectory was idiosyncratic and will refer to it as the “search atypicality” index. For an overview of the described procedure, see Fig. 2.

Fig. 2

An illustration of the data analysis procedure. Two search trajectories in the top panel (green and red) are initially similarly shaped and ordered relative to their (different) overall locations. That is, their corresponding initial subsequences (clicks 1–3, marked with dark colors), have similar X/Y coordinates. These coordinates are set relative to the subsequences' respective spatial medians (in the examples, these correspond to item 2 in each case). For example, “0,‐1” means “directly to the left of the median” (see panels A/B in the top‐right). Because of these similarities, the two subsequences have a high Needleman–Wunsch similarity score, and so are mapped by t‐SNE to adjacent points on the plane (see the bottom panel). Being located in a higher density area of the corresponding distribution, they have lower search atypicality scores (i.e., are less idiosyncratic, or more common) than the final subsequence (clicks 4–6) of the red trajectory, represented by point C.

Results

Overall the most common search trajectory was flipping successive circles horizontally, from left to right. Irrespective of the experimental condition, ranking subsequence shapes in terms of their frequency of occurrence resulted in exactly the same list of most common patterns. At the same time, as can be seen in Fig. 3, the prevalence of the common search patterns generally seems to decrease as information quantity increases. To test the statistical significance of this claim, we estimated a mixed‐effects regression with random subject effects, the overall (aggregated) frequency of the nine most common subsequence shapes seen in Fig. 3 as the dependent variable, the information quantity as the main independent variable, and the overall length of the search sequence (number of clicks during the trial) as an additional control. This showed that the common subsequence shapes occurred less frequently when information quantity was higher ().

Fig. 3

The frequency of occurrence of the nine most common three‐element search subsequence shapes across experimental conditions. The box in the top‐right corner of each panel depicts the respective search pattern.

The claim that, rather than changing the search trajectories' typical characteristics, information instead clears those characteristics, making the search more idiosyncratic, can be further examined using the proposed search atypicality index. To begin with, the average values of the index across the experimental conditions are shown in Table 1.

Table 1

Mean values of the search atypicality index across experimental conditions

		Information Quality
		Low	High
Information quantity	None	0.297
	Low	0.382	0.406
	Medium	0.398	0.404
	High	0.403	0.426

To test the significance of the differences across conditions seen in Table 1, we first estimated a mixed‐effects regression with random subject effects, the search atypicality index as the dependent variable, information quantity as the main independent variable, and the overall sequence length as an additional control. This showed that the search subsequence shapes were more idiosyncratic when information quantity was higher (, p < .001).

Next, we incorporated the effect of information quality (i.quality = 0/“low”, 1/“high”) into the analysis. This has no bearing on trials in which the quantity of information was zero because the quality of the hints does not matter when no hints are shown. Thus, we dropped those trials from further analysis and estimated a mixed‐effects regression with random subject effects, search atypicality as the dependent variable, sequence length as a control, and both information quantity and quality (as well as their interaction) as independent variables.

The regression estimation results are shown in Table 2. Search atypicality increased when the overall sequence length was smaller (). Controlling for sequence length, search atypicality increased both when more hints were shown () and when the hints more accurately revealed the target's location (). However, the interaction between information quantity and quality was not significant ().

Table 2

Regression estimates of search atypicality

	β	SE	t	p
Intercept	0.5279	0.0144	36.531	<.001*
i.quantity	0.0282	0.0067	4.185	<.001*
i.quality	0.0179	0.0062	2.844	.004*
i.quantity*i.quality	0.0028	0.0096	0.294	.769
seq.length	−0.7215	0.0196	−36.707	<.001*

Note: *p .05.

Discussion

Previous research demonstrated the stability of information sampling strategies within‐ (Boot, Becic, & Kramer, 2009) but a considerable variation between individuals (Irons & Leber, 2020), to the extent that one's attentional patterns may constitute a “signature” distinguishing her from others (Bargary et al., 2017). Despite these individual differences, researchers have successfully examined how the average characteristics of the sampling process, like the shape of the mouse click search path, are influenced by environmental features such as the spatial distribution of the targets (Kerster et al., 2016). It has been shown that sampling is guided by previously acquired information, particularly the previously encountered or primed search target locations (Geng & Behrmann, 2005; Talcott & Gaspelin, 2020).

Nevertheless, so far, the focus of this area of research has been on how experimental conditions of interest affect information sampling on average, rather than how they affect the variability of this process across people or the likelihood of a person's search strategies being idiosyncratic. In contrast, based on our short study, we would argue that the extent to which the sampling process is idiosyncratic is, in itself, an important property that can serve as a variable of interest, rather than being a source of noise that obstructs research on human decision‐making.

In particular, we showed how, thanks to machine‐learning techniques, one can combine complex spatiotemporal features of a search path into a single metric that describes how unusual that path is in the population, much like what the p‐value does for simple numerical measures. More precisely, dimension reduction techniques were used to construct a “density map” representing the distribution of all three‐click sections of the recorded search trajectories, accounting for both their spatial and temporal features, but disentangling the shape of the section pattern from where in the search field it was positioned. We then used a multivariate analog of the p‐value to obtain, for each individual trajectory, a measure of how probable (i.e., typical rather than idiosyncratic), it is within the estimated population distribution. In general, such a metric can be used to investigate how behavioral or information‐processing idiosyncrasies are influenced by factors of interest, or how these factors may be reverse inferred from the metric.

Here, we hypothesized that being a priori better informed about the likely location of the search target would result in a more idiosyncratically shaped search path. We found that the hypothesis is supported by evidence, that is, by the fact that both the quantity and the quality of the hints given at the start of a trial were positively related to the search atypicality index (controlling for the overall length of the search sequence).

The likely reason for this was that, as hypothesized, poorly informed agents were more reliant on default search routines dictated by the fixed, uniform features of the search field (e.g., scanning from left to right in straight lines), rather than on their individual interpretation of noisy prior information. This explanation is supported by the fact that the frequency of occurrence of the most common subsequence shape patterns decreased with information quantity. Rather than changing the search trajectories' typical characteristics, information instead clears those characteristics, making the search more idiosyncratic.

In other words, we do not, and cannot, point to any specific spatiotemporal patterns in search behavior that would be signatures of being well informed. Quite the contrary, what well‐informed search paths seem to have in common is that they have relatively little in common, that is, every informed search path is informed “in its own way.” This may be related to the work of Findling et al. (2019), who found computational noise to be a core feature of learning, and suggested that an increase in that noise accompanying a high learning rate could be a significant source of behavioral variability in reward‐guided decisions. Furthermore, as recently demonstrated by Wu et al. (2021), in random, dynamic environments, people are less reliant on observing others to obtain information, while balancing individual and social learning to avoid correlated social information and maladaptive information cascades. This may be beneficial if—as our results suggest—common trends seen in social information may systematically arise from the behavior of those individuals who are relatively poorly informed.

In terms of the limitations of the present investigation, an alternative explanation of the results could be that the “informed” search paths are informed by diverse (randomly drawn) sets of hints (as opposed to the uninformed search paths being “informed” by the same blank slides). However, this interpretation seems less likely because, when the information quality is low rather than high, any hints that are shown are more diverse (due to a greater degree of noise), and yet the search trajectories are less idiosyncratic.

A more significant limitation is the fact that we presented a highly specific search task, one in which the search field was quite small (constraining the search path shapes), had a quite rigid chessboard layout, and contained at most one target. It may be that other search environments, like the one analyzed by Kerster et al. (2016), would produce qualitatively different results. Along the same lines, one can envisage using other metrics of “atypicality.” In particular, the spatiotemporal properties of information sampling processes could be modeled and represented in other ways, for example, using hidden Markov models (Coutrot, Hsiao, & Chan, 2018), and the probability of occurrence of a given pattern in a population could be assessed with a variety of machine‐learning anomaly detection techniques (Chandola, Banerjee, & Kumar, 2009).

Despite the above limitations, this short communication provides an early indication that the shape and temporal structure of search paths—when disentangled from their overall location—is more idiosyncratic when the search is guided by more accurate information. This suggests that recent eye‐tracking work with text stimuli can extend to other interfaces and domains, although much more work is needed to fully establish the scope of these findings. More generally, it might be fruitful to further explore how an individual's cognitive processes are related to those of her counterparts, exploiting cognitive idiosyncrasies as a source of predictive power instead of noise.