| Literature DB >> 32341161 |
Atsushi Kikumoto1, Ulrich Mayr2.
Abstract
People can use abstract rules to flexibly configure and select actions for specific situations, yet how exactly rules shape actions toward specific sensory and/or motor requirements remains unclear. Both research from animal models and human-level theories of action control point to the role of highly integrated, conjunctive representations, sometimes referred to as event files. These representations are thought to combine rules with other, goal-relevant sensory and motor features in a nonlinear manner and represent a necessary condition for action selection. However, so far, no methods exist to track such representations in humans during action selection with adequate temporal resolution. Here, we applied time-resolved representational similarity analysis to the spectral-temporal profiles of electroencephalography signals while participants performed a cued, rule-based action selection task. In two experiments, we found that conjunctive representations were active throughout the entire selection period and were functionally dissociable from the representation of constituent features. Specifically, the strength of conjunctions was a highly robust predictor of trial-by-trial variability in response times and was selectively related to an important behavioral indicator of conjunctive representations, the so-called partial-overlap priming pattern. These results provide direct evidence for conjunctive representations as critical precursors of action selection in humans.Entities:
Keywords: EEG decoding; conjunctive representations; rule-based action
Mesh:
Year: 2020 PMID: 32341161 PMCID: PMC7229692 DOI: 10.1073/pnas.1922166117
Source DB: PubMed Journal: Proc Natl Acad Sci U S A ISSN: 0027-8424 Impact factor: 11.205
Fig. 1.(A) Event files and partial-overlap priming pattern. Shown are two examples from simplified situations with two possible action rules and response options, along with the idealized data pattern. The trial n-1 event file is shown as the yellow oval, the trial n event file as the green oval. In the first example, consecutive event files do not overlap, which, just as when there is complete overlap (not shown), allows efficient performance. In the second example, the response representation overlaps but the rule does not, leading to partial-overlap costs––just as would be the case with rule overlap but nonoverlapping responses (not shown). Note that examples of all possible trial-to-trial transition types in the current paradigm are embedded in Fig. 2. (B) Sequence of trial events in the rule selection task. (C) Spatial translation rules mapping specific stimuli to responses in Experiment 1 (horizontal, vertical, and diagonal rules). Two different cue words were used for each rule. Responses were made on four keys, each spatially compatible with one of the four possible stimulus locations. (D) Schematic steps of the representational similarity analysis. For each sample time (t), a scalp-distributed pattern of EEG power () was used in a first step to decode trial-specific rule/stimulus/response configurations, producing classification probabilities for each of the possible configurations. In the second step, these classification profiles for each trial and time point were simultaneously regressed onto model vectors for the potentially relevant representation. The figure shows all possible vectors as model matrices (the x-axis represents the correct constellations for the decoder to pick, and the y-axis represents the “confidence” with which each constellation is predicted). For each specific trial, the “vertical” vectors corresponding to the relevant action constellation are picked as predictors; that is, the red boxes show all four vectors for one specific action constellation, where the shading of matrix elements indicates the predicted classification probabilities, with darker shadings representing higher probabilities. Note that the conjunction matrix arises from multiplying corresponding elements of the constituent feature matrices with one another. The idealized classification profile represents an example in which a unique conjunction and rule information are expressed as a peak at the correct label of the S-R mapping and some confusion with incorrect instances of the same rule (i.e., horizontal). The coefficients associated with each predictor (i.e., in terms of t values) reflect the unique variance explained by each of the constituent features and their conjunction. (E) Spatial translation rules mapping specific stimuli to responses in Experiment 2. A word or symbol was used as a cue for each rule. As described in Methods, to achieve complete orthogonalization, rule-S-R constellations were divided into two groups of eight constellations each (G1 and G2) that were separately analyzed and then averaged. (F) Models for the RSA used within each of the two 8-constellation groups in Experiment 2.
Fig. 2.Mean RTs and errors as a function of all possible transition types in Experiments 1 and 2. Alongside each data point, examples for the corresponding trial-to-trial transition type are shown (Top, trial n-1; Bottom, trial n). Experiment 1 allows transitions resulting from crossing the rule repetition/change factor and the response repetition/change factor. Experiment 2 also allowed differentiation of pure stimulus changes and response changes in cases of rule change trials. However, for subsequent analyses on the relationship between conjunctive representations and overlap costs (Fig. 4), we focused on the transitions with complete S-R changes or repetitions. Error bars specify within-subject 95% confidence intervals.
Fig. 4.EEG-related results for Experiment 2. A–D, same as in Fig. 3., except that in contrast to Experiment 1, this experiment allowed differentiation between S-R and rule-S-R conjunctions.
Fig. 3.EEG-related results for Experiment 1. (A) Average single-trial t values associated with each of the basic features and their conjunction derived from the RSA analysis (Fig. 1). Shaded regions specify the SE around the mean. The colored squares at the bottom of the figure denote the significant time points using a nonparametric permutation test. (Inset) The same RSA fit scores when the conjunction was not included as predictor in the RSA analysis. (B) Time course of t values from multilevel linear models predicting the variability in trial-to-trial RTs (i.e., the impact of representations on behavior), using RSA scores of all features as the simultaneous predictors. (C) Average RSA scores of the conjunction model as a function of rule repetition/change and the response repetition/change factors for the early (0 to 300 ms) and late (300 to 600 ms) periods in the poststimulus interval. (D) Modulation of RT partial-overlap costs in trial n as a function of the strength of conjunction codes (median split) in trial n-1. Error bars specify within-subject 95% confidence intervals.