Understanding the flexibility of action-perception coupling.

The idea that observing an action triggers an automatic and obligatory activation of an imitative action in the motor system of the observer has recently been questioned by studies examining complementary actions. Instead of a tendency for imitation, cooperative settings may facilitate the execution of dissimilar actions, resulting in a relative disadvantage for imitative actions. The present study aimed at clarifying the contribution of associative learning and interference of task representations to the reversal of congruency effects. To distinguish between the two, an experiment was designed, in which we increased the effects of associative learning and minimized the effects of task interference. Participants completed a series of imitation and complementary action runs, in which they continuously imitated or complemented the actions of a virtual co-actor. Each run was alternated with a test run showing the same actions but including color-cues, and the participants were instructed to respond to color instead of the actor's posture. Reaction times to test runs showed no reversal of facilitation effects between the imitation and complementary action conditions. This result strongly argues that associative learning cannot adequately account for reversed facilitation effects. Our study provides additional support for action-perception models that allow flexible selection of action-perception coupling and challenges the existing models purely based on stimulus-response associations.

A remarkable amount of empirical evidence has been reported in the literature so far suggesting a close relation between action observation and action execution. Different studies show for instance that observing actions of other people stimulates motor areas in the observer’s brain (see Rizzolatti, Fogassi, & Gallese, 2001 for review). How exactly action observation and action execution relate to each other is still not fully understood. The dominant view in the literature is that observing an action triggers an automatic and obligatory activation of an imitative action in the motor system of the observer (e.g., Fadiga, Craighero, & Olivier, 2005). According to this view, activated motor representations map directly onto the observed actions, and a successful execution of an action that is different from the one observed would require inhibition of the automatic tendency for imitation (e.g., Brass, Zysset, & von Cramon, 2001a).

Recently, however, this direct and rigid view of action–perception coupling has been challenged by studies examining complementary actions. The basic idea is that, although imitation might be obligatorily introduced by action observation in some conditions, and although it might be beneficial for social development of individuals (e.g., Williams, Whiten, Suddendorf, & Perrett, 2001), an automatic tendency to imitate may actually impede successful cooperation between individuals. When working together, like for instance in joint action tasks, co-actors would mostly be producing complementary or opposite actions rather then imitating each other in order to complete a task successfully and reach the joint goal (Sebanz, Bekkering, & Knoblich, 2006). It seems that, in complex social situations, action coordination relies more strongly on actions complementary to the actions of others than on imitative actions (Newman-Norlund, van Schie, van Zuijlen, & Bekkering, 2007; Shibata, Suzuki, & Gyoba, 2007).

Emphasizing the role of complementary actions in action–perception domain, Graf, Schütz-Bosbach, and Prinz (2009) have just recently suggested that complementary actions have a great functional importance for human interactions. Understanding social interactions might benefit more from representations of complementary actions than previously believed. For instance, being able to predict someone’s action more precisely by watching the actions of another person involved in the same interaction might just make a difference between success and failure of that social interaction. Neri, Luu, and Levi (2006) have shown that visual discrimination of a human agent, presented as a masked point-light display, is influenced by the presence of a second agent. Namely, visual detection performance was better when the two agents were acting in a meaningful synchronized sequence, even though the temporal aspect of the stimuli was actually irrelevant for successful task execution. Neri et al. suggested that human implicit knowledge about the relations of actions between interacting agents generates the synchronicity effect observed in their study.

Additional evidence for the role of representations of complementary actions in human interactions was recently provided by van Schie, van Waterschoot, and Bekkering (2008). The authors demonstrated a reversal of automatic imitation effect when executing actions in a cooperative setting, suggesting a flexible action–perception coupling possibly influenced by task representations. In their study, participants either imitated (task A) or complemented (task B) a virtual co-actor’s grasp on a manipulandum. In both tasks, in 40% of all trials, a color-cue was presented, requiring the execution of a predefined grasp independently of the observed action of the virtual actor. Responses in these color (test) trials were compared between congruent (same) and incongruent (opposite) actions of the participant and the co-actor to determine automatic response activation in this cooperative setting. They report that responses in congruent trials were faster than in incongruent trials when these colored trials were presented in the context of the imitation task, indicating a congruency effect. This effect, however, completely reversed in the color trials presented in the complementary-action task. The reversal of congruency effects suggests that the cognitive system is able to circumvent the automatic tendency to copy directly actions or postures of others and to flexibly set task-goal appropriate stimulus–response (S–R) associations to optimize performance.

Van Schie et al. (2008) clearly demonstrate that the mapping between action observation and action execution is not obligatory or automatic but may reflect the nature of the task goal (e.g., imitation or complementary action). It seems again that the coupling between action and perception is more flexible than typically assumed in the imitation literature. Yet, the exact nature of this flexible action–perception coupling needs further specification. A possible explanation is that prolonged practice of specific S–R associations determines the way the congruency effect behaves. According to the associative sequence learning (ASL) theory (Heyes, 2001), the cortical connections mediating motor activation through action observation arise primarily through correlated experience of observing and executing the same actions. In other words, the benefit observed in congruent trials arises due to a lifetime of correlated experience between executed actions and perceptual consequences. Consistent with this view, Heyes, Bird, Johnson, and Haggard (2005) showed that practicing incongruent associations modulates automatic imitation. Specifically, the imitation effect was abolished after incompatible training. Furthermore, Catmur, Walsh, and Heyes (2007) demonstrated that training participants to perform a different action than the one observed reversed the imitation effect. Catmur et al. suggest that both imitation effect and its reversal are based on sensorimotor learning through social interactions with others. Both Heyes et al. and Catmur et al. demonstrate that associative learning could account for automatic imitation (through previously established S–R associations) as well as for its context dependent abolition or reversal (through recently established S–R association). It seems thus possible that the reversal of the congruency effect as reported by van Schie et al. was caused by associative learning. The study included a considerable amount of trials (120), in which participants executed and trained the complementary action task. In this way, these new S–R associations established in the complementary context might have caused the reversal of the congruency effect.

Although not denying the possible influences of learned S–R associations and experience (ASL), van Schie et al. (2008) argued for an approach less dependent on associative learning and more related to the goal level of tasks. The idea they put forward is that the successful reversal of congruency effect might be due to task interference effects arising in paradigms involving different tasks, one of which van Schie et al. applied in their study. Previous research using task-switching paradigms, in which flexible responding to stimulus features is required according to task-specific S–R mappings, has stressed the importance of the task history and the interference that might arise between the competing tasks (e.g., Allport, Styles, & Hsieh, 1994). Interference may occur both at the level of task representations and at the level of specific stimuli (e.g., Koch & Allport, 2006; Koch, Prinz, & Allport, 2005; Waszak, Hommel, & Allport, 2003, 2004, 2005; Wylie & Allport, 2000). For instance, Koch and Allport demonstrated stimulus-based priming of tasks in a task-switching paradigm. Their study showed that the imperative stimuli themselves could act as task cues and activate specific task representations reflecting the previous task context in which these stimuli occurred. It is thus well possible that, in the study of van Schie et al., task-specific representations were activated when stimuli that were needed for the primary task (i.e., imitation or complementary action) were included in a secondary task (i.e. responding to the color of the same stimuli). In this way, the imitation task may have facilitated the processing of congruent actions in intermixed test trials, whereas the same stimuli speeded up the selection of incongruent actions in the context of a complementary-action task.

The current study investigated in more detail the two possible theoretical explanations for the reported reversed congruency effects. We based our study on the study conducted by van Schie et al. (2008) and aimed at clarifying the contribution of exhaustive training of stimulus–response (S–R) mappings on the one hand and the contribution of task representation influences on the other to the reversal of congruency effects. To distinguish between the two, an experiment was designed, in which we (a) increased associative learning effects and (b) minimized task interference effects between the primary and the secondary task. As a result, finding a reversed congruency effect would only be possible to explain in terms of associative learning. On the other hand, a failure to replicate the reversal in congruency between the two conditions (imitation and complementary action) would point toward the implication of task-interference in the original study.

We increased associative learning simply by increasing the number of trials of the primary task in which the participants executed and trained the specific S–R associations. While van Schie et al. (2008) used 120 trials, we included 192 trials for imitation and complementary action each. Interference between the primary and the secondary task, on the other hand, was minimized by dividing test (color) trials and normal (imitation and complementary action) trials in separate runs. Just recently, Dreisbach and Haider (2008, 2009) have shown that specifying task representations, that is, the task rule that connects the stimuli on a more global level, allows for focusing of attention on task relevant stimulus features and prevents other possibly interfering stimulus features from being processed. In order to further maximize separation between task blocks, the task rules were explicitly introduced on screen before each run and the participants were instructed to actively prepare the upcoming task.

Our analysis focused on the test trials, comparing the congruent (executed action is the same as the observed action) and the incongruent (executed action is opposite to the observed action) trials in the imitation task context and the complementary-action task context. If action–perception coupling is sensitive to the activation of multiple interfering task rules that need to be kept online, separating the two tasks in different blocks should lead to a disappearance of reversed congruency effects. If, however, extensive training of specific S–R associations is the main determinant of action–perception coupling, we should find a similar reversal of the congruency effect as in van Schie et al. (2008).

Method

Participants

Twenty participants, 15 women and 5 men, were all undergraduate students from Radboud University Nijmegen. Their ages ranged between 18 and 28 years, with a mean age of 21.4 years. Two participants were left-handed, and the remaining 18 were right-handed. All participants had normal or corrected-to-normal vision.

Stimuli and tasks

We used the exact same stimuli as van Schie et al. (2008), which were images presented on a computer screen depicting a virtual co-actor and a manipulandum. The images were presented in pairs, with the first image depicting the co-actor in a non-active posture followed by the second image showing the co-actor grasping the manipulandum, using either a full grip or a precision grip (see Fig. 1). Participants were instructed to either imitate (task 1) or to complement (task 2) the grasping behavior of the co-actor. In test trials, the manipulandum in the image was presented in a deviant (green or purple) color. The participants were instructed to respond to the color in these trials irrespective of the behavior displayed by the virtual actor. The two colors were used as cues for a preinstructed grasping action, which was either full grip or precision grip depending on the instruction. The actual tasks were organized in runs of six trials. Before each run, the upcoming task (imitation, complementary action, or color task-run) was introduced on the screen by a cue. This task cue stayed on the screen as long as the participants needed to actively prepare the upcoming task run. Accordingly, the participants were explicitly instructed to use the cue presentation interval to prepare for the upcoming task run.

Fig. 1

Experimental stimuli and time course used in the study. Stimuli were images depicting the co-actor in a non-active posture (a) followed by a second image (imperative stimulus) showing the co-actor grasping the manipulandum, using either a full grip (b) or a precision grip (c). In test trials, the manipulandum was colored green or purple (c). The participants pressed the start button (1) to initiate a trial. After selecting the movement as the response to the second image, the participants released the start button, gripped the manipulandum using the appropriate grip (2), and returned to the initial position pressing the start button for the next trial (3)

Apparatus and procedure

The experiment consisted of a single session with one participant at a time. The participant was seated at a table facing a cathode ray tube computer screen (19 in. effective screen) at a distance of approximately 100 cm. To serve as a starting point, a response box was placed on the table directly in front of the participant and aligned with the participant’s midline. Between the response box and the computer screen, at a distance of 55 cm from the participant, a touch-sensitive manipulandum was fixed to the table (see Fig. 1). The manipulandum was custom-made to detect grasping actions of the participant either at the cylinder base (r = 3.0 cm; height = 8.0 cm) or at the top (r = 0.8 cm; height = 1.8 cm). A PC running Presentation software (Neurobehavioral systems Inc., Albany, CA) was used to detect release of the response box (reaction time, RT) and grasping of the object (movement time, MT and grip selection) at millisecond resolution.

Written and verbal instructions were given explaining the tasks and explicitly asking the participant to only initiate their action when being sure which action to perform and to avoid switching between grasps on the fly. This instruction was given to avoid the possibility that participants postponed their decision to the end of the grasping movement, making sure that RTs provided a true measure of response selection. Before the actual experiment, each participant practiced the tasks and was trained not to postpone the decision. Each trial began with the participants pressing the start button with their index finger and the presentation of an image showing the co-actor in a non-active posture (se Fig. 1). After 1 s, the image was displaced by a second image (imperative stimulus) showing the co-actor grasping the manipulandum at the top (precision grip) or at the bottom (full grip). Consecutive presentation of the two images generated apparent motion of the co-actor’s hand moving toward the manipulandum in the image. After making the decision, the participant released the start button, executed the required action by grasping the manipulandum with a precision or full grip, and returned to press the start button for the next trial. RTs were measured relative to the onset of the imperative stimulus, which remained on the screen until the end of the trial. Grasping errors were registered as well as early releases when the start button was released before the onset of the imperative stimulus.

The experiment included eight experimental blocks separated by breaks and consisting of 96 trials each. Imitation and complementary action tasks were separated during the experiment. This was done in such a way that, for half of all participants, the first four blocks included imitation and color runs (imitation blocks), while the last four blocks included complementary action and color runs (complementary action blocks). The other half of the participants followed the reversed order of imitation and complementary action blocks. Within an imitation block, participants alternated between eight imitation task-runs and eight color task-runs (test runs), always starting with an imitation task-run. So, within an imitation block, participants executed a total of 16 task runs, consisting of six trials each. The same structure was implemented within complementary action blocks. In this way, the experiment consisted of 32 imitation runs (i.e., four imitation blocks x eight imitation task-runs), 32 complementary action runs (i.e., four complementary action blocks × eight complementary action task-runs), and 64 (color) test runs (each of the eight experimental blocks included eight test runs). Half of all trials required a precision grip and the other half a full grip. Both the imitation condition and the complementary action condition were preceded by a practice block of 96 trials.

Data analysis

Trials with errors of grip and trials with RTs or MTs more than 2.5 SD from the participant’s mean were excluded from statistical analysis. RTs and MTs were analyzed with separate statistical designs using repeated measures ANOVA. For normal (non-colored) trials, a one-way repeated measures ANOVA with the factor Action context (imitation vs. complementary action) was conducted, whereas for (color) test trials a 2 × 2 repeated measures ANOVA was conducted, with the factors Action context (imitation vs. complementary action) and Congruency (same action vs. opposite action). Finally, we compared the test trial data of the present study with the data from Experiment 2 of van Schie et al. (2008) in which color trials were presented intermixed with imitation and complementary actions trials instead of separated in task runs. Here, we focused on possible differences in (reversal of) congruency effects between the two studies, with Study (current vs. van Schie et al.) as the between factor. An alpha level of 0.05 was used for all statistical tests in this study.

Results

On average, participants made a small percentage of errors (0.94%) in grip selection. Furthermore, 4.2% of the correct trials were not included in the analysis due to the 2.5 SD exclusion criterion applied to RTs and MTs.

The ANOVA applied to RTs to normal trials showed a main effect of action context, F (1, 19) = 5.43, p < 0.05, reflecting faster RTs for imitative than for complementary actions (see Fig. 2). For test trials, no significant main effect of action context was observed, F < 1. The ANOVA yielded a main effect of congruency only, F (1, 19) = 21.55, p < 0.001, indicating faster RTs for same (464 ms) than for opposite actions (476 ms). Critically, no significant interaction between action context and congruency was observed, F (1, 19) = 2.25, p = 0.15. This indicates that, when responding to color, participants executed the task faster when the required action of the participant matched the action of the virtual actor (relative to the condition when the executed action differed from the observed action) independently of the action context (see Fig. 2). The ANOVA applied to MTs showed no significant main effects or interactions.

Fig. 2

Means of average response time as a function of task (imitation and complementary action) for normal trials (dotted lines) and color test trials (solid lines). The data presented here are (a) from the study conducted by van Schie et al. (2008, Experiment 2) and our study (b, c) depicting the data for all trials and first trials, respectively. The test trials included predefined actions (either a precision grip or a full grip) independently of the action of the virtual actor. In this way, the action executed by the participant could either be the same as (filled squares) or opposite from (unfilled squares) the action of the virtual actor

To control for the possibility that the training effects on test trials were not detected due to spreading the analysis over all trials of a task run, we conducted the same analysis of RTs focusing on first trials only. This analysis showed that the main effect of action context in normal trials was not significant, F (1, 19) = 1.12, p = 0.30. For test trials, the analysis yielded a similar but smaller main effect of congruency as observed in the previous analysis that involved all trials that almost reached significance, F (1, 19) = 3.75, p = 0.068. This marginally insignificant congruency effect implies that also in first trials, participants tended to respond faster (515 ms) when the required action matched the observed action of the virtual co-actor than when the two actions did not match (523 ms). Importantly, also this analysis yielded no significant interaction between action context and congruency (F < 1, see Fig. 2), confirming the observation of similar congruency effect for both action contexts in previous analysis of all test trials.

Studies comparison

The ANOVA applied to RTs to test trials showed a main effect of study, F (1, 34) = 11.48, p < 0.005, reflecting generally faster RTs in the present study (470 ms) than in Experiment 2 (576 ms) of the study by van Schie et al. (2008). Importantly, a significant three-way interaction between action context, congruency, and study was observed, F (1, 34) = 66.34, p < 0.001. This indicates that, while in our study no reversal of congruency effects occurred, the data of Experiment 2 (van Schie et al.) demonstrate a significant reversal of congruency effects (see Fig. 2 for comparison).

Discussion

The present study focused on two possible theoretical explanations for the finding that action observation facilitates activation of complementary actions in cooperative settings (e.g., van Schie et al., 2008). Specifically, we investigated whether the reported reversal of congruency effects could better be explained as an effect of associative learning or as an effect of (interfering) task representations. To this end, we conducted an experiment, in which we minimized the possibility for task interference to exert an effect, while leaving room for associative learning mechanisms to reverse congruency effects between imitation and complementary action conditions.

Importantly, the experiment showed no reversal of congruency effects. Instead, a consistent effect of congruency was found over conditions, reflecting faster RTs to congruent as compared to incongruent movements irrespective of the context in which test runs were presented. An additional analysis focusing on the first trials of the test runs confirmed that perception-action associations built up during task runs did not differentially affect and transfer to test runs. Additional analysis comparing the data of the present study with the data of Experiment 2 of the study conducted by van Schie et al. (2008) confirmed the absence of the reversed congruency effects in our study. It thus seems that the faster responses to incongruent stimuli as reported in the study of van Schie et al. critically depend on the intermixing of task and test trials, and that these effects do not reflect the low level associations that may have been built up between stimuli and responses during task performance. Thus, the present results provide further support for the theoretical notion that task representations may temporarily overrule existing long-term S–R associations in accordance with task requirements and that this flexibility in action–perception coupling cannot adequately be explained in terms of associative learning effects.

Action accounts based on associative learning define actions as response to stimulation. Which response will be given to a certain stimulus depends on the strongest association the stimulus has developed through experience. Likewise, the ASL account (Heyes, 2001) suggests that actions are selected based on the strongest S–R association. Also any kind of flexibility in action selection occurs according to the ASL account purely through S–R associations. Specifically, the idea is that people can learn new associations (within the same or a new context), which are then active next to the already existing ones. Through extensive training the newly formed S–R associations get stronger and can therefore overcome old associations and get selected within the same action context. When acquired within a new context, the selection of a specific response is then determined by contextual cues. Such an account can explain many phenomena reported in the imitation literature, like automatic imitation (e.g., Press, Bird, Walsh, & Heyes, 2008), spatial compatibility (Brass, Bekkering, & Prinz 2001b), as well as abolishment (Heyes et al., 2005) and reversal (e.g., Catmur et al., 2007) of automatic imitation through practice. The challenge for the ASL account (as well as the other action models based on associative learning) arises when about the same amount of extensive incompatible training in some conditions does (van Schie et al., 2008) and in other conditions does not (our study) show the reversal of congruency effects. We suggest that the reversal of congruency effects as reported by van Schie el al. is due to task-representation effects governing the action–perception coupling in situations of more dynamic task requirements.

A long literature on cognitive control has shown that optimal task execution in a dynamic environment depends on an efficient interaction between maintaining the current goal and protecting it from distractions on the one hand and a change of focus, if required, on the other. Flexible cognition is certainly of high importance in many real-life situations that require context-specific goal-directed behavior. It is interesting to note here that, although the evidence for flexible cognition is well established in cognitive psychology (e.g., cognitive control literature), the research field of imitation and action observation has not really implemented this knowledge so far. Research on task switching, for instance, has shown that people are able to implement different task rules to the same stimulus set and to rapidly switch between the rules when required. In these settings, task history seems to play an important role. Previous activation of a particular task representation may facilitate the execution of a similar task or impede the execution of a competing task in paradigms involving dynamic task switching. In other words, previous activation of a task representation may persist and influence the execution of future tasks (c.f. Altmann & Gray, 2008).

The efficiency with which individuals may shield the task performance from previous task requirements likely depends on the temporal separation or overlap between tasks in time. In particular the requirement to maintain access to the mental representation of an ongoing task (i.e., task code) in episodic memory (Altmann & Gray, 2008) may make an individual particularly sensitive to the presentation of stimulus features that are associated with specific responses via the task code. Although the functional role of task representations, according to Dreisbach and Haider (2009), is to shield the system from processing task-irrelevant information, this will only filter out information that is irrelevant to task, while allowing task related stimulus features to activate their response codes. In the study of van Schie et al. (2008), the requirement to maintain a concurrent task representation probably allowed the task-irrelevant stimulus features of test stimuli (i.e., action postures) to activate their response codes and directly interfere with participants’ responses to the task-relevant stimulus feature (i.e., color). Although in the present study participants still frequently switched between the primary (respond to posture) task and secondary (respond to color) task, the occurrence of both tasks was highly predictable and furthermore cued in advance to maximize cognitive separation and minimize temporal overlap. Since the current design did not require participants to maintain different task representations activated concurrently, current task execution was less affected by interference arising from other task representations.

Furthermore, as a consequence of predictable task occurrence, participants probably better prepared the tasks in the present study. This might explain why task execution was generally faster in our study than in the study by van Schie et al. (2008). Koch (2005) has shown that task predictability can be used for effective task preparation (see also Poljac, Koch, & Bekkering, 2009) which may in turn further help task shielding by directing attention toward task relevant information while ignoring task-irrelevant information.

We should note that the explanation we propose for the reversal of congruency effects in terms of task-representation effects would certainly need further empirical testing. Our study provides direct evidence that the reversal of congruency effects cannot sufficiently be explained in terms of associative learning, but the evidence for the effects of task representations stays indirect. Our idea of overlapping task representations influencing the way that action–perception coupling takes place is in line with the ideas of shared representations proposed in the literature so far. It has been suggested that these shared (action) representations allow us to understand the actions, goals, and intentions of others (Rizzolatti et al., 2001). The nature of the shared representations has been suggested in terms of perceptual and motor events coded as a shared set of features (Hommel, Müsseler, Aschersleben, & Prinz, 2001). This theory of event coding (TEC; see also Hommel, 2009) proposes that the salience of particular features depends on task, context, and the direction of attention. According to TEC, this feature weighting can be influenced by both intentional (response related) and attentional (stimulus related) influences. Related to our study, an important and interesting question to address would be at which level overlapping task representations would affect action selection and execution. It is possible that overlap in task representations on the level of stimulus and/or response sets generates priming of tasks as already reported in the task-switching literature (e.g., Koch & Allport, 2006). On the other hand, the imitation literature provides evidence for the idea that task representations would (also) influence action selection on the level of action goals (e.g., Bekkering, Wohlschläger, & Gattis, 2000). Consistent with this to suggestion, Longo, Kosobud, and Bertenthal (2008) just recently reported that the default level of common coding in action imitation occurred at the level of action goals. Interestingly, the authors also showed that automatic imitation is modulated by top-down influences. They demonstrated that the level of action coding could be changed (e.g., toward coding on the level of movements) depending on task requirements. This again suggests more flexibility in action–perception coupling than commonly believed in the literature and supports our idea of task representations playing a major role in action–perception coupling in dynamic task conditions. Especially for cooperative settings, we would expect that the effects of task representations take place on the goal level, since dynamic interactions between individuals mostly involve execution of complementary actions with the aim to successfully accomplish a joint goal.

Another important finding in our study concerns the observed congruency effects.1 We found that the participants were generally faster in responding to color when action execution corresponded to the observed action, independently of the action context. This finding seems to support the idea that action observation automatically yields imitation in the observer (e.g., Fadiga et al., 2005). We think that action–perception coupling follows the automatic and direct processing in some conditions. Moreover, we think that the cognitive system allows for both automatic and flexible action–perception coupling. Consistent with our view, Erlhagen et al. (2006) have recently proposed a model that seems to implement both the direct (automatic) and the flexible action–perception coupling. The model was developed for learning by imitation in robots based on different behavioral and neuroimaging findings reported in the animal and human imitation literature. Their model involves four interconnected brain areas, namely the superior temporal sulcus (STS), area PF, area F5, and the prefrontal cortex (PFC). Based on the animal literature, the STS-PF-F5 connection is believed to represent the neural basis for a matching between the visual description of an action in area STS and its motor representation in area F5. This connection allows for the direct and automatic imitation of the observed action. Importantly, however, the model allows also for the flexible action–perception coupling through the reciprocal connections between PF and PFC. These connections make it possible to dominate the direct matching between STS and F5 if necessary. In this way, the model nicely implements two important physiological findings reported in the imitation literature and allows the plasticity of the system. Namely, the STS-PF-F5 route is consistent with the operation of the (primate and human) mirror neuron system (MNS) underlying the performance of both imitative (see e.g., Rizzolatti et al., 2001) and complementary actions (e.g., Newman-Norlund et al. 2007). Neuroimaging studies suggest that human MNS involves STS, the inferior parietal lobe (matching the animal PF), and inferior frontal gyrus (matching the animal area F5; see e.g., Rizzolatti & Craighero, 2004). Around one-third of mirror neurons seem to match directly the exact motor (i.e., effector and movement specific) properties of observed actions, while the remaining two-thirds are linked mainly to the goal of the action and reveal a much broader matching between action observation and execution (i.e., involving different effectors and/or different postures), which in principle could support incongruent action–perception mappings in joint action. Most importantly, the flexible action–perception coupling is realized in the model by the connection between the PF area and the PFC through which goal representations from PFC can modulate and set the coupling between visual (STS) and motor (F5) representations if required.

An important issue to address here considers the prediction of conditions in which the cognitive system might use one of the two action–perception routes. We would suggest that as long as task requirements are clearly defined and predictable (as in our study), task goals could easily be accomplished through the direct action–perception coupling. Dynamic environments, however, in which successful task execution depends on an efficient interaction between the activation of the current and (possibly different) future action goals (as in the study of van Schie et al., 2008) would probably require more controlled action–perception coupling. Our daily-life interactions with others would most probably need to use the connections between PF and PFC, which allow us to flexibly adapt action selection when needed.

In conclusion, our data argue for models allowing flexible action–perception coupling through dynamic implementation of task representations governing goal-directed actions in cooperative settings rather than (only) through associative learning.