Conceptual knowledge for understanding other's actions is organized primarily around action goals.

Semantic knowledge about objects entails both knowing how to grasp an object (grip-related knowledge) and what to do with an object (goal-related knowledge). Considerable evidence suggests a hierarchical organization in which specific hand-grips in action execution are most often selected to accomplish a remote action goal. The present study aimed to investigate whether a comparable hierarchical organization of semantic knowledge applies to the recognition of other's object-directed actions as well. Correctness of either the Grip (hand grip applied to the object) or the Goal (end-location at which an object was directed) were manipulated independently in two experiments. In Experiment 1, subjects were required to attend selectively to either the correctness of the grip or the goal of the observed action. Subjects were faster when attending to the goal of the action and a strong interference of goal-violations was observed when subjects attended to the grip of the action. Importantly, observation of irrelevant goal- or grip-related violations interfered with making decisions about the correctness of the relevant dimension only when the relevant dimension was correct. In contrast, in Experiment 2, when subjects attended to an action-irrelevant stimulus dimension (i.e. orientation of the object), no interference of goal- or grip-related violations was found, ruling out the possibility that interference-effects result from perceptual differences between stimuli. These findings suggest that understanding the correctness of an action selectively recruits specialized, but interacting networks, processing the correctness of goal- and grip-specific information during action observation.

Introduction

An important skill underlying our daily interaction with people involves understanding other’s actions. When a speaker at a conference grasps a microphone, you automatically infer that he wants to start his talk. However, you would be quite surprised if the speaker moved the microphone to his ear instead of to his mouth. As the example illustrates, we use semantic knowledge about objects and their associated goals to understand other’s intentions. Semantic knowledge about objects can be described both in terms of knowing how to use an object (i.e. knowledge about the appropriate grip for interaction with the object) and knowing what to do with an (i.e. knowledge about the typical goal-location of the object; Van Elk et al. 2007). Throughout this paper, “grip-related aspects of object use” refers to the hand posture by which an object is grasped, while “goal-related aspects of object use” refers to the spatial end location to which actions with objects are typically directed.

Considerable evidence shows that both aspects (grip-related and goal-related aspects of our knowledge about objects) are mediated by different brain areas. Ideational apraxic patients have difficulty with retrieval of semantic information about objects (i.e. providing a description of the object’s use) but they are still able to grasp objects in a correct fashion. In contrast, ideomotor apraxic patients are characterized by preserved semantic knowledge about objects but show a failure to select correct grasping actions for many objects (Ochipa et al. 1989; Buxbaum 2001). In line with these findings, recent studies suggest that distinct brain areas are involved the preparation and execution of grip-related and goal-related aspects of object use (Majdandzic et al. 2007; van Schie and Bekkering 2007). When subjects prepared an action on the basis of a pre-cued grip (at which part to grasp an object), enhanced activation was found in the parieto-occipital sulcus, supporting visuo-motor transformations for grasping the object. In contrast, planning an action on the basis of its final goal location (end position for transportation) resulted in activation in pre-frontal brain areas, supporting the planning of movements towards more distant behavioral goals. Together, both neuropsychological and neuroimaging studies suggest that frontal brain areas contribute to guiding actions towards final goals, whereas parietal areas are more closely involved in grip-related aspects of object manipulation.

Recent studies from our lab suggest that action goals also play an important role at a semantic level. For example, preparing an action with an object (e.g. bringing a cup towards the mouth) was found to facilitate the processing of words, corresponding to the goal of the prepared movement (Lindemann et al. 2006). In a follow up ERP study, the semantic nature of the effect was confirmed, by showing an N400-priming effect for object-congruent compared to object-incongruent words (Van Elk et al. 2008). The priming effect was only found when subjects prepared a meaningful action with an object (e.g. bringing a cup towards the mouth), but not when subjects prepared a meaningless action with the object (e.g. bringing a cup towards the eye). These findings suggest that the preparation of meaningful actions relies on the activation of goal-relevant semantic knowledge. Finally, in a recent study it was found that subjects are faster when making a semantic categorization response towards an object, when the direction of responding corresponds to the typical goal location of the object (e.g. faster movements towards the body when observing a cup, but faster movements away from the body when observing a hammer; Van Elk et al., submitted). Together, these studies suggest that the planning and execution of object directed actions is organized primarily around specific goal locations associated with the object.

However, to bring an object towards its goal location, the appropriate handgrip has to be selected as well. Rosenbaum was among the first to note that specific hand grips are selected to accomplish a more remote action goal (Rosenbaum et al. 1992). For example, when subjects prepare a grasping movement, they typically select a hand grip that allows them to end the movement in a comfortable position, which is known as the end-state comfort effect. This finding directly supports the notion of a hierarchically organized motor system, in which action goals guide the selection of lower level action features (i.e. hand grip used for grasping).

Still, relatively little is known about the functional organization of the semantic knowledge that enables us to understand behavior of others. Both goal- and grip-related aspects need to be processed in order make sense of the other person’s behavior as well as to understand the (in)correctness of an observed action. If you observe a person grasping a cup at the handle, for instance, you would probably infer that he is grasping to drink, whereas if the person would grasp the cup at the upper part he would probably grasp the cup to put it away. An intriguing question is whether a comparable hierarchical organization of semantic knowledge, as observed in the planning and execution of object directed actions, applies to the observation of actions as well.

Considerable evidence suggests that similar brain areas are recruited during both the execution and the observation of actions (Rizzolatti and Craighero 2004). For example, in nonhuman primates neurons in premotor cortex respond both when the monkey performs an object directed action itself and when the animal observes the experimenter performing the same action (Rizzolatti et al. 1996). This so-called mirror neuron system supposedly plays an important role in the recognition and understanding of observed behavior. Interestingly, the majority of mirror neurons appears to be sensitive to the overall goal of an observed action (broadly congruent mirror neurons), irrespective of the kind of grip used or whether the final action outcome can be perceived (Gallese et al. 1996; Umilta et al. 2001; Ferrari et al. 2005). Thereby, mirror neurons probably support a goal directed interpretation of observed actions. However, to process the goal of an action, the grip applied to an object needs to be taken into account as well, as it can provide valuable information about the actor’s intention (e.g. grasping to eat or grasping to throw).

The present study aimed at identifying the relation between goal- and grip-related aspects of object use in action observation. To this end, we used an action observation setting in which subjects attended either to the correctness of the goal location (“is the object held at the correct goal location?”) or to the correctness of the grip (“is the object grasped with an appropriate grip?”) applied to an object. In a previous experiment it was found that observation of goal-violations (e.g. observing a cup near the ear) resulted in a stronger interference with semantic categorization responses than observation of grip-violations (e.g. observation of a cup with an odd hand grip; Van Elk et al., submitted). The present study corroborated on these findings to investigate whether the hierarchical relation between goals and grips in action execution, applies to judging the correctness of observed actions as well.

Specifically, we aim to investigate differences in speed and accuracy of judging the correctness of action goals and grips. Moreover, we seek to find out if these judgments are performed independent of each other or directly interact. Given the dominant role of action goals in our interactions with objects, we expected that attending to the goal location of an action is relatively easier than attending to the grip of an action, which should become apparent in faster reaction times for trials in which subjects attended to the goal location compared to the grip of the action. Secondly, because both the goal- and grip-related aspects are probably involved in understanding the correctness of an observed action, an interference effect of grip-violations was expected when subjects attended the goal of the action. Vice versa, goal-violations should interfere when attention is directed at the grip of the action. However, given the notion that action goals play a dominant role in the understanding of observed actions, a stronger interference effect may be expected for goal-violations when judging the correctness of grips, compared to grip-violations when judging the correctness of goals.

To investigate the relation between goals and grips in action observation, we used pictures in which the correctness of the goal location and the grip of the action were independently manipulated. Fourteen different objects were chosen that each had a clear goal location associated with the face and a predefined specific manner of grasping (e.g. cup, comb, telephone etc., see Table 1). For each object, four different pictures were taken of an actor using the object in a (1) correct fashion, (2) applying an incorrect grip to the object, (3) directing the object to an incorrect goal, or (4) holding the object with an incorrect grip near an incorrect goal location (Fig. 1). Correctness of the goal or grip applied to the object was defined with respect to the prototypical action associated with the object. Whereas objects typically afford several action possibilities (e.g. a cup can be used to drink or to catch a fly), in daily life we usually associate objects with prototypical grasps and goal locations (the most frequent action associated with a cup is “grasping to move towards the mouth”).

Table 1

List of objects used in the present experiment

Object
Water bottle
Hair brush
Camera
Telephone
Comb
Cup
Flute
Mouthorgan
Bicycle helmet
Magnifying glass
Microphone
Razor
Spoon
Toothbrush

Fig. 1

Example of different stimulus categories used in the experiment representing the independent manipulation of the (in)correctness of the Goal or the Grip applied to an object

Experiment 1

In the first experiment subjects were instructed in separate blocks to decide whether either the goal location or the grip of the observed action was correct or incorrect. More precisely, in “attend-to-goal” blocks subjects had to decide whether the goal location at which the actor held the object, allowed the functional use of the object (e.g. holding a cup near the mouth allows drinking, whereas holding a cup near the eye does not). In “attend-to-grip” blocks, subjects were explicitly instructed to decide whether the hand-grip applied to the object was correct or incorrect with respect to normal object use. Subjects responded by means of a left/right button press to distinguish between correct and incorrect goals and grips. In addition, subjects were instructed to respond as fast and accurately as possible. Block-order (“attend-to-goal” vs. “attend-to-grip”) and mapping of response buttons were counterbalanced between participants.

Before the experiment started, an object recognition task was administered to ensure that the subjects recognized all object stimuli used in the experiment. After careful explanation of the subject’s task, the subjects practiced the task during 14 practice trials. Practice was repeated in the second block. Each block consisted of 112 trials (14 objects × 4 different stimulus categories × left/right handed actor) and stimuli were presented in a random order. Pictures were presented at a resolution of 1,024 × 768 pixels on a 19′ computer monitor, located approximately 60 cm away from the subject. The picture remained on the screen until a button-press was detected and between trials a blank screen was presented for 1 s. For analysis of reaction times, incorrect responses and reaction times exceeding the subject’s mean by more than two standard deviations were excluded from analysis.

Subjects were students from the Radboud University of Nijmegen and received course credits or a small experimental fee (4 €) for participation. In the first experiment 16 right-handed subjects were tested (4 males, mean age 21.6 years).

Results

Reaction times and error rates for both “attend-to-goal” and “attend-to-grip” blocks are represented in Fig. 2. In “attend-to-goal” blocks subjects made categorization errors in 5.4% of all trials and in “attend-to-grip” blocks categorization errors were made in 6.1% of all the trials. Error rates did not differ significantly between action blocks (attend to goal vs. attend to grip), F < 1. The low error rates indicate that subjects managed quite well to evaluate the correctness of both the goal and the grip with respect to normal object use.

Fig. 2

Reaction times (upper graphs) and error rates (lower graphs) for Experiment 1. At the left side, graphs represent “attend-to-goal” blocks, whereas graphs at the right side represent “attend-to-grip” blocks. Bars at the left side represent correctness of the relevant stimulus dimension and bars at the right side represent incorrectness of the relevant stimulus dimension. Light bars represent correctness of the irrelevant stimulus dimension and dark bars represent incorrectness of the irrelevant stimulus dimension

Reaction times of both blocks were analyzed using a 2 (Block: “attend-to-goal” vs. “attend-to-grip”) × 2 (Goal: correct vs. incorrect) × 2 (Grip: correct vs. incorrect) repeated measures ANOVA. A main-effect of Block was found, reflecting faster reaction times when subjects attended to the goal of the action (847 ms) compared to when they attended to the grip of the action (991 ms), F(1,15) = 40.1, P < 0.001. In addition, a main-effect of Goal was found, reflecting faster reaction times to observation of correct (878 ms) compared to incorrect goals (959 ms), F(1,15) = 58.3, P < 0.001. A main-effect of Grip reflects faster responses to correct (906 ms) compared to incorrect grips (931 ms), F(1,15) = 4.8, P < 0.05. Furthermore, an interaction-effect between Goal and Grip was found, reflecting that the interference-effect of incorrect goals differed between observation of correct grips (interference-effect = 142 ms) and incorrect grips (interference-effect = 20 ms), F(1,15) = 61.4, P < 0.001. Finally, an interaction-effect was found between Block and Goal, reflecting a stronger interference-effect of goal-violations when subjects attended to the Grip (difference between correct and incorrect goals = 113 ms) than when they attended to the Goal of the action (difference between correct and incorrect goals = 30 ms), F(1,15) = 10.3, P < 0.01. To further investigate the interference-effects of goal- and grip-violations in “attend-to-goal” and “attend-to-grip” blocks, reaction times were analyzed for separate blocks, using a 2 (Goal: correct vs. incorrect) × 2 (Grip: correct vs. incorrect) repeated measures ANOVA.

Reaction times obtained during blocks in which subjects attended to the goal of the action are represented in the left upper graph in Fig. 2 and were analyzed using a 2 (Goal: correct vs. incorrect) × 2 (Grip: correct vs. incorrect) repeated measures ANOVA. Analysis of “attend-to-goal” blocks revealed a main-effect of Goal, F(1,15) = 12.2, P < 0.005 and a main-effect of Grip F(1,15) = 12.1, P < 0.005. Interestingly, a significant interaction-effect was found between Grip and Goal, F(1,15) = 20.5, P < 0.001. The interaction effect between Goals and Grips was further investigated, using post hoc T tests, with Bonferroni correction for multiple comparisons lowering significance criterion α to 0.013. A significant difference between correct and incorrect grips was found only when the goal of the action was correct, t(15) = ­5.7, P < 0.001 (left side of upper left graph; effect-size grip = 77 ms). When the goal of the action was incorrect, no additional effect of grip-violations was observed, t(15) = 1.5, P = 0.22 (right side of upper left graph).

Analysis of the error rates for “attend-to-goal” blocks revealed significant differences between pictures representing objects at a correct goal with an incorrect grip and all other stimulus categories (P’s < 0.001) indicating, that subjects made most errors (3.3% of all trials) when the Grip of the action was incorrect and the Goal was correct (Fig. 2, lower left graph, second bar).

Reaction times for blocks when subjects attended to the grip of the action are represented in the right upper graph of Fig. 2 and were analyzed using a 2 (Goal: correct vs. incorrect) × 2 (Grip: correct vs. incorrect) repeated measures ANOVA. Analysis of “attend-to-grip” blocks revealed a main effect of Goal, F(1,15) = 55.9, P < 0.001. No main effect of Grip was found, F(1,15) = 1.0, P > 0.33. A significant interaction was found between Goal and Grip, F(1,15) = 35.8, P < 0.001. The interaction-effect between Goal and Grip was further investigated using post hoc T tests, with Bonferroni correction lowering significance criterion α to 0.013. A significant difference between correct and incorrect goals was found, only when the grip of the action was correct, t(15) = ­10.8, P < 0.001 (Fig. 2, upper right graph, 1st and 2nd bar). When the grip of the action was incorrect, no additional interference of goal-violations was observed, t(15) = 1.8, P > 0.1 (Fig. 2, upper right graph, 3rd and 4th bar). Thus, when subjects attended to the grip of an action, a strong interference of goal-violations was observed only when the grip was correct (effect-size of Goal: 188 ms). Investigation of the error rates revealed that subjects made most errors (3.3% of all trials) when only the Goal of the action was incorrect and the Grip was correct (P’s < 0.001; Fig. 2, lower right graph, second bar).

Discussion

In line with our prediction, we found that attending to the grip of an action requires more elaborate processing than attending to the goal location of the action, as reflected in slower reaction times for trials in which the subject judged the correctness of the grip. Importantly, analysis of reaction times revealed a consistent effect of goal- and grip-related violations on making decisions about the correctness of either the goal or the grip of the action. A strong interference with subjects’ decisions about the correctness of the goal location of an action was observed only when the grip applied to the object was incorrect. When the goal location of the action was incorrect, no additional interference of the irrelevant stimulus dimension was observed. Vice versa, when subjects decided on the correctness of the grip of the action, a strong interference was found only when the action was directed at an incorrect goal. When the grip applied to the object was incorrect, no additional interference of irrelevant goal-violations was observed. As expected, goal-violations had a stronger interference-effect on reaction times (effect-size = 188 ms) than grip-violations (effect-size = 77 ms).

Analyses of error rates showed that subjects made most errors only when the irrelevant stimulus dimension was incorrect. That is, when subjects attended to the goal of an action, most errors were made when the grip was incorrect, whereas when subjects attended the grip of the action, most errors were made when the goal location was incorrect, thereby supporting reaction time findings.

The present findings are in line with our prediction that action goals play a dominant role in understanding the correctness of observed actions. In addition, the interference effect of grip-violations suggests that the (in)correctness of the hand grip applied to an object interferes with making decisions about the correctness of the action goal. Thereby, these findings suggest that processing the goal and grip of an observed action are complementary processes that enable us to make sense of what another person is doing.

In the first experiment, the interference-effect of goal- or grip-related violations was shown to be dependent on task instruction (“attend to goal” vs. “attend to grip”). Apparently, the interference-effects of goal- and grip-related violations are dependent on the activation of specific semantic knowledge about objects. However, it might be argued that reaction time effects actually reflect differences between stimulus categories at a perceptual level. Both when subjects attended to the goal and grip of the action a facilitation of responses was found for pictures representing objects being used in a correct fashion (correct goal and correct grip). To rule out the possibility that perceptual differences between different stimulus categories contributed to reaction time effects, a second experiment was conducted, in which subjects categorized pictures on the basis of action-unrelated features. If attention is directed at stimulus features that are unrelated to the action, the interaction-effect between Goal and Grip should disappear (e.g. no facilitation for pictures representing correct object use). If, however, the effects found in Experiment 1 are due to a difference at a perceptual level the effect should also remain in a non-conceptual knowledge based task.

Experiment 2

The same picture stimuli were used as in the first experiment. Subjects were instructed to make a subjective judgment about the orientation of the object in the picture. For example, if they considered the object to be oriented in a more horizontal fashion, they were required to push the left button and to push the right button if they considered the object to be oriented in a more vertical position. Assignment of response-buttons to horizontal or vertical judgments was counterbalanced between participants. Subjects performed one block of 114 trials. Sixteen subjects (1 male, mean age: 21.9 years) participated in the second experiment and received course credits or 4 € for participation.

On average, subjects classified 59.7% of the objects in all pictures as being oriented horizontally and 40.3% as being oriented in a vertical fashion. To classify subjects’ categorization responses as correct or incorrect, two different analyses were conducted.

In a first analysis the consistency of the categorization per picture was assessed, by calculating a consistency ratio. The absolute difference between responses classifying the orientation of the object as horizontal or vertical per picture was divided by the total number of responses, resulting in a value between 0 (no consistency; equal number of horizontal and vertical classifications) and 1 (full consistency). Pictures that had a consistency lower than 0.8 were considered to be ambiguous. In this way, 12 pictures were classified as ambiguous and excluded from reaction time analysis (9.4% of all stimuli). Reaction times were analyzed, using a 2 (Goal: correct vs. incorrect) × 2 (Grip: correct vs. incorrect) repeated measures ANOVA. Analysis of reaction times revealed a main effect of Goal, F(1,15) = 7.2, P < 0.05, reflecting slower responses to pictures in which an object was directed at an incorrect goal compared to a correct goal (effect-size Goal = 24 ms). Importantly and in contrast to the first experiment, no significant interaction between Goal and Grip was observed, F < 1.

Because each individual picture was presented twice to each subject, in a second analysis it was determined whether individual subjects were consistent in their categorization of specific pictures (the same picture should always be classified as being horizontal or vertical). According to this criterion, subjects made inconsistent categorization responses in 6.9% of all the trials. After exclusion of ambiguous responses, analysis of reaction times revealed only a marginally significant main effect of Goal, F(1,15) = 3.5, P = 0.08, indicating a trend towards slower responses to pictures representing objects at incorrect compared to correct goals (effect-size Goal = 26 ms; Fig. 3). Again, no interaction was found between Goal and Grip. Analyses of error rates are represented in the lower part of Fig. 3 and revealed a significant main effect of Grip, F(1,15) = 6.4, P < 0.05, indicating that subjects were more uncertain in categorization of the orientation of the object when the grip applied to the object was incorrect.

Fig. 3

Reaction times (upper graph) and error rates (lower graph) for Experiment 2 in which subjects attended to the orientation of the object. Bars at the left side represent stimuli in which the goal-location was correct and bars at the right side represent incorrect action goals. Light bars represent stimuli in which the grip was correct, whereas dark bars represent stimuli in which the grip was incorrect

In the second experiment subjects observed the same pictures as in Experiment 1, in which the correctness of the goal location and the grip applied to an object were independently manipulated. Rather than attending to action-related features (categorization of the correctness of either the goal or the grip of the action), in the second experiment subjects were required to categorize the object as being oriented in a more horizontal or vertical fashion. Whereas in the first experiment a highly significant interaction between Goal and Grip was found, no interaction-effect was found in the second experiment. Only a trend was observed for slower responses to pictures in which the object was presented at an incorrect goal location. Thus, reaction time data differ in important respects from the interference effects of goal- and grip-related violations that were observed in Experiment 1. Thereby, findings from Experiment 2 rule out the possibility that interference effects of goal- or grip-violations could be explained on the basis of perceptual differences between stimulus categories alone. Only when subjects focused on action-related aspects (correctness of goal or grip), observation of goal- or grip-related violations interfered with the categorization response.

General discussion

The present study aimed at the question how people understand the (in)correctness of an observed action and how semantic knowledge about goal- and grip-related aspects of object use is involved. On the basis of the dominance of action goals in the planning of our own actions, we expected that action goals should play an important role in the understanding of observed actions as well. To separate the relative contribution of goal- and grip-related features of action understanding, in the first experiment we asked subjects to selectively attend to the correctness of either the goal or the grip of an observed action. At least two findings support our hypothesis that action goals play a dominant role in action understanding.

First, reaction times were significantly slower when subjects attended to the grip of the action, compared to when they attended to the goal of the action. Apparently, goal-related information is more easily activated, which might be due to the fact that it can be defined more directly in spatial terms (i.e. end location). Secondly, it was found that goal-violations caused a stronger interference with categorization responses than grip-violations. Both the faster reaction times when subjects attended to the goal compared to the grip of the action and the stronger interference-effects of goal-violations than grip-violations are in line with our prediction that action semantic knowledge used in action understanding is organized primarily around goals.

Importantly, the distinction between goal- and grip-related aspects of actions can be described at different levels within the hierarchy of action representation (Grafton and Hamilton 2007). Thus, at a semantic level, the present study suggests that goal-related information is more easily activated than grip-related information. At a perceptual level it could well be that the advantage of processing goals relative to grips is reflected in an advantage of global over local visual processing (Antes and Mann 1984). More precisely, the functional mechanism by which subjects attend either to goal- or grip-related aspects of an observed action might be described in terms of attending to global or local features of the picture.

In an additional analysis we investigated whether global-over-local precedence might be a valid way to characterize the advantage of goals over grips at a perceptual level. To this end, we split up the stimuli in two different categories, according to whether the object was grasped with a full grip (whole-hand grip) or with a precision grip (fine grip between index finger and thumb). If attending to the goal of an action relies on global visual processing, it might be relatively easier to process full grips compared to precision grips. Interestingly, when subjects attended to the goal of the action, an advantage of processing full grips compared to small grips was found, as reflected in faster reaction times and lower error rates for observation of full grips (Supplementary Results and Supplementary Fig. 1 in Electronic supplementary material). In contrast, when subjects attended to the grip of the action, no advantage of full grips over precision grips was found. These findings suggest that processing goal-related semantic information is reflected at a perceptual level in global visual processing, whereas processing grip-related information increases sensitivity to local visual features. Thereby, the present findings support the view that the supposed hierarchy underlying the observation of actions can be described at multiple levels of complexity (Grafton and Hamilton 2007).

However, it is important to note that the global-over-local precedence at a perceptual level cannot account for the interaction effects between goals and grips found at the semantic level in the present study. In visual attention paradigms, interference effects of the irrelevant stimulus dimension have only been reported for global aspects (e.g. Navon, 1977). Irrelevant local information typically does not interfere with making global judgments. In contrast, in the present study, interference effects were found both for task-irrelevant goal- and grip-related violations, suggesting that interference effects cannot be explained at a purely perceptual level.

In the present study, correctness of the goal or grip of an observed action was independently manipulated. However, in action execution the correctness of the grip is often specified in relation to the end goal (e.g. grasping a knife at the blade is clearly incorrect in case one wants to butter bread, but not if one wants to put the knife in the dishwasher). Accordingly, faster reaction times for evaluation of goals compared to grips might actually reflect that evaluation of grips involves an evaluation of the correctness of the action goal as well. If evaluation of grips would require evaluation of goal information as well, a strong interference should be expected for incorrect goals, independently of the correctness of the grip. Instead, goal- or grip-related violations caused an interference-effect only when the task-relevant stimulus dimension was correct. For example, when subjects attended to the grip of the action, an interference-effect of goal-violations was found only when the grip of the action was correct. When the relevant stimulus dimension was incorrect, no additional interference effect of the irrelevant dimension was found. This suggests that action knowledge about the correct use of an object is only retrieved when the attended action feature (e.g. grip) is consistent with the stored action knowledge about an object (e.g. precision grip). In case the attended action property is inconsistent or incorrect with stored action knowledge (e.g. using a spoon with a full grip), additional retrieval of the irrelevant action feature is inhibited or obstructed.

The selective interference of irrelevant stimulus properties found in the present study is in line with findings from a recent study by Bach et al. (2005) in which subjects were required to selectively attend to either the spatial or the functional match between a tool and a target object. It was found that irrelevant spatial mismatches (e.g. object with wrong orientation) only affected the judgment of functional matches (e.g. key near a doorlock) but no interference was found for categorization of functional mismatches (e.g. screwdriver near a doorlock; Bach et al. 2005; Experiment 2A and 2B). Based on these findings Bach et al. argue that irrelevant spatial information can either facilitate or inhibit an action representation that is already activated on the basis of functional information. In case of a functional mismatch no action representation is available and therefore no additional interference effect is observed for the irrelevant spatial mismatch.

This interpretation fits nicely with data from the present study, suggesting that subjects rely on the activation of semantic representations when required to attend to goals or grips. For example, when subjects judge the correctness of the goal and the observed goal is correct, the activated semantic representation can either be facilitated or inhibited by the concomitant observation of grip-related information. In contrast, when the goal of the observed action is incorrect, no semantic representation is available and therefore no additional interference by the irrelevant stimulus dimension is observed. Interestingly, present findings show interference effects of the irrelevant stimulus dimension both for categorizing the correctness of goals and grips. Apparently, semantic representations are activated when attention is directed at either goal- or grip-related aspects of object use, indicating that both goals and grips are part of our semantic knowledge about objects. Whereas Bach et al. (2005) assessed the spatial and functional relation between two objects, the present study further extends these findings by showing how knowledge about both goals and grips is involved in understanding the correctness of an observed action.

Further support for the selective processing of task-relevant features was obtained in the second experiment, in which subjects attended to an action-unrelated stimulus feature (i.e. orientation of object). No reaction time interference effects of goal- or grip-violations were observed for categorization of object orientation, suggesting that semantic knowledge about objects is selectively activated, only when subjects had to think about action-related features. More precisely, when subjects focused on the correctness of the goal or the grip of the observed action, their responses were facilitated by observation of an object being used in a correct fashion and inhibited by the observation of incorrect object use. When subjects categorized the orientation of the object, no action semantic knowledge needed to be activated, as reflected in the absence of an interaction between goals and grips. Thereby, we could rule out the alternative explanation that reaction time effects could be explained by perceptual differences between stimuli.

Based on the neuropsychological dissociation between knowing what to do with an object (goal) and knowing how to grasp an object (Ochipa et al. 1989) and on previous findings suggesting that immediate and distal action goals are mediated by distinct neural systems (Majdandzic et al. 2007; van Schie and Bekkering 2007) it is suggested that distinct but complementary brain areas support goal- and grip-related aspects of action planning. In the present study we investigated whether a similar functional relation between goal- and grip-related features applies to the observation of object directed actions as well. Our data suggest that whereas attention can be selectively directed to either the goal or the grip of an observed action, both goal and grip-related features are to some extent automatically processed. Recent fMRI data suggest that observation of unusual goals is associated with activation in the inferior frontal gyrus, which is considered part of the human mirror system (de Lange et al. 2008; however for different view, see: Turella et al. 2007). In contrast, observation of unusual grips resulted in higher activation in the lateral occipito-temporal complex around the extrastriate body area, probably reflecting the processing of body postures in an action context. Thereby, these data suggest specialized networks for processing goal- or grip-related aspects of observed actions.

Taken together, the present findings suggest that our knowledge about the use of objects is organized primarily around typically associated goal locations. In addition, our data support a close interaction between the processes supporting goal- and grip-related aspects of an action. Although the neural mechanisms supporting the selective processing of goal- or grip-related aspects call for further investigation, the present study suggests at a behavioral level a possible hierarchical organization of the semantic knowledge used in understanding the correctness of others’ actions.

Below is the link to the electronic supplementary material.

Supplementary Results (DOC 26 kb)

Reaction times (upper graph) and error rates (lower graph) for additional analysis of Experiment 1 in which subjects attended to the goal or the grip of the action. At the left side, graphs represent 'attend-to-goal' blocks, whereas graphs at the right side represent 'attend-to-grip' blocks. Light bars represent objects grasped with a precision grip and dark bars represent objects grasped with a full grip. (EPS 1015 kb)