The role of scene type and priming in the processing and selection of a spatial frame of reference.

The selection and processing of a spatial frame of reference (FOR) in interpreting verbal scene descriptions is of great interest to psycholinguistics. In this study, we focus on the choice between the relative and the intrinsic FOR, addressing two questions: (a) does the presence or absence of a background in the scene influence the selection of a FOR, and (b) what is the effect of a previously selected FOR on the subsequent processing of a different FOR. Our results show that if a scene includes a realistic background, this will make the selection of the relative FOR more likely. We attribute this effect to the facilitation of mental simulation, which enhances the relation between the viewer and the objects. With respect to the response accuracy, we found both a higher (with the same FOR) and a lower accuracy (with a different FOR), while for the response latencies, we only found a delay effect with a different FOR.

Introduction

Expressing spatial relations is an important aspect of everyday communication. By using spatial terms, we indicate the location of one object in relation to another, to ourselves, to an interlocutor or to cardinal points. These different ways of expressing a spatial relation depend on the choice of frame of reference (FOR). A FOR can generally be described as a set of axes that defines space (Carlson, 1999). The point of intersection constitutes the origin (Miller and Johnson-Laird, 1976). The relative FOR establishes a ternary relationship which comprises a reference object, a located object, and a viewpoint. Using the intrinsic FOR, however, leads to a viewpoint-independent binary relationship between a reference object and located object (Levinson, 1996, 2003). In the present study, the origin of the relative FOR coincides with the egocentric perspective of the viewer whereas the origin of the intrinsic FOR is object-centered. The absolute FOR depends on environmental features such as cardinal points and will not be considered in the present study.

Crucially, spatial projective terms such as “next to,” “in front of” and “behind” (“neben,” “vor” and “hinter” in German) are ambiguous if it is unclear which FOR is adopted. Different FORs appear to be used differently in everyday life. There have been many attempts to identify preferences for specific FORs leading to ambiguous results. The relative FOR, being perceptually available and avoiding the extra computational effort needed for mental rotation, has been considered predominant by some authors (Linde and Labov, 1975; Levelt, 1982, 1989) whereas other authors have claimed that the intrinsic FOR predominates (Miller and Johnson-Laird, 1976) or is at least preferred (Carlson-Radvansky and Irwin, 1993; Carlson-Radvansky and Radvansky, 1996; Taylor et al., 1999). How FORs are chosen and maintained has been studied intensively (e.g., Carlson-Radvansky and Irwin, 1994; Carlson, 1999; Watson et al., 2004; Ball et al., 2009). Research has shown that, when choosing a FOR, all FORs are initially active (Carlson-Radvansky and Irwin, 1994) until one is selected. This selection is affected by various situational factors, for instance by functional relations between the objects (Carlson-Radvansky and Radvansky, 1996), motion characteristics (Levelt, 1984), gravity (Friederici and Levelt, 1990), or by alignment to the FOR chosen by the interlocutor in dialogue (Watson et al., 2004). These results indicate that there may not be a uniform default FOR but rather that the FOR selection is affected by situational influences.

A question that has, to our knowledge, not been addressed yet is whether the type of scene used to present the stimuli has a direct effect on the acceptability and processing of different FORs. This question arises from considerations of the disparities between FORs and of the role of embodiment.

A principle difference between the relative and the intrinsic FOR is that only the former requires the viewer as an origin. The relative FOR is indispensable for our navigation in the world as its use involves computation of relevant spatial relations. As the relative FOR originates in the viewer, an embodiment of the viewer may be considered a necessary prerequisite. Using the relative FOR for depictions therefore requires a mental simulation of a positioned viewer in the scene. Mental simulation in the processing of spatial relations irrespective of FOR has been reported elsewhere (e.g., Coventry et al., 2010).

We assume that there is little incentive for such a mental simulation of a viewer in depictions that exclusively involve configurations of objects and do not contain natural elements and that the relative FOR may thus be less preferred than the intrinsic FOR. However, if object configurations are embedded in depictions of natural environments, the use of the relative FOR may become more likely, as it is easier for a viewer to imagine being in a natural environment than in a scene that only contains “floating” objects.

Recent studies varied in their construction of scenes. Studies that have found a preference for the intrinsic compared to the relative FOR vary from using only a depiction of two or more objects without background elements (e.g., Carlson-Radvansky and Radvansky, 1996; Taylor and Rapp, 2004) to line-drawing scenes with rudimentary background elements (Carlson-Radvansky and Irwin, 1993). However, Taylor and Tversky (1996) showed that speakers chose different frames of reference for their descriptions of spatial environments depending on the characteristics of the scene they were shown.

We assume that presenting a realistic scene might result in a processing advantage for the relative FOR, as viewers are more likely to perform a mental simulation and establish a relation between the objects and themselves. Therefore, we hypothesize a higher acceptability of the relative FOR in more naturalistic scenes.

We investigated this hypothesis by presenting identical object configurations in two different scenes, and measuring the acceptability and reaction times (RTs) in a sentence-picture verification task. Sentences in German such as “Die Pflanze ist vor dem Stuhl” (“The plant is in front of the chair”) were used to assign a reference frame to the picture. In order to present a realistic scene, we chose a living-room scenario so that, in one version, the depiction showed a room with two embedded objects, whereas in the other version the same two objects were depicted in front of a white background.

In addition to the influence of scene type on FOR selection we were also interested in FOR-related priming effects. Recent studies have shown that the time needed for spatial term assignment in a FOR is prolonged when a different FOR has previously been processed (Carlson-Radvansky and Jiang, 1998; Carlson and van Deman, 2008). This effect has been interpreted as inhibition of the non-selected FOR (Carlson and van Deman, 2008). However, this investigation of priming effects focused on RTs, and did not include an analysis of response accuracy. If RTs were prolonged due to inhibition, we hypothesize that the accuracy ratings could also be affected. Inhibition of the non-selected FOR should lead to more rejections of targets following a prime trial with a different FOR than with a neutral or identical FOR. More rejections are expected because the inhibited FOR may not only be more difficult to process, which is revealed by longer response latencies, but also, in cases of stronger inhibition, be less available.

Even though priming effects in FOR selection have not been reported for RTs, other studies have suggested their possible existence. Watson et al. (2004) reported that interlocutors in dialog tended to use the same FOR as their interlocutor had previously used. This alignment effect was argued to result from priming of FORs. Thus, it is plausible to assume that not only can a different FOR delay responses, but that the same FOR can also speed up responses due to FOR priming, and that these effects might be observable in both RTs and accuracy ratings.

In order to assess these different possible effects of same and different FORs on accuracy and RTs, we included three conditions in the experiment: a match condition in which prime and target used the same FOR, a mismatch condition, which required switching the FOR between prime and target and a control condition, in which the prime trial did not disambiguate between FORs. Thus, the FOR on the target trial could either have been activated by having been used in the preceding trial (match condition) or inhibited by being available but not being selected (mismatch condition). The control condition served as a baseline for comparisons.

Our expectations were that having the same FOR would result in more acceptances of target trials in the match condition than in the control condition, while having different FORs would result in more rejected target trials in the mismatch condition than in the control condition. With regard to RTs, different FORs are predicted to lead to longer response latencies in mismatch target trials than in the control target trials, as described in earlier studies (Carlson-Radvansky and Jiang, 1998; Carlson and van Deman, 2008), while having the same FOR was predicted to yield shorter response latencies for target trials in the match condition compared to the control condition.

Materials and methods

Participants

Fifty students of Bielefeld University (21 men, 29 women) ranging in age from 20 to 62 (M = 26.9, SD = 8.5) were paid for their participation in the experiment. All participants were native German-speakers and each of them saw only one version (either the one with or the one without background).

Stimuli and design

The experiment comprised 400 trials, consisting of 96 prime and 96 target trials as well as 200 distractor trials and 4 prime-target pairs with definite “no” answers according to both FORs. Sentence-picture verification for the prime trials were correct for the neutral configurations and for the relative and intrinsic FOR in 32 cases each. With regard to the target trials, 48 cases were correct for the relative and 48 for the intrinsic FOR. We thus made sure that 50% of the trials had as correct response “yes” and the other 50% had “no” as a correct response. The distractor trials consisted of 100 “yes” and 100 “no” response trials.

Stimuli consisted of a sentence and a picture presented subsequently. The sentence was presented auditorily (in German) and spatially described the object configuration in the picture. Sentence duration was approximately 2 s and during its presentation, participants saw a white screen. Immediately after the presentation of the stimulus sentence, the picture was shown.

Pictures of object configurations were created using indoor planning software (“Sweet Home 3D”) in two versions which differed in background. In one version, following Henderson and Hollingworth's (1999) idea, we used a semantically coherent, human-scaled view of a living-room (Figure 1). This version constituted a true scene (Henderson and Ferreira, 2004), which will be referred to here as “scene with background.” In the other version, the object configurations were shown in front of a white background (Figure 2). This version (ersatz scene, Henderson and Ferreira, 2004) will be referred to here as “scene without background.” Participants saw either the version with or the version without background, therefore there were equal number of scenes with and without background. The size of the scene was 33 × 17 cm and the unconstrained viewing distance was approximately 70 cm.

Figure 1

Scene with background and both FOR.

Figure 2

Examples for prime-target-pairs in each condition.

Three types of pictures were created: experimental, neutral, and distractor. The experimental pictures consisted of two discrete objects in the foreground (reference object and located object). We used three different triaxial reference objects (chair, armchair, sofa), which were rotated on the vertical axis at angles of 0, 90, and 270° in order to vary the mapping of the horizontal intrinsic axes to the horizontal relative axes. Reference objects in the prime and target pictures had the same orientation and were always in an upright position. The located objects were biaxial (plant, stool) thus revealing no predefined horizontal orientation and were placed along the horizontal axes of the reference object (in front of, behind, to the left/right of). For the 0° rotation, the located object was only positioned to the left/right of in order to dissociate the relative and the intrinsic FOR. In order to keep the number of trials within a reasonable limit, we did not present every object in every possible combination of rotation, located object, and reference frame in all three conditions as this would have led to 180 prime target pairs. Thus, we reduced the number of target pictures to 16 which were presented with both FOR in all three conditions resulting in 96 prime target pairs. The 32 target trials (16 with a relative FOR and 16 with an intrinsic FOR) in each experimental condition consisted of 8 trials with 0° rotation, 12 trials with 90° rotation, and 12 trials with 270° rotation. The position of the located object was controlled for the axis between prime and target trials: the located object was positioned on the same axis in 16 prime and target trials and across axis in the other 16 trials per condition. The reference object and located object were positioned at the same, short distance from each other throughout the picture sequence.

In neutral pictures, the located object was placed along the vertical axis of the reference object leading to an alignment of the FORs. This alignment eliminated the need for the viewer to choose between the intrinsic and the relative FOR. For these configurations, eight additional objects were introduced (bench, box, chest, bottle, lamp, notebook, fish tank, carpet). Fifty distractor pictures were created that contained only a single discrete object in the foreground (pieces of furniture, toys, a book, a bottle, etc.).

The presentation of the visual stimuli was preceded by an auditorily presented sentence (in German) describing the object configuration and implicitly assigning the intrinsic or the relative FOR (or both in the control condition). The sentence “The is the ” was played over loudspeakers. See Figure 1 for an example picture of a scene with background with intrinsic and relative FOR. In the distractor trials the sentence “The