Spared ability to perceive direction of locomotor heading and scene-relative object movement despite inability to perceive relative motion.

BACKGROUND: All contemporary models of perception of locomotor heading from optic flow (the characteristic patterns of retinal motion that result from self-movement) begin with relative motion. Therefore it would be expected that an impairment on perception of relative motion should impact on the ability to judge heading and other 3D motion tasks.
MATERIAL AND METHODS: We report two patients with occipital lobe lesions whom we tested on a battery of motion tasks. Patients were impaired on all tests that involved relative motion in plane (motion discontinuity, form from differences in motion direction or speed). Despite this they retained the ability to judge their direction of heading relative to a target. A potential confound is that observers can derive information about heading from scale changes bypassing the need to use optic flow. Therefore we ran further experiments in which we isolated optic flow and scale change.
RESULTS: Patients' performance was in normal ranges on both tests. The finding that ability to perceive heading can be retained despite an impairment on ability to judge relative motion questions the assumption that heading perception proceeds from initial processing of relative motion. Furthermore, on a collision detection task, SS and SR's performance was significantly better for simulated forward movement of the observer in the 3D scene, than for the static observer. This suggests that in spite of severe deficits on relative motion in the frontoparlel (xy) plane, information from self-motion helped identification objects moving along an intercept 3D relative motion trajectory.
CONCLUSIONS: This result suggests a potential use of a flow parsing strategy to detect in a 3D world the trajectory of moving objects when the observer is moving forward. These results have implications for developing rehabilitation strategies for deficits in visually guided navigation.

Background

Everyday living requires visual information about movement of the observer relative to the environment, for the maintenance of posture (e.g. [1]), the identification of objects moving in the scene (e.g. [2]), and as argued by some, the visual guidance of walking (e.g. [3], though see [4]). Visual information is also necessary for detection of “behaviourally urgent” (e.g. [5]) parts of the scene that are approaching the observer, those that are on a collision course with the observer.

We [6-9] and others [10-18], have shown that patients with posterior cortical lesions are impaired on aspects of low-level motion processing. Does it follow that they will consequently show deficits on the “higher level” perception of scene-relative movement and the detection of objects’ trajectories on a collision course with the observer?

A traditional hierarchical model of visual motion processing [19,20] might suggest so. However, it has been shown that the higher level motion processing (e.g. perception of wide field motion, recognition of 3D structure from motion or of biological motion) may be preserved despite significant deficits in lower level motion processing [9,21-25].

Here we report two patients, SS and SR, both with damage to the occipital lobe. We describe their performance on a battery of low-level motion processing tasks and on a number of 3D movement (self-movement and collision detection) tasks. We find that they retain the ability to perceive observer-scene movement despite deficits on low-level motion processing.

Material and Methods

Subjects

Patients

Patient SS, a 29 year old right handed, college educated woman suffered a left Posterior Cerebral Artery infarction involving the occipital lobe (Figure 1 SS-A,C). Patient SR, a 41 year old right handed engineer had a right Posterior Cerebral Artery infarction involving the posterior portion of the occipital lobe (Figure 1 SR-A,C). On neurological examination both were found to be alert, oriented, cooperative, without dysarthria (problems with speech articulation) or paraphasias (jumbling sentences or words), and without any motor deficits. Short term and long term memory were normal. Eye movements (saccades and smooth pursuits) and fixation were normal.

Figure 1

Lesion localization: Panels (SS-A and SR-A) structural MRI showing the lesion location in the occipital lobe. Panels (SS-B and SR-B) Goldman perimetry overlaid on results of Quadrant Vision test ([26]) showing the extent of the scotoma. Panels (SS-C, D and SR-C, D) illustrates the retinotopic mapping by fMRI (in the Siemens 3T scanner) and the lesion (in brown) visualized on the flattened cortex of the occipital lobe. Retinotopic areas in both the intact and damage brain hemisphere are shown for comparison; SS has a dorsal lesion (SS-A), involving the retinotopic areas V2, V3, V3a and slightly touching on area V1. SR shows a dorsal lesion (SR-D, involving the occipital lobe areas some of V1, and more significantly the areas V2 and V3 (SR-C, D).

Both patients had stable homonymous inferior quadranatopia. Goldmann (patient SS) or Humphrey (patient SR) perimetry were repeated twice 6 months apart (here we show the left eye). At the time of psychophysical testing of motion perception visual fields were also mapped in our laboratory with the Quadrant Vision test [26] used for mapping central visual field loss. In the Quadrant Vision test we measured detection of targets within the central 20° of the visual field. The targets were briefly flashed white disks of various dimensions (ranging from 0.75–1.5 deg) embedded in a background of white random dots (4 arcmin in diameter) that were either static, flickering or coherently translating (3 deg/sec), left or right. The Quadrant Vision test was repeated every time the patient visited for assessment of visual motion perception. Because responses were consistent over conditions and sessions the data is pooled together. In Figure 1, SS-B and SR-B, the probability of detection obtained with the Quadrant Vision test ranges from chance (0.5) to perfect (1.0) detection and is indicated by the colour bar. The results of the Goldman perimetry indicated by the bold contour lines (which refers to the areas of normal vision) are overlaid on the Quadrant Vision data.

Both patients had normal acuity with corrective lenses that they wore during the testing sessions and during the fMRI retinotopy mapping for retinotopic lesion localization. SS had a visual acuity of 16/20 in both eyes. SR had a visual acuity of 20/25 in both eyes. SS and SR’s static contrast sensitivity, assessed with the Pelli-Robson chart [27] was normal for each eye. Color vision tested with the Farnsworth-Munsell-100 hues test [28] and with the Standard Pseudoiscochromatic plates [29] was normal. Stereovision tested with the Randot clinical test [30] was impaired; they could detect that “something” was there but could not make out the shape. Performance on Efron shapes [31] which is a neuropsychological test for assessing the ability to perform shape discrimination was normal, as was performance on the VOSP neuropsychological battery [32] which comprises eight visual object and spatial perception tests (four of each). Both patients had high level, demanding jobs, which they resumed after discharge from the acute care hospital and they reported no difficulty with responding to the job demands.

Healthy control subjects

Demographically matched healthy controls were also given the same tests as the patients in order to compare and discuss SS’s and SR’s performance with the performance of the healthy controls. The number of healthy controls varied from test to test, they are listed in the data plots referring to each psychophysical test. Healthy control subjects varied in age between 25 and 41 years. None had any history of neurological or psychiatric disease.

Informed consent was obtained from the patients and the healthy control subjects according to the requirements of Boston University Human Subjects’ Committee for the psychophysics and neuropsychological testing and according to the requirements of the Athinoula’s Martinos Centre for Biomedical Imaging for the imaging studies.

Behavioral tasks

2D Motion perception

Performance of SS and SR was compared to healthy control subjects on a battery of 2D motion processing tasks that tested both local and global motion processing. These tests were administered to SS and SR several times.

In all the tests, shown schematically in Figure 2 the stimulus diameter was 10 deg., dots were white (79.55 cd/m2), 6arcmin in diameter, moving at 3deg/sec and shown against a gray (10.22 cd/m2) background. In all tests, except 2b&c the stimuli were sparse random dot cinematograms (RDK) with dot density of 2 dots/°. In 2b&c the stimuli were dense random dot displays, 50% white and 50% black dots. Here the shapes were defined by a cluster of contiguous random pattern dots, of the same statistical properties as the background but moving in a different direction (panel B) or speed (panel C) than the background. Test difficulty was manipulated by varying the direction (B) or speed (C) differences between the motions of the cluster of dots and that of the background. In the other tests, test difficulty was manipulated by varying the percent coherence of the dots. The coherence of moving dots was varied (tests schematically shown in E,F,G), each dot within the motion sequence was randomly assigned as “signal” or “noise” on a frame-to frame basis such that the signal dots moved in a consistent direction (up or down) and the noise dots were randomly repositioned within the stimulus aperture [21,33].

Figure 2

Schematic view of the battery of 2D motion tests administrated and the zscores of SS and SR compared to the performance of the healthy controls: (A) Speed Discrimination; (B) Form from motion defined by direction differences (C). Form from motion defined by speed differences (D). Direction discrimination; (E). Motion Coherence – translation direction; (F). Motion Coherence: Discontinuity and (G). Motion Coherence: 2D Form from motion.

The 2D Form-from-Motion tests portrayed in panels b and c, were four alternative forced choice tasks with the choices displayed below the stimulus, all the other tests were two alternative forced choice tasks. In all tasks the stimuli were displayed by an adaptive staircase method (Vaina, Gryzwac [34]), and threshold was calculated as the mean and standard deviation of the last 6 reversals. For each patient a z-score was calculated relative to the performance of the healthy controls. Z-scores for both left and right visual field stimulus presentation are shown for each test in Figure 2 in the panel that illustrates schematically each of the 2D motion tests. A z-score of 2 or more was considered impaired performance.

Perception of radial motion

Forward movement of an observer results in a characteristic radial pattern of motion at the eye. Here we examine the patients’ ability to detect/discriminate radial motion by manipulating the coherence level (amount of signal to noise). The stimulus had the same characteristics as the Motion Coherence-Translation, the only difference was that the signal dots had radial direction of motion (expansion or contraction), instead of planar (left or right). Like in the tests described in Figure 2 above, here as well the stimuli were displayed by an adaptive staircase procedure, and the threshold was computed as the mean and SD od the last six reversals.

Heading perception

The previous test demonstrated that patients can detect/discriminate the radial patterns of motion that are associated with the forward movement of an observer. Here we examined how well the patients can pick up characteristics of radial flow fields and determine their direction of (simulated) self-movement by correctly detecting the focus of expansion (FOE). This and the following tests were administered to the patients at the time when their perception of radial motion was normal.

The stimulus consisted of a dynamic random dot field displayed in an aperture subtending 20×30 deg. The motion of field of dots simulated what the observer would see if approaching two transparent planes of dots at distance of 400 and 1000 cm. The simulated motion of the observer was pure translation with a simulated speed of 200 cm/s and the direction varied between the extreme values of 10° to the left and right of the center of the display. Subjects were instructed to fixate on a mark placed 2° off the left or right border of the display at the horizontal midline. Observers watched the motion sequence for 800 ms. At the end of the motion a new static random dot pattern, with the same spatial statistics, was displayed together with a vertical target line (8.96° long) that intersected the horizontal midline of the display. Observers had to judge whether they had been heading to the left or the right of the vertical line. The angle between the heading and the target line was varied according to a staircase procedure. The threshold angle for accurate heading judgement was calculated as the average of the last 6 staircase reversal values.

Perception of approach without optic flow

Schrater et al. [35] showed that normal observers can perceive their direction of (simulated) self-movement during approach on the basis of the scale (spatial frequency) change that results from a decrease in the distance to the objects ahead. Using the stimuli briefly described below, we employed an experimental paradigm to investigate sensitivity to scale change and optic flow (detailed description of the stimuli in [36].

The stimuli subtending 20.7×20.7deg simulated the approach to a continuous textured plane. The image had a mean luminance of 16.77cd/m2. In one condition observers could only use optic flow information (see Figure 5, panel A), in the other condition observers could only use scale change information (see Figure 5, panel C). In a two interval forced choice task (each lasting 960 msec), subjects discriminated between an interval containing their simulated forward motion defined by either optic flow or scale changes, or an interval containing neither.

Figure 5

In panel (A) we show schematically the optic-flow only stimulus created by applying a constant bandpass filter to every frame of a looming stimulus, thus removing any changes in spatial frequency. In panel (C) we show schematically the scale change-only stimulus created by using the spatial frequency spectrum of each looming frame as a bandpass filter for a new random white-noise image (thus removing any temporal correlations and pattern motion from the stimulus). These stimuli are described in detail in [36]. Panel (B) shows the results for SS and SR along with controls for the optic flow only stimuli. Panel (D) shows the results for the scale-change only stimuli. The shaded area refers to the trend of results of the healthy control subjects. This stimulus was presented with central fixation only.

Detection of colliding object trajectory

The display subtended 23×23° and consisted of a ground plane portraying a park with trees and blue sky with white clouds (see Figure 6). Superimposed on the background were 5 gray spheres (mean diameter 2 deg) that moved in apparent depth relative to the observer at an average of 25 cm/sec. In each 1 second motion sequence, the trajectory of on randomly selected “target” was specified to intercept the observer’s current location (or future location in the case of the moving observer) and hit him/her at the vertical midline three seconds from the start of the motion sequence. Using a constant stimuli paradigm, the spheres moved with trajectories of 0° (the target) or pseudorandomly picked from ±10°, ±15°, ±20 of visual angle measured from the vertical midline of the observer. In all trials the spheres trajectory did not cross from one half of the display to the other. The origin in depth of the moving spheres was at 108 cm.

Figure 6

Collision detection task. (A) Five gray spheres move forward. The spheres are numbered 1–5. One of them is the target which is on a collision course with the observer’s vertical midline. At the end of the motion sequence (1 sec) the spheres disappear As soon as possible and as reliable as possible after the end of the motion, in a 5AFC task observers determine which (numbered) sphere was on a collision course (the target). (B) Data on the two test conditions are shown, observer static, and observer in simulated forward motion. Performance of healthy observers, group mean and SE, is indicated in the left panel and the then the performance of the patients SS and SR is illustrated in the two panels to the right.

Observers were asked to identify which of the 5 spheres was the target (a 5AFC task), that is which sphere would collide with him/her at the vertical midline (had 0° trajectory). Results are shown in Figure 6B. There were two test conditions. The viewpoint in the scene was either static (“static observer” condition), or the viewpoint was moving forward (“moving observer” condition). Simulated forward movement was produced by moving the ground plane towards the observer at the same average speed as the spheres (25 cm/sec).

Results

2D Motion perception

The z-scores reported in Figure 2, indicate that SS and SR were significantly impaired on all three form-from-motion tests (panels B, C and G) and on the motion discontinuity test (panel F). Common to these tests was that the stimuli contained edges defined by motion differences and to extract the edges, subjects had to process relative motion in the xy plane. SS and SR showed normal performance on the speed discrimination (panel A) and motion coherence test (panel E), and on the direction discrimination task within the intact hemifield (panel D). The patients’ performance on the motion battery tests did not change over time.

This is an interesting pattern of results. Although SS and SR were impaired on tests where performing the task they had to extract edges or discontinuities from motion, on an additional test (not shown) where the edges were defined by luminance difference between a central cluster of random dots and the background, patients’ performance was normal, suggesting that their difficulties were selective to relative motion in the x,y plane. Their deficit on direction of motion could be accounted for by the fact that task instructions required observers to determine whether the direction of the dots in the display was to the left or right from the imaginary vertical. As such this task was also one of relative motion in the x,y plane. SS and SR’s performance was normal on speed discrimination and on the motion coherence task. In the next set of experiments we further address global motion processing and perception of motion in depth.

Perception of radial motion

The radial motion test (schematically shown in Figure 3A) was repeated on separate visits, two weeks apart (three times with SR and twice with SS) from the very earliest visit, we were able to identify a rapid improvement before performance stabilized – and when the other tests were administered (described in Figures 2 above and Figures 4–6 below). This improvement is captured in Figure 3B. It can be seen that initially SR was outside the normal range for stimuli presented in either visual field, but he rapidly improved and achieved normal performance. SS also showed an improvement in performance but from a lower baseline. After performance improvement, in both patients and for all conditions, the z-score <2 indicate that they achieved normal performance in both visual fields.

Figure 3

Radial Motion test (A). In a single interval task, observers were required to discriminate the direction of radial (expansion vs. contraction) motion formed by a proportion of coherently moving “signal” dots. All stimulus parameters and the definition of signal and noise dots were identical to those in the Motion Coherence test described above. The stimulus was displayed for 500 msec. (B) The data points represent threshold (±SD) for radial motion for the healthy controls and for patients SS and SR on two or three visits 2 weeks apart.

Figure 4

Heading task: (A) Schematic illustration of the stimulus. It consisted of a random-dot-kinematogarm (RDK) displayed in a square aperture subtending 20×30 deg. The motion of the field of dots simulated a straight translation trajectory of the observer at 200 cm/sec. Direction varied between ±10° from the center of the display, and it was varied through a staircase procedure. Observers fixated 2° of the outer margin of the display at midline, left and right, so that the stimuli were presented in the left or right visual field. After 800 msec of motion, a static random dot mask with a vertical line appeared at a horizontal angle from the true heading. In a 2AFC task subjects indicated whether their heading was to the right or to the left of the vertical line. (B) Shows the performance (Threshold of target offset ±SD) of patients SS and SR and 6 healthy control subjects.

Heading perception

The previous test demonstrated that patients can detect/discriminate the radial patterns of motion that are associated with the forward movement of an observer. Here we examined how well the patients can pick up characteristics of radial flow fields and determine their direction of (simulated) self-movement by correctly detecting the focus of expansion (FOE). A schematic figure of the test is illustrated in Figure 4A. This and the following tests were administered to the patients at the time when their perception of radial motion was normal.

Patients’ performance plotted in Figure 4B, shows that that their performance was within normal ranges (z-scores were <0.5, in both patients and for stimuli shown in either visual field). SS and SR accurately judged heading despite severe deficits on several tasks of low level motion perception, including direction discrimination. An explanation of these results may be provided by our recent study [37] in which we demonstrated that when judging straight line heading observers tolerate a large amount of directional noise in the velocity vectors. Furthermore, the performance on a large battery of visual motion tests in our patient RA [6,38], also showed that straight line heading judgments do not require highly accurate perception of the direction of 2D image velocities.

Perception of approach without optic flow

Patients SS and SR detected both optic flow and scale change as accurately as the healthy control subjects (in both tests and for all speeds, Z scores were either not significant (z-score <1) thus indicating that they were not impaired on the detection of self-movement. Furthermore, this task indicates that the patients’ motion perception in depth (z-axis) was preserved.

In summary, on tasks that involved global motion and direction of motion in depth, and also they do not contain form or edges detection, performance was preserved in the normal range. Moreover, the quadranatopic visual field loss did not interfere with performance on the task.

This task can be seen to comprise many of the components tested in the other tasks. It involves perception of motion direction in depth and we also manipulate the presence of global motion. Given the results so far we would expect SS and SR to be poorer on the task than the normal observers due to their impairment on direction discrimination tasks and also because of the visual field loss. As seen in Figure 6B, in both the moving and stationary observer condition the task was easy for the controls (>93% correct). In contrast the patients were severely impaired (chi-square>20, p<0.001) on the static observer condition. Although the performance of SS and SR was lower than that of the healthy controls, a moving observer advantage was clearly seen (for SS chi square=58.1, and for SR=82.5, with p>0.0001). These data indicate that both patients were significantly better at detecting the colliding object trajectory in the test condition illustrating the simulated forward motion of the observer.

Discussion

SS and SR, patients with occipital lobe damage, showed deficits on motion processing when the task involved computation of motion-defined edges, but they had no deficits on a range of tasks that involved observer-scene motion. This shows that precise basic motion measurements are not necessary for the perception of observer-scene motion (see also [6]).

In the collision detection tasks SS and SR showed a moving observer condition advantage which is in-line with a preservation of observer-scene motion processing.

The results of the collision detection task prompt the question: why are observers better in the moving observer condition? The moving condition contains more potentially distracting and confusing image motion, so it might be predicted to be more difficult. We offer the following potential explanation.

An object on a collision course is identified by an unchanging visual direction relative to the observer in 3D; an object on any other course continuously changes its direction. Therefore the observer must search amongst moving distractors for a target that maintains its visual direction. Laboratory experiments show this is a difficult task that requires a serial search (e.g. [39]) and this difficulty can be recognised in the natural world: at sea, sailors are familiar with the constant bearing, decreasing range (CBDR) hazard. A ship on a collision course maintains its visual direction and sailors have to be trained and posted on the bridge to look out for CBDR situations. This difficulty in spotting an object that does not change in direction explains the difficulty of the static observer condition. However the moving observer condition may be different.

In the observer-moving condition the movement of the target and distractor objects is unchanged. However the ground plane is moved to simulate forward movement of the observer. It is the movement of the ground plane that may make the task easier. Johannson [40] suggested that the brain divides retinal motion into two components, a global motion component and a relative motion component. Recent work on “flow parsing” suggests the brain identifies objects moving within the scene by parsing (or dividing) out optic flow components (patterns of global motion that are characteristic of self-movement) from the retinal flow field, and so isolating retinal motion due to the movement of objects within the scene (e.g. [2,41-51]). Either framework can be applied here. In the static condition there is no net global motion and so no component of motion is parsed out. In the observer-moving condition, due to the movement of the ground plane, there is a net radial component of motion. If a radial component is removed then it will be removed from all the objects in the scene, including the target (see [45] for an illustration of this effect). The result of subtracting an outwards radial component of motion off the target will be that it ends up with an inward components of motion (again see [45] for a demonstration of such an effect). Therefore in the moving condition, the observer’s brain is no longer trying to detect an absence of motion (a difficult task), but rather a difference of motion in depth (the object uniquely identified by inward motion), a somewhat simpler task.

Conclusions

In summary, in this paper we have shown that perception of observer-scene movement can be preserved despite deficits in low-level motion processing (relative motion in the xy plane). We also showed that processing relative motion in 3D can be dissociated from processing relative motion in the xy plane.