Inferred motion perception of light sources in 3D scenes is color-blind.

In everyday scenes, the illuminant can vary spatially in chromaticity and luminance, and change over time (e.g. sunset). Such variation generates dramatic image effects too complex for any contemporary machine vision system to overcome, yet human observers are remarkably successful at inferring object properties separately from lighting, an ability linked with estimation and tracking of light field parameters. Which information does the visual system use to infer light field dynamics? Here, we specifically ask whether color contributes to inferred light source motion. Observers viewed 3D surfaces illuminated by an out-of-view moving collimated source (sun) and a diffuse source (sky). In half of the trials, the two sources differed in chromaticity, thereby providing more information about motion direction. Observers discriminated light motion direction above chance, and only the least sensitive observer benefited slightly from the added color information, suggesting that color plays only a very minor role for inferring light field dynamics.

The human visual system is remarkably successful at disentangling lighting from object properties in the ambiguous light signal entering our eyes. Several studies show that observers estimate both surface properties and light field parameters from static views of 3D scenes (see Maloney, Gerhard, Boyaci, & Doerschner, 2011, for a recent review). We have shown that observers also estimate dynamic light field properties in 3D scenes, an ability linked with improved sensitivity to surface reflectance (Gerhard & Maloney, 2010a). Analyses showed that perceiving light field dynamics depends heavily on the 3D surface geometry of the scene's objects, much more so than on the raw luminance patterns in the images (Gerhard & Maloney, 2010b). However, previous dynamic light stimuli were achromatic. Here, we ask whether color improves sensitivity.

Following previous studies, our subjects viewed rendered 3D scenes stereoscopically. These stimuli simulated a sun setting over a hilly landscape (Figure 1a) viewed from above (Figure 1b). Monocularly, peaks and valleys cannot be disambiguated in such scenes so lighting direction is inferred correctly only up to a 180° flip, and disparity must be used to avoid the concave/convex ambiguity. Trials were 1,500 ms: the initial 750 ms were static so observers could acquaint themselves with the geometry and initial lighting, in the final 750 ms, the out-of-view collimated light source, the “sun,” rotated 10° overhead in one of the four directions (Figure 1c). The task was 4AFC to indicate the sun motion direction. Feedback was never given, and there was no training.

Figure 1.

Hilly landscapes and lighting. (a) An example landscape seen from a view never used experimentally but shown for illustrative purposes. (b) Example stereo pairs for crossed and uncrossed fusion. Subjects viewed the scenes stereoscopically from directly overhead as shown. Monocular views do not disambiguate hills from peaks; only stereo disparity resolves the concave/convex ambiguity and therefore the direction to the light. (c) The sun rotated 10° over the landscape in one of the four directions in the image plane. The task was 4AFC to indicate motion direction. (d) In half of the trials, the sun had chromaticity coordinates equal to one of the eight hues equally spaced on a circle in CIE La∗b∗ space centered on D65. The hue with ∗ has CIE x, y coordinates (0.375, 0.425). In the other half of the trials, the sun was D65. The sky was always D65. (e) A hilly landscape illuminated by a yellowish sun and neutral sky.

The sun's initial position above the scene was randomized on every trial both in azimuth (angle in the image plane) and in elevation, held between 90° (perpendicular to the ground plane) and 70° (slightly off-perpendicular). Random initial positions forced observers to track light motion because position in any one frame was uninformative for the task.

An additional fixed light source also illuminated the scene: a homogeneous “sky.” The sky was always neutral in hue and a quarter intense as the sun, which had intensity 100 cd/m2. The lights' intensities were fixed, but we varied their chromaticity coordinates. In half of the trials, both had chromaticity coordinates corresponding to D65, the standard neutral daylight hue (achromatic condition). In the other half, the sun had one of eight hues (chromatic condition) shown in Figure 1(d). Figure 1(e) shows a chromatic scene. In total, 1,216 trials were pre-rendered: 608 achromatic and 608 chromatic (76 per hue). Blocks alternated by condition, with first block type counterbalanced over subjects. Hue trial types were intermixed in chromatic blocks.

A new random Gaussian landscape was generated for every trial. We ensured that shading was the only cue to light direction by assuming Lambertian reflectance and disallowing combinations of sun trajectories and scene geometry causing cast shadows since they are an unambiguous cue to light direction even monocularly. We rendered 16 frames for each trial, in which the sun rotated in increments of 0.625°. The experiment was written using the Psychtoolbox (Brainard, 1997; Pelli, 1997) and run on a stereoscope of two linearized monitors. All details of the equipment, scene generation, and stereo rendering process are identical to those described in Gerhard and Maloney (2010b).

Eight subjects participated. All had normal or corrected-to-normal acuity, passed the Ishihara color plate test, and passed a stereo test emphasizing concavity/convexity judgments from Gerhard and Maloney (2010a). All gave informed consent before participation and were remunerated $8/hour. Experiments were approved by the New York University Institutional Review Board and were in accordance with the Helsinki Declaration.

Results are shown in Figure 2. Observers performed above chance overall, achieving between 31% and 77% correct after a correction for guessing; however, color information did not significantly improve sensitivity to light motion direction (Figure 2a) except for the least sensitive observer who improved by 7%. Observers were equally sensitive to all motion directions (Figure 2b) and for all hues (Figure 2c).

Figure 2.

Results. (a) Motion discrimination performance with 95% confidence intervals for each subject (corrected for guessing, chance = 0). Only the least sensitive observer improved significantly (by 7%) with added color. (b) Sensitivity did not depend on motion direction. We calculated d′ for each motion direction separately by randomly splitting each subject's data into four sets of 304 trials. Average d′ values ±1 SEM are plotted by motion direction (Up, Down, Right, Left). (c) Sensitivity did not depend on hue. Average percent correct (corrected for guessing, chance = 0) is plotted with colored 95% confidence intervals at corresponding locations in CIE La∗b∗ space.

Added color information does not appear to play a large role in sensitivity to light field dynamics. Previous analyses emphasizing the importance of scene geometry provide much more explanatory power for this kind of light field inference in hilly scenes illuminated by a moving sun and sky (Gerhard & Maloney, 2010b).