Awareness of the light field: the case of deformation.

Human observers group local shading patterns into global super-patterns that appear to be illuminated in some unitary fashion. Many years ago, this was noticed for the case of uniform, unidirectional illumination. Recently, we found that it also applies to convergent and divergent illumination flows, but that human observers are blind to rotational light flow patterns (in the sense of being unable to group the local shading patterns). We now report that human observers are also blind to deformation patterns. This is perhaps interesting because convergent, divergent, rotational, and deformation patterns all occur in natural light fields. This is an idiosyncrasy of the human visual system, on par with the fact that visual awareness fails to present the observer with saddle shapes.

Introduction

The conventional stimulus in shape from shading research for about a century has been a circular disk filled with a linear intensity gradient (Metzger, 1975; Ramachandran, 1988a, 1988b). It represents most likely an attempt to catch the essential information in shading patterns, which is supposed to be the first-order intensity variation in a small neighborhood about a point (Turhan, 1935). The result is somewhat troublesome, because of the effect of supposedly irrelevant factors. For instance, not just the intensity gradient determines the resulting perception, the circular outline is just as important (a square outline yields a very different perception; Wagemans, van Doorn, & Koenderink, 2010). However, this basic pattern has been adopted in a large volume of research, which is the reason why we used it in the present study (Figure 1).

Figure 1.

The conventional stimulus pattern used in psychophysical shape from shading research. It is a circular disk, filled with a linear intensity gradient, placed on a background of the average intensity of the disk.

Most observers report that the basic stimulus appears to them as a pictorial relief, modulated in depth. A small group of observers sees a flat intensity gradient (we are not in a position to put a number on this, from our experience, we would guess at least one out of ten). Observers who report a depth articulation declare to see either a “cap” (convexity, like the outside of an eggshell) or a “cup” (concavity, like the inside of an eggshell). The depth ambiguity has been familiar for a long time (Rittenhouse, 1786). As we have argued elsewhere, the fact that only caps or cups are reported is perhaps surprising, since inverse optics shows that cylinders (either convex or concave) and saddle shapes are equally valid interpretations. Here “valid” refers to “Shape From Shading” (SFS) theory, based on standard radiometry (Boyd, 1983; Lambert, 1760; Moon & Spencer, 1981). SFS has a large literature; we mention only Belhumeur, Kriegman, & Yuille (1999), Forsyth & Ponce (2002), Horn & Brooks (1989), van Diggelen (1959), From a Bayesian viewpoint, saddles should be the preferred interpretation (Koenderink & van Doorn, 2003b). Whereas cylinders are indeed reported for a square outline, saddle shapes are (in our experience) never spontaneously reported. The “cap or cup” decision is often determined by the direction of the gradient, which is possibly interpreted as the direction of illumination in the “scene”.

It is well known that a group of such basic stimulus patterns tends to “synchronize” (Kleffner & Ramachandran 1992). That is to say, when you view a group of such stimuli with gradient directions all lined up, one tends to be aware of a group of all caps or all cups (Figure 2, left). If there is an outlier in a flock of caps, it is often reported as a cup and vice versa (Figure 2, second from the left). This finding is also related to the work on the perception of light flow direction (Koenderink & Pont 2003; Koenderink and van Doorn, 1996; Koenderink, van Doorn, & Pont, 2004; Koenderink, van Doorn, Kappers, te Pas, & Pont, 2003; Koenderink, Pont, van Doorn, Kappers, & Todd, 2007; Pont & Koenderink, 2003, 2004). In the Appendix, we explain the concept of “light field” and “surface illuminance flow” in some detail. Roughly speaking, the “light field” describes the transport of radiant power, whereas the “surface illuminance flow” describes the aspect of the illumination that photometrically reveals the roughness of the illuminated surface.

Figure 2.

Left, a group of basic stimuli with the same gradient direction (uniform illuminance flow). Second from left has an outlier at the center. Third from left has concurrent gradient directions (divergent illuminance flow). Right, a group of basic stimuli with a rotational illuminance flow.

Recently, we have found (van Doorn, Koenderink, & Wagemans, 2011) that this “synchronization” (a kind of “grouping”) also occurs for divergent and convergent illuminance flows (Figure 2, third from left) but not for rotational illuminance flows (Figure 2, right). This is interesting, because all first-order singularities of the illuminance flow field (i.e., divergence, rotational, and deformation fields) occur in the daily environment (Mury, Pont, & Koenderink, 2007, 2009a, 2009b). Thus, the apparent selective sensitivity appears perhaps surprising. In any case, it is a nontrivial property of the structures in human visual awareness. The stimuli shown in Figure 2 may be understood as abstractions from naturally occurring patterns, rendered suitable for a structured empirical investigation.

In Figure 3, we show the basis for the linear (local) approximation of illuminance flow fields. The zeroth-order, unidirectional, uniform field can be locally deformed by linear combinations of these four patterns. When the uniform flow vanishes, at singular points of the illuminance flow field, one generically has a purely linear flow pattern. All these components occur in the natural environment. Thus, it makes sense to find out whether deformation patterns can group (like divergent patterns) or not (like rotational patterns).

Figure 3.

A basis for the first-order structure of the illuminance flow. Any first-order structure can be expressed as a linear combination of these components. At top left one has a divergence, at top right a rotation, in the bottom row, two mutually independent deformation patterns. The divergence and rotation are specified by magnitude alone, the deformations also have an orientation.

In this paper, we test whether human observers are able to group deformation patterns as they do uniform or divergent patterns or whether they are blind to these as they are to cyclic (rotational) patterns.

Methods

Observers

Observers AD, JK, JT, and JW were the authors, whereas EG, MS, RK, and SC were naive to the issue. We do not consider the addition of a group of naive observers to be essential, because we are interested in a possible form of selective blindness. If a group of experienced observers turns out to be unable to group deformation patterns, it is hard to imagine why this would be different for naive observers. Moreover, the advantage of the experienced observers is that they are very well documented as observers in various subdomains of visual perception. Three (AD, JK, and JW) were also observers in a previous, related study (van Doorn et al., 2011). One (JT) has a known tendency to report only “caps” (never “cups”) when confronted with the conventional stimulus pattern. As it turned out to be the case (see Results), both groups of observers yielded equivalent results.

Stimulus design

The stimulus design was similar to that used in an experiment upon which we have reported before (van Doorn et al., 2011). The only difference is the number of basic stimulus patterns placed on the annulus (Figure 4), which is eight in the present and six in the previous experiment. The number is important, because the deformation stimuli cannot be represented very well using only six samples, whereas eight samples suffice.

Figure 4.

Left, The stimulus patterns used in the experiment. At top left a uniform field, at top right a random field (of course this is just one instance, all instances are different). The bottom row shows two deformation fields with different orientations. The red cross is used for fixation. Right, the same patterns, but shading gradient indicated with arrows.

Experiments

We report on two related experiments. Both potentially yield an answer to our research question, although the second experiment perhaps allows some additional conclusions.

Simultaneous task

In the “simultaneous task,” we presented stimulus patterns like those shown in Figure 4, randomly rotated over some multiple of 45º. The observer fixated the red cross, then was presented with the pattern for 2 s, after which the observer had to indicate whether the eight patterns were either

all caps,

all cups,

not all the same.

This is a simple, overall decision. When all subpatterns are either cup or cap, the whole pattern looks like it was made of one piece, it becomes a single Gestalt.

The patterns were presented on a computer screen. The background gray disk was 15 cm in diameter and binocularly viewed from 50 cm. Room lights were dimmed. The trials were self-paced by the observers (See Figure 5.)

Figure 5.

The presentation sequence for the simultaneous task. Each trial starts with a blank plus fixation mark. This enables the observer to fixate the red mark. The observer initiates the next presentation, which presents the stimulus pattern for 2 s. In the next phase, the blank pattern is on until the observer has responded. The response indicates a triadic judgment: all caps, all cups, or mixed.

Observers were tested with 500 presentations in all (1,000 AD, JK, JT, and JW), equally distributed over uniform, random, and deformation fields, in random order. In the case, the observer felt fully uncertain, the trial could be dumped, and another one initiated. This happened in not more than about one in 100 cases. Overall, all observers judged the task as “easy but dull”. All but one observers completed the task in one session, observer JW needed two sessions.

For the uniform patterns, we checked how the all cap/cup report correlated with the direction of the gradient (Figure 6). We find the expected pattern for the observers. Outlier is observer JT, who (as we knew) will never report all cups. The remaining observers have a strong but idiosyncratic preference for cap over cup.

Figure 6.

At left, the polar frequency plot for reporting “all caps” (red) or “all cups” (blue) as a function of the direction of the gradient. The concentric areas show the quartiles over all observers. Notice the decided preference for cap over cup. At right, the plot for observer JT. This observer is special in that he never reports awareness of all cups.

Conclusions simultaneous task

The results of responding “caps”, “cups”, or “different” are shown as a triangle plot in Figure 7. One has to judge whether the deformation results are more like the random or the uniform results. The answer is immediately obvious. The result for the deformation patterns is like that for the random patterns, and very different from that for the uniform patterns.

Figure 7.

Triangle plot of responding convex (“all caps”), concave (“all cups”), or “different”. The elliptical areas show the 95% and 50% probability areas, the dots show individual cases. The bluish area corresponds to the uniform pattern, the reddish area to the random patterns, whereas the yellowish area represents the deformation patterns. Conclusion is that the results for the deformation patterns resemble those for the random, rather than the uniform patterns.

The conclusion is that human observers fail to group deformation into a single Gestalt. The deformation patterns look qualitatively different from the uniform patterns (and, to the three observers who participated in an earlier experiment, to the convergence or divergence patterns). They look more like the random patterns (and again, to the three observers who participated in an earlier experiment, to the rotation patterns).

Successive task

In the “successive task,” we again presented stimulus patterns like those shown in Figure 4. The observer fixated the red cross and was then presented with the pattern for 1 s. After this, there came two successive periods of half a second each. In each period, one subpattern was indicated with an easily visible dot. For the first period, the dot was red; for the second, it was blue. In this task, the observer has to indicate, for both the first and the second interval, whether the indicated subpattern was judged to be a cap or a cup. Thus, there are four possible responses, namely cap-cap, cup-cup, cap-cup or cup-cap. In the former two cases, the two partial responses are correlated; in the latter two cases, they are anti-correlated. (See Figure 8.)

Figure 8.

The sequence of presentations in a trial of the successive task. The trial starts with a blank plus fixation mark, enabling the observer to fixate. The observer initiates the following three presentations. First the stimulus pattern is presented for 1 s. Then, follow two one-half-second presentations in which the fiducial patterns are identified: the first through a red, the second through a blue dot. The fourth presentation is blank again. It ends when the observer has responded. The response involves a cap-cup decision for the first (red) pattern and one for the second (blue pattern). Thus, possible responses are cap-cap, cap-cup, cup-cap, and cup-cup.

The timing of the presentations is important. Naive observers initially tend to complain that the presentation is fast, although they evidently are able to handle it. Experienced observers sometimes complain of the slowness of the presentations because they notice occasional cap-cup or cup-cap flips which interfere with the task. The timing that we eventually settled on is somewhat of a compromise. The temporal intervals were chosen such that it is unlikely that the second fiducial would have flipped with respect to its appearance during the indication of the first fiducial. We could draw on our previous experience here. The probability of a flip is less than a percent or so.

The patterns were presented on a computer screen. They were again 15 cm in diameter and binocularly viewed from 50 cm. Room lights were dimmed. The trials were self-paced by the observers. Again, random, uniform, and deformation patterns were generated with equal probability. They were randomly rotated over multiples of 45º. We used a total of 500 presentations (1,000 for AD, JK, JT, and JW). This task is rather more difficult than the simultaneous task. In a few percent of cases, the observer failed to see the first or second indication. Such cases could be discarded, and an additional trial generated.

In the case of observer JT, who never reports to be aware of cups, the “cup” judgment was replaced with a “not cap” judgment. Judging from the results, this at least partly succeeded. At least the random and deformation cases are evidently distinct from the uniform case. Observer JT reports to be aware of pictorial relief; he apparently does not merely respond to the gradient directions. In the case of caps, these appear as convincingly three-dimensional; in the case of cups, these are “less convincing” and might as well be interpreted as planar gradients.

What is interesting from a phenomenological viewpoint is that the observers apparently used the relief appearance, rather than the (two-dimensional) gradient representation. It happens occasionally that one is aware of a cap-cap event, whereas (in reflective afterthought) one notices that the gradients were opposite. In such (rare) cases, the microgenetic process possibly assumed different local illumination directions for the first and second fiducial subpattern.

From the results, we may compute autocorrelation functions (see Figure 9). These specify precisely how seeing cap or cup at the second presentation depends upon cap or cup at the first presentation, as a function of the angular separation of the subpatterns.

Figure 9.

At top, the median autocorrelation functions for all observers except JT and the three stimuli (from left to right: uniform, random, and deformation). The error bars indicate the interquartile ranges. The correlations specify how the judgment at the second location depends on that at the first location, as a function of the angular distance between the locations. (notice that the outermost positions are repeats of the opposite location.) Yellow indicates positive, blue negative (anti-) correlation. The dashed red lines indicate naive predictions. At bottom, the autocorrelation functions for observer JT (the 95% significance intervals are indicated in red). This observer evidently shows a different pattern. JT's results for the uniform patterns are not different from those for the other observers. His results for the random and deformation patterns are different but do not contradict our overall conclusions.

Conclusions successive task

The results from the successive task are—at the expense of some hard labor—much more detailed than those of the simultaneous task. A cursory glance shows that the deformation case is much like the random and very much unlike the uniform case. Thus, the conclusion that human observers appear to be blind to the deformation patterns is immediate, without the need for any significance testing.

Of course, one has certain predictions. In the case of the uniform pattern, we expect the appearance to be either all caps or all cups. Thus, one expects the autocorrelation function to be equal to one (perfect correlation) for all angular separations. This expectation is borne out to a surprising extent.

For the random case, one expects (of course) the correlation to be perfect if the angular separation is zero. Anything less would imply cap-cup flips within a half-second interval. This is pretty much borne out. One also expects correlations for non-zero angular intervals to be essentially (within the random scatter) zero. Perhaps surprisingly, this is not borne out. Except for observer JT, who fails to report cups in any case, observers tend to (minor) anti-correlation, except for the smallest angular separations, for which there is a (minor) tendency to positive correlation. We do not understand these (minor) trends, which are evidently systematic.

For the deformation case, one expects (in case grouping would fail to pertain) antipodes (around 180° angular separation) to anti-correlate separations of plus or minus ninety degrees not to correlate at all. Except for the (special) case of observer JT, this appears to be borne out by the data. However, the difference with the random case (which we fail to fully understand) is only slight, so we can hardly conclude otherwise than that the deformation case is very much like the random one.

Overall conclusions

The overall conclusion from these experiments is clear-cut: human observers do not group deformation patterns like they do uniform, convergent or divergent ones. We have not been able to robustly differentiate the results for the deformation patterns from the results of the random patterns. They are categorically different from the results for the uniform patterns though. Thus, the research question posed at the initiation of the experiments has been definitively answered.

There remain interesting unanswered questions. For instance, it is unclear how the anti-correlations for non-zero angular separations in the random case arise. We guess that this has to do with local grouping processes (for angular separations of less than half the annulus) that mutually compete to settle the global grouping. The “grouping” seems to be accompanied by an “anti-grouping”. We have met similar effects in a previous experiment for the case of rotations. This appears to be a worthy target for experiments yielding more detailed data. Such experiments are not likely to be easy though in view of the fact that it took several decades to move from preliminary qualitative observations to detailed quantitative data. There is clearly more detail to be probed. For instance, our observers occasionally spontaneously noticed a dissociation between pictorial relief (cap-cup distinction) and visual field (gradient direction) properties. It might be possible to probe this.

A side issue, which is nevertheless of interest, is the difference between observer JT and the others. All observers are aware of compellingly three-dimensional caps, but whereas all but one observer are also aware of equally compelling cups, observer JT is not. Whereas the “cups” are obviously “non-caps” (which is why JT was able to complete the experiment), they are not compelling as three-dimensional objects but might as well be described as planar figures. In the past, we have encountered similar observers, as well as observers who failed to see any conventional stimulus as convincingly three-dimensional. In two cases, we met with reviewers who declared that they failed to “get the illusion”, meaning that they were not visually aware of caps or cups. Apparently, one may distinguish among various groups of distinct observers. There appears to be no literature on the issue; it seems a potentially rewarding research topic.

In conclusion, the human visual system appears to be idiosyncratic in that it easily groups unidirectional, convergent, and divergent illuminance flow patterns but fully ignores rotational and deformation patterns. This is the case even though the first-order patterns all occur in the natural environment (Mury et al., 2007, 2009b). This “selective blindness” is reminiscent of the inability to be visually aware of the conventional stimulus pattern as anything else but cap or cup (Wagemans et al., 2010). In the case of surface illuminance flow (Pont & Koenderink 2003, 2004), we actually know that the saddles which are ignored are actually more abundant in the natural environment than the caps and cups taken together (Koenderink & van Doorn, 2003b). To understand such idiosyncrasies, one will need to study the affordances of these patterns in further detail than has been attempted so far. Perhaps insights from the arts, rather than from classical physics will prove useful there (Baxandall, 2005; Jacobs, 1988; Luckiesh, 1916; Koenderink & van Doorn, 2003a).