SFS? Not likely….

SFS (Shape From Shading) theory is based upon the Lambertian paradigm. Our visual demonstrations imply that this paradigm fails to apply to the conventional stimuli used to probe vision.

The shading cue is important in vision of form (Metzger, 1975). The theory is based on Lambert's (1760) account of surface scattering, implying that the luminance at the eye depends on the inclination of the local surface with respect to the global direction of illumination. Shape From Shading (SFS) algorithms of machine vision are based upon this concept (Horn & Brooks, 1989).

A sampling of artistic techniques suggests that visual awareness might not derive from such algorithms. Here, we demonstrate that the conventional probe does not imply SFS at all. We address the phenomenology involving the shape of the region of interest (ROI) of the “conventional stimulus” of SFS research.

The effective stimuli for SFS are spatial luminance gradient variations. The conventional stimulus is a uniform luminance gradient, localized in an ROI, arguably the simplest possible instance. A circular ROI minimizes form cues. In use for decades, the conventional stimulus became widely known through Ramachandran's (1988) work.

ROI shape is important (Sun & Schofield, 2012; Wagemans, van Doorn, & Koenderink, 2010). A square leads to a cylindrical, a triangular to a conical form (Figure 1), suggesting that the luminance gradient is not the only cue of relevance. The “shape cue” (due to the outline) and the “shading cue” somehow combine. The removal of the outline destroys the impression of form (Erens, Kappers, & Koenderink, 1993).

Figure 1.

At left the “conventional stimulus” of SFS. At center and right, the ROI has been changed from circular to square or triangular. Observers are aware of (from left to right) a sphere, a cylinder, and a cone. (Perhaps slightly “flattened in depth,” sometimes inverted in depth.) The nominal “SFS stimulus” is the same in all cases.

We suggest a novel interpretation: the uniform gradient is irrelevant. What matters is the change of the luminance discontinuity along the contour. This is not “cue combination,” but the effect of a single cue of different kind. We offer a “visual proof.”

Indeed, the uniform gradient inside the disk is irrelevant. Consider the two images at the top of Figure 2. The left image is the conventional stimulus and the right disk is uniform (no gradient!), whereas the background is given a uniform gradient. Observers are aware of a spherical cap in both cases (Shapley & Gordon, 1985). An even more extreme case is the image at the bottom left of Figure 2, where both the disk and the background are uniform. Only the circular contour is modulated. Again, observers report a spherical cap. The “sphericity” is judged about as good as that elicited by the conventional stimulus. On initial exposure the cap looks perhaps slightly flatter than in the case of the conventional stimulus, but this tends to change after a while. SFS algorithms do not even apply here.

Figure 2.

Main “visual proof.” At top left the conventional stimulus. At top right the background, not the disk, gets the gradient. At bottom left both the background and the disk are uniform, except for a narrow strip about the circular outline. (Proof at bottom right, the black strip occludes the immediate neighborhood of the circular circumference.) This works as well as the conventional stimulus. Apparently the gradient in the disk is largely irrelevant. In all cases (except bottom right, of course) one has an awareness of “light coming in from above.”

The visual proof perhaps reminds one of the illusions related to lateral inhibition and Mach bands, illustrated in Figure 3. You can “color” a disk without spending paint on its interior, you merely modulate its outline (Cornsweet, 1970). Such effects are extensively used by visual artists (Ratliff, 1965). It is how you paint the full moon using black ink.

Figure 3.

You can “color” a disk by tampering with its contour. At top left a light disk on a darker ground. At top right the background and the disk are both uniform and of the same luminance (“proof” at bottom left). The disk appears “lighter” than its background. You may change the polarity (bottom right). This illusion is different from the shading case shown in Figure 2 (bottom left). There is no apparent “direction of illumination” here.

The “watercolor illusion” reported by Pinna (1987) also fills in large areas. It does not lead to effects reminiscent of SFS though.

Does this imply human vision fails to use SFS at all? No, but it implies that the conventional stimulus is too poor to reveal this. It offers no gradient variations, except at the contour. These variations involve gradients of much greater magnitude than the uniform gradient in the interior. Thus, the contour gradient variation dominates the appearance. The effect reveals an additional cue (“contour-shading gradient cue”?), whereas it has little impact on the study of SFS per se.

Is the method used in bottom left of Figure 2 also used in the visual arts? Yes, all the time. The method is common in early twentieth-century painting. It allows one to “shade” for form, without being forced to take recourse to “tonal” painting. This is crucial when artists care to preserve the picture plane as an object, instead of suggesting a ghostly “window” into another world. Because the shading is local, it requires more effort to integrate mutually distant parts of a single outline. It involves highly non-trivial Gestalt formation. This leads to novel sources of ambiguity that the artist might exploit. It was heavily used in cubism (Picasso, 1910), and extensively applied in works as that by Klee (1930).