Contour erasure and filling-in: New observations.

Contour erasure is a newly established form of flicker adaptation that diminishes the saliency of object edges leading to their complete disappearance (Anstis, S. 2013. Journal of Vision, 13(2):25, 1-14). If these "disappeared" objects are then viewed on textured backgrounds, the observers experience filling-in, the illusory sense of background completion in the absence of physical input. In a series of observations, we demonstrate that contour erasure can greatly speed up the filling-in (or fading) of brightness. Based on these observations, we suggest that contour adaptation happens early in the magnocellular pathways.

Introduction

This report discusses perceptual filling-in that occurs in two rather different situations. The first is Troxler fading, in which a peripheral spot appears to fade out during steady fixation, whereupon it appears to be filled-in with the colours and textures of its immediate surroundings. The second is the perceptual filling-in of the natural blind spot.

Perceptual filling-in is the interpolation of missing information across visual space (Spillmann & de Weerd, 2003; Weil & Rees, 2011). It is a ubiquitous and important part of visual perception, since parts of the retinal image often fall upon non-functioning retinal regions such as the blind spot or retinal blood vessels. Eye blinks can interrupt visual messages in time, and the occlusion of one object by another can interrupt them in space. Despite these interruptions in scene viewing, we experience a unified, continuous, and coherent perception of the world. Filling-in may be one mechanism contributing to this process. Perceptual filling-in can take many different forms, and Weil and Rees (2011) provide a useful classifying scheme (see Table 1). These authors divide filling-in into stimulus-dependent types, which depend upon the exact stimulus being viewed, and stimulus-independent types, such as filling-in of the blind spot or of retinal scotomata, which apply to all stimuli at all times. They further divide the stimulus-dependent and stimulus-independent filling-in into fast and slow types. Fast filling-in includes the perception of illusory contours such as the Kanisza (1976) square or neon spreading (Bressan et al., 1997; van Tuijl & de Weert, 1979) or the filling-in of afterimage colours (Francis & Kim, 2012), and also the amodal completion of objects that are partly hidden from view by occluding objects. Slow forms of stimulus-dependent filling-in include Troxler fading and artificial scotomas (for reviews see, Anstis, 2010; Komatsu, 2006; Pessoa & de Weerd, 2003; Weil & Rees, 2011). Moreover, pattern disappearance of peripherally viewed stimuli can be evoked by brief masking stimuli (Kanai & Kamitani, 2003; Moradi & Shimono, 2004) and/or adaptation to high-contrast stimuli (Motoyoshi, 2010; Motoyoshi & Hayakawa, 2010).

Table 1.

Different types of filling-in, sorted according to their speed of occurrence and stimulus dependency (after Spillmann & de Weerd, 2003 and Weil & Rees, 2011).

Stimulus dependent	Stimulus independent
Fast
Illusory contours	Filling-in at blind spot
Neon-colour spreading	Filling-in of retinal scotoma
Watercolour effect
Retinal afterimages
Slow
Troxler fading	Stabilised retinal images
Contour erasure	Scene fading with blurred images
Figure–ground texture segregation
Note. The forms of filling-in we address in this report are highlighted in bold font.

We shall not attempt here to discuss all forms of filling-in. Instead, we reflect our own current research interests by concentrating on filling-in during Troxler fading, on the one hand, and on the filling-in of the retinal blind spot, on the other. A novel feature that we introduce here is the newly discovered process of contour erasure (Anstis, 2013), which accelerates the border erosion process that precedes Troxler fading.

Our observations

When do objects fade from view? Troxler (1804) pointed out that during strict fixation, peripherally viewed stationary objects tend to fade out and disappear. This report adds to the original observations Troxler made more than two centuries ago. Indeed, objects need not be far into the periphery, and they do not even need to be stationary. Hamburger, Prior, Sarris, and Spillmann (2006) demonstrated that the surface of directly viewed, stationary objects could fill, assuming strict fixation. Schieting and Spillmann (1987) and Anstis (1996) found that during strict fixation a flickering spot viewed in the periphery gradually appeared to flicker less strongly and finally disappeared from view. Indeed, entire stationary scenes, when blurred, can fade from view (Simons et al., 2006). Here we show that moving objects can also fade from view. Movie 1, shown at http://i-perception.perceptionweb.com/journal/I/volume/5/article/i0624rep, shows an annulus of dense random dots that rotates clockwise against a background of randomly twinkling dots. During strict fixation on the central spot, the whole annulus gradually seems to become filled with the same twinkling dots as the surround, so that it slowly disappears from view. The whole field now appears to be uniformly filled with twinkling dots.

However, a hidden non-uniformity appears when the motion is stopped. Now all the background spots look (and are) stationary, but within the annular region we can perceive a strong counter-clockwise motion aftereffect.

What happens in the visual system when the annulus gradually disappears?

Do the areas (the background and the annular region) come to look alike? To consciousness, the answer is yes, since all the dots in the whole field appear to twinkle randomly. But what about the neural signals? These must be different, since when the motion is stopped an aftereffect of motion is seen in the annulus but not in the background. So, at some unconscious level, the annulus and the background are processed differently. This is an illustrative example of the difference between conscious and unconscious perception: during adaptation direction-selective motion detectors are adapted, leading to fading and to the motion aftereffect upon cessation of motion.

Alternatively, are the motion-defined boundaries between the annulus and background subject to adaptation? This might make the ring and background textures indiscriminable, perhaps owing to fading of the borders followed by some kind of texture filling-in.

Adaptation of boundaries implies that peripheral fading takes place in two stages. Spillmann and de Weerd (2003) proposed that the perceptual filling-in of a figure by its background during fixation results from a two-stage process. First, there is a slow adaptation of mechanisms that normally keeps the figure segregated from its background (cancellation). This adaptation may involve both low-level edge-detection and high-level figure–ground segregation processes. Second, after these segregation processes have been suppressed by adaptation, there is a fast interpolation process in which the background colours or textures are substituted into the area previously occupied by the figure. We shall think of this two-process theory with the metaphor of a dam. The idea is that the borders of a stationary, peripherally viewed spot erode gradually over time, like a dam that is gradually worn away until it finally is breached. Following the breach, there is rapid process of filling-in, like water gushing in through the broken dam. Contour erasure is an adaptation to the dynamic components of the pattern that weakens the perception of the annulus borders, thereby permitting the onset of rapid filling-in.

Normally, visual borders erode slowly and gradually. But in this report, we shall show how it can be dramatically sped up by a newly discovered “contour adaptation” (Anstis, 2013), which allows filling-in to take place more rapidly. Movie 2 shows this contour adaptation in action. Contour adaptation leads to the temporary disappearance of the adapted crosses.

Movie 3 shows another example of contour adaptation and compares it to adaptation to a flickering surface. There are two round adapting shapes. On the left is a flickering outline circle. On the right is a flickering blurred disk, containing virtually all of the information of a disk except for the outline. Stated differently, the left and right circles specialise in high versus low spatial frequencies. You can observe that the flickering outline circle makes its whole low-contrast grey test disk disappear perceptually, while the blurred flickering disk does not. Thus the outline circle is a much more efficient adaptor than the blurred disk. The fact that adapting out the edges makes the whole disk disappear has interesting theoretical implications for brightness and darkness perception: the neural mechanisms that signal light–dark edges are coding the surface brightness of the whole disk with respect to the background. Similar phenomena have been reported for the filling-in of coloured borders of afterimages (Hamburger, Geremek, & Spillmann, 2012).

Spatial congruency of contour adaptation and filling-in

How spatially selective are these aftereffects of contour adaptation? The adapting circles do need to be congruent with the edges of the test disks. Movie 4 shows three grey test disks, all of the same size. Superimposed on these are three adapting flickering outline circles, of three different sizes. One circle is larger than its test disk; one is smaller: and the top circle, as in the English fairytale of Goldilocks and the Three Bears, is just right. Running the movie will show that only the top test disk disappears from view. This is because it is spatially congruent with its adapting circle. The ill-fitting adapting circles produce no aftereffects. In particular, the adapting circle, which is larger than its test disk, does not reduce the visibility of items that lie within the circle without touching its borders. It does, however, appear to have an effect on the perceived size of the disks. The lower left disk appears smaller after contour adaptation to the oversized ring, where the lower right disk appears to be larger following adaptation to the undersized ring.

These are probably figural aftereffects (Köhler & Wallach, 1944). Since only the disk with the spatially superimposed flickering ring disappears, we conclude that contour erasure occurs only for disks that are spatially congruent with the flickering adapting contours.

Next consider the case of Movie 5, in which the test disk is a completely round disk but the adapting outline is only a semicircle, congruent with only the left-hand half of the test disk. Following adaptation to the flickering half-circle, the test disk shows a pronounced spatial gradient, with the disk looking like an intact light grey at its unadapted right-hand edge, graded spatially to a mid-grey at its adapted left-hand edge that approximately matches the mid-grey of the surround, giving the disk the appearance of a half-moon[1]. Again this demonstration points to the importance of edge polarity in signaling the lightness of the whole disk, but how the spatial gradient is computed is not yet known.

In Movie 6, a vertical midline divides a grey test disk into two halves, light on the left and dark on the right. The disk is initially surrounded by a black background. Adaptation to a flickering line that lies along the vertical midline gradually erases the border between the light and dark halves, and following adaptation the disk looks uniformly grey. But on the next display, everything is kept the same except that now the disk's surround is set to mid-grey. Crucially, this mid-grey is slightly darker than the left half of the disk and slightly lighter than the right half. This confronts the visual system with a paradox. The left and right halves of the disk look lighter and darker than the reference mid-grey of the surround where they meet along the curved perimeter of the disk, yet the two halves of the adapted disk, where they met along the straight midline, until now looked just the same! The visual system solves this dilemma by interpreting the left half as lighter than the right half (which it really is)—but the midline border between the two halves, although physically sharp, looks decidedly blurred. As the background switches between black and grey on alternate test periods, the test disk appears to be alternately homogeneous and bipartite, even though it never changes physically. So brightness edges along the rather distant periphery of the disk radically affect the perceived brightness along the vertical midline. This shows that long-range interactions (Das & Gilbert, 1995; Spillmann & Werner, 1996) are involved when the visual system computes the perceived brightness of different areas.

Anstis (2013) showed that a flickering adapting contour could push a spatial step of luminance below threshold only if the step was rather small. Steps of higher contrast would be approximately halved in subjective contrast. However, Movie 7 shows that contour erasure can negate an arbitrarily large luminance difference, provided this is broken down into a staircase of sufficiently small spatial steps. Each step can then be erased by adapting it out separately with its own flickering contour. Movie 7 contains two stepped pyramids of luminance, comprising four concentric dark squares and four concentric light squares, respectively. The lightest and darkest squares (the smallest squares) are separated by nine spatial steps of luminance. Each step is small enough to be pushed below threshold by adaptation to a flickering contour. So following adaptation to congruent flickering square outlines, the whole set of eight filled squares rapidly fades down into two very blurred blobs, one dark and one light, and after about 30s of strict fixation they virtually disappear, so now the lightest and darkest squares look the same brightness as each other. Exact copies of the lightest and darkest squares are shown above the display for purposes of comparison. These remain visible and highly salient, whereas the identical squares that are embedded in the pyramids have faded out to match the mid-grey surround.

Pop-out and contour erasure

In visual search, a sufficiently visible target can “pop out” pre-attentively from a set of background distractors. Movie 8 illustrates how contour erasure can convert the percept of test annuli into that of apparently solid light or dark disks, and, in turn, these perceptually homogenous disks themselves can lead to pop-out where the original annuli do not. The columns in Movie 8 show light and dark test annuli. The rows show small and large adapting flickering circles, which are adjusted in size so that they just fit the inner and outer edges of the test annuli, respectively. Consider the two light grey annuli in the left hand column. Erasing the inner edge of the upper annulus causes its light grey to fill-in the hole of the “donut,” making it look like a large, light disk. Erasing the outer edge of the lower annulus makes the mid-grey surround fill in the annulus from the outside. The annulus itself disappears from view. However, the hole in the donut is still darker than the still-visible inner edge of the light annulus, so that now this hole in the annulus looks like a small, dark disk. The opposite is true for the dark annuli in the right column. So now at will we can create subjective dark disks, by eroding the percept of either the inner edge of a dark annulus or the outer edge of a light annulus. Conversely, we can create subjective light disks, by eroding either the inner edge of a light annulus or the outer edge of a dark annulus. Thus, contour erasure allows us to generate filling-in, followed by pop-out, anywhere we wish on the retina.

How has the target become so salient among the distractors? In Movie 9, all but one of the flickering circles are mere decoys, to avoid giving unwanted informative spatial cues to the observer. They have the wrong sizes or positions to interact with any of the test annuli. But just one circle is positioned so that it just fits snugly on to the inner edge of one annulus. It adapts, and temporarily erases, the inner edge of that annulus. So now the pale grey of the annulus fills in the hole in the donut, producing a subjectively solid light disk. The fact that this experiment works suggests that filling-in precedes pop-out in the processing hierarchy of the visual system.

Movie 10 shows a second display in which a filled-in annulus pops out. Before adaptation, none of the low-contrast test annuli pop out. They just have random sizes and luminance polarities. None of the adapting flickering circles pops out either. But the correlation between the adapting circles and test rings means that after adaptation, one annulus looks like a light disk and pops out from the other annuli, which look like randomly sized dark disks.

Taken together, these results show that visual processes underlying filling-in occurs earlier than those related to pop-out. But it is also evident that filling-in is an active process that generates a visible representation of the disks and annuli, and is not simply a process of ignoring (faded, unwanted, or uninteresting) stimuli.

How many kinds of filling-in are there?

Ardon Lyon (personal communication to author S.A.) and Ramachandran (1992) have shown that filling-in of the natural blind spot has the same effects as the contour erasure that we showed in Movie 9. They noted that viewing a set of randomly arranged annuli, with the middle of one annulus landing on the natural blind spot, would “fill-in” the centre of that annulus so that it looked like a solid disk. Furthermore, this solid disk would “pop out” pre-attentively from the distracting annuli. Such observations are notoriously difficult to make because acuity is so poor in the peripheral retina near the natural blind spot. However, we suggest that the same filling-in process may be at work on their blind spot as in our contour erasure.

Durgin, Tripathy, and Levi (1995) tried to reconcile two different kinds of filling-in by claiming that filling-in of the blind spot is like amodal completion of objects hidden by one's thumb. This may well be so: but amodal completion cannot be the same as contour erasure, which, as we have seen, actively converts annuli into subjective disks that can pop out—a task that is quite beyond amodal completion. In addition, contour erasure can easily reduce luminance contrast but is totally ineffective with colours (Anstis, 2013). Amodal completion, on the other hand, shows similar effects for both luminance and colour.

Our results and those of Ramachandran (1992) suggest that the same kind of filling-in may take place for contour erasure and for the natural blind spot. According to Durgin et al (1995), filling-in of the natural blind spot may be the same process as amodal completion. But, as we have just argued, contour erasure and amodal completion seem to be very different processes.

These contradictory conclusions suggest that not all kinds of filling-in are equal. Philosophers (less so neuroscientists) often look for a canonical form of filling-in, which would be the same in all situations—illusory contours, amodal completion, Troxler fading, and the like. Together with earlier results cited above, our findings suggest that filling-in may not be a single, ubiquitous process. Instead, it may take on many different forms in different parts of the visual system.

To summarise, contour adaptation is a dynamic process involving flicker. We have shown elsewhere (Anstis, 2013) that it is a monocular process, since an adapting circle viewed by one eye has no effect on a test disk seen by the other eye. We also showed elsewhere (Anstis, 2013) that it is achromatic, since it can erase grey but not coloured test disks. We conclude that this dynamic, monocular, achromatic process of contour adaptation probably happens early in the magnocellular pathway (Callaway, 2005). Thus, contour adaptation would desensitise magnocellular-pathway neurons, which otherwise would provide a fast feed-forward projection to the dorsal visual stream. According to Bullier (2001), this magnocellular thalamo-cortical projection informs dorsal stream neurons as to what is displayed. This information is then projected back into the visual cortex to guide parvo- and koniocellular processes that support form and colour perception. Following contour adaptation, this fast feed-forward mechanism appears to be suppressed. Lacking information about these edges the ventral stream fills in brightness information, either from the next salient edge or from the background.

We have shown that contour adaptation can speed up the onset of filling-in when a peripheral object gradually fades out. As described above, filling-in is the perceptual phenomenon in which the observer perceives something in the absence of visual input (Pessoa & de Weerd, 2003; Ramachandran, 1993). Such a lack of input can also be caused by the natural blind spot, the location in the visual field that contains no receptors due to the papilla and head of optic nerve. Stimulation around the blind spot can lead to partial or complete perceptual filling-in at that location in the visual field (Spillmann et al., 2006).

Perceptual filling-in: An active or passive process?

There is still no agreement in the literature as to whether filling-in is an active or passive process. These two views are related to the way visual information is represented and stored (Churchland & Ramachandran, 1994; Pessoa, Thompson, & Noe, 1998; Weil & Rees, 2011). The symbolic or propositional view states that individual components in a visual scene (e.g., cars, bicycles, and pedestrians in a street scene) are represented in a high-level fashion as objects (Dennett, 1991, 1992, 1998). Objects would have stored representations containing the features that define the object, and it is these representations that would subserve recognition. Such symbolic representations would require no filling-in process since there is no retinotopic representation underlying this form of coding and storage. The opposing view is that filling-in is an active process that uses a retinotopic representation of the scene. Missing information would be, according to this view, actively completed based on information in the surround. Monocular stimulation of the area in and around the blind spot leads to activation of neurons in V1 with large receptive fields (Komatsu, Kinoshita, & Murakami, 2002), suggesting some sort of active completion process at the retinotopic locus of the blind spot, and of real or artificial scotomata. These different views cannot be easily proven or disproven based merely on the subjective phenomena associated with filling-in.

An artificial scotoma from contour erasure

Now we can connect together the various threads of our argument. We have shown how contour erasure can accelerate filling-in during Troxler fading. We have also discussed the filling-in of retinal blind spots, which is largely stimulus-independent. We shall now ask whether contour erasure can produce artificial scotomata that mimic the behaviour of the natural blind spot.

Movie 11 shows a field of low-contrast vertical stripes. These are interrupted by two horizontal bars, one light grey, the other medium grey. These will be our artificial blind spots. If one glances at the letters x or y on the right, the bars are clearly visible in peripheral vision. But now fixate x strictly while the movie flickers thin outlines around the edges of the two bars. Each time the flicker stops the bars are slightly less visible and their borders, where they interrupt the stripes, are less distinct. Eventually the bars cannot be clearly made out at all. If you switch your gaze to y, the horizontal bars immediately reappear, showing that adaptation has taken place. Now ask yourself whether the vertical stripes are still occluded by the two bars, or whether they appear to fill-into the two areas occupied by the bars, making them looked striped. The question cannot be unambiguously answered, since peripheral acuity is just too poor to resolve the texture of the background at that location in the visual field. But before adaptation the spatial interruptions to the vertical stripes were visible, and after adaptation they are not. The inability to see the gaps, owing to low acuity, is a good subjective description of how the blind spot looks. We suggest that contour erasure makes very satisfactory artificial scotomata, the perception of which resembles our perception of the natural blind spot.

Some conclusions

Our various demonstrations show how contour adaptation can enhance the filling-in process by speeding up the otherwise slow process of Troxler fading. We have shown that these forms of filling-in can also occur in central vision. Artificial scotomata also fill in if contour adaptation is applied to the edges of these patches. It appears that filling-in is a multifaceted process that is used by the brain to interpolate between sensory information contained in complex stimulus configurations where parts of object may be missing. The observation that contour erasure works best for low contrast luminance edges suggests that this form of adaptation is directly suppressing the magnocellular pathway. If this assumption were true, then even low contrast adaptation should induce contour erasure, which is an idea that would require further experiments. In the absence of contour information the visual system “fills in” the missing parts with the background. This form of filling-in has been shown to be stimulus dependent: contour adaptation is only effective when the adapting stimulus spatially aligns with the object contour. By means of filling-in missing information is completed to optimise behaviour for the visually guided task at hand.

Copyrighted material

All movies were created by author S.A., using Adobe Director on a Mac laptop computer. Permission for these is given for reuse under a Creative Commons noncommercial, non-derivative license.