Microsaccades, Drifts, Hopf Bundle and Neurogeometry.

The first part of the paper contains a short review of the image processing in early vision is static, when the eyes and the stimulus are stable, and in dynamics, when the eyes participate in fixation eye movements. In the second part, we give an interpretation of Donders' and Listing's law in terms of the Hopf fibration of the 3-sphere over the 2-sphere. In particular, it is shown that the configuration space of the eye ball (when the head is fixed) is the 2-dimensional hemisphere SL+, called Listing hemisphere, and saccades are described as geodesic segments of SL+ with respect to the standard round metric. We study fixation eye movements (drift and microsaccades) in terms of this model and discuss the role of fixation eye movements in vision. A model of fixation eye movements is proposed that gives an explanation of presaccadic shift of receptive fields.

1. Introduction

The main task of the visual system is processing and decoding visual information, recorded by the retinal photoreceptors, and constructing a model of the external world. The photoreceptors convert the light signal into electric signals which are sent to retinal ganglion cells and then by a conformal retinotopic mapping to LGN, then to the V1 cortex, V2 cortex etc. The visual system has a hierarchical structure and consists of many subsystems connected by direct and feedback.

The neurogeometry of vision deals with the construction of continuous models of various visual subsystems in terms of differential geometry and differential equations.

There are three level of the models of the visual subsystems:

Static, without taking into account time, i.e., under assumption that the eye and the perceived object (stimulus) are stationary;

Semi-dynamic, when the stimulus is stationary and the eye is moving;

Dynamic, when both the eye and the stimulus are in motion.

Over the past two decades, great progress has been made in understanding the functional architecture of early vision in static and constructing the neurogeometric models of early vision systems (primary visual cortex V1, hypercolumns), see [1,2,3,4,5,6,7,8,9]. The models are based mostly on the results obtained in experiments on anesthetized animals.

In natural vision, the eye always participates in different movements. According to the classical experiments of A. Yarbus [10], the compensation of the eye movement leads to the loss of vision of stationary objects in 2–3 s. Moving objects remain visible, albeit poorly. Later experiments show that the most important phase of the fixation eye movements is the drift. Compensation of microsaccades does not lead to loss of vision.

It was remarked by M. Rucci, E. Ahissar and D. Burr [11].

“As there are no stationary retinal signals during natural vision, motion processing is the fundamental, basic operating mode of human vision.”

They also note that due to this there is no big difference between semi-dynamic and dynamic vision.

In the first part of the paper, we will briefly discuss the main results concerning the static vision, which are the base points to deal with dynamic one. Currently, there are some advances in the study of the dynamic case, [12,13,14,15] although the description of the visual processes becomes significantly more complicated and new phenomena arise, such as saccade remapping [16,17], shift of the receptive field, compression of the space and time during saccades [18,19]. The main difference between static and dynamic vision is the following. As it is generally accepted, in static vision all information comes from the activation of retinal photoreceptors. In dynamic vision, the process of perception is determined by the interaction of the visual information from the retina and the dynamical information about eye movements, coded in the ocular motor system.

Even when the gaze is focused on a stationary point, it participates in different type of movements, called fixational eye movements (FEM). For a long time, most neurophysiologists did not pay serious attention to FEM. The situation has changed in the last two decades, see [20]. Both experimental and theoretical works have appeared that substantiate the important role of FEM in vision. Primarily the works by M. Rucci and their coauthors [11,21,22,23,24,25] contain detailed and critical analysis on many experimental results about different types of FEM—tremor, drift and microsaccades, and new ideas about their role in vision.

In the dynamic case, the eye movements are controlled by ocular motor system and a copy of motor command, called corollary discharge or efference copy, is sent from superior colliculus through MD thalamus to frontal cortex. It plays an important role in visual stability, i.e., the compensation of the shift of retinal stimuli and perception stable object as stable, see [26,27,28] for results and discussions on the problem of visual stability.

A deeper understanding of the mechanism of FEM depends on further progress in description of image processing in retina, visual cortex and in ocular motor control of eyes movements.

Fixational eye movements are stochastic in nature. There were proposed various stochastic models of FEM as a random walk, see [29,30,31]. We especially note the works [32,33]. In the most works, FEM are modeled by a random walk on the plane or on a lattice in the plane. However, the information about eye rotation, which is contained in corollary discharge, treats the eye as a ball and not as a plane. For more realistic model of FEM, which will be consistent with corollary discharge information, we need more sophisticated model of saccades and drift, where such movements are considered as rotations of the eye ball. Due to this, it is important to describe the configuration space of the eye.

A priori the configuration space of eye ball , rotating around its center O, is the orthogonal group (which can be thought as the 3-sphere with identified antipodal points, ).

A big surprise even for the great physicist and physiologist H. von Helmholtz was the law, discovered in the middle of the 19th century by F.C. Donders and supplemented by J.B. Listing. It states that, when the head is fixed, the real configuration space of eye positions is two-dimensional. More precisely, the direction of the gaze uniquely determines the position of the eye, described by the retinotopic orthonormal frame . From the point of view of the modern control theory, such a constraint is quite reasonable. The difference between the motion control on the 3-sphere and on a surface is similar to the difference in piloting a plane and driving a car.

One of the main results of the work consists of interpreting Listing’s law in terms of a section (which we call Listing’s section) of the Hopf bundles over a punctured sphere where i is the direction to the nodal point of the eye sphere (in the standard position) and is the direction to the center of the fovea. Listing’s section is an open 2-dimensional hemisphere of a 3-dimensional sphere , identified with the group of unit quaternions. This simple description of Listing’s law provides a way for construction of more realistic stochastic models of FEM and oculomotor system that control eyes movements. For example, denote by the eye sphere in the standard position. Let be two points and the corresponding points of Listing’s hemisphere . Then the saccade with the initial gaze direction A and the final gaze direction B is the segment of the unique geodesic (the great semicircle) of Listing hemisphere (with the standard metric) through points . The corresponding evolution of the gaze is the segment of the circle (with the deleted point ), which is the section of the punctured sphere by the plane, generated by the points . So the space of saccades is the direct product of two copies of Listing’s hemisphere.

We propose a deterministic model of fixation eye movements (drift and microsdaccades) in terms of Listing’s hemisphere. The microsaccades are considered as a mechanism of remapping the visual information, which depends of the choice of the salient point as the next gaze target. It gives a simple description of the presaccadic shift of receptive fields. We use this model to define a distance between point stimuli . Then we shortly recall the basic fact of diffusion geometry, initiated by R.R. Coifman and S. Lafon [34,35], and discuss the extension of the model to the stochastic case, when the drift is considered as a random walk on Listing’s hemisphere, in the framework of diffusion geometry.

2. Information Processing in Early Vision in Static and Functional Structure of Retina and Primary Visual Cortex

In static, visual information is coded in firing of retinal photoreceptors, cones and rods. In the first approximation, the input function of the retina may be considered as the function on retina, which describes the density of energy of light, recorded by photoreceptors. The visual information is primary processing in retina and it sent to primary visual cortex V1 and then to V2, V3 and other visual systems for further processing and decoding. The visual information is coded in visual neurons which are working as filters that is functionals on the space of input function, which value depends only on the restriction of the input function to a small domain of the retina, called receptive filed (RF). The linear neurons are working as linear filters, i.e., the linear functionals, described as the integral of the input function with some weight , called the receptive profile. In reality, most visual neurons have spatiotemporal character, that is their reply depend also on time integration of the input function.

2.1. The Eye as an Optical Device and Input Function

The eye is a transparent ball together with a lens L which focuses light rays to the retina R, see Figure 1. The retina occupies a big part of the boundary sphere of the eye ball. The lens is formed by the cornea and the eye crystal. We will assume that the optical center of the lens or nodal point N belongs to the eye sphere .

Figure 1

The Human Eye. Adapted from Wikipedia.

A beam of light emitted from a point A of a surface and passing through the nodal point N is not refracted and falls to the point of the intersection of the retina R with the ray . A beam from the point A which passes through any other point of the lens is focused and come to the same point . So we get a central projection of the surface given by the map where is the second point of intersection of the ray with the retina R, see Figure 2. The central projection generically is a local diffeomorphism.

Figure 2

Central projection.

Note that if is the frontal plane (orthogonal to the line of sight) which is far enough away compared to the size of the eyeball, then the central projection is approximately a conformal map.

The (density of) energy of light coming from a point of the surface to the point of retina is approximately proportional to the (density of) energy of light , emitted from the point A. So the input function of the retina (where is the set of non negative numbers), contains information about the density of energy of light, emitted from the surface . The aim of the static monochromatic vision is to extract from the input function information about geometry of the surface. We will not speak about other characteristics of the recorded light, for example, the spectral properties, which are responsible for color vision. It seems that the polarisation plays no role in human vision.

It was discovered by D. Hubel and T. Wiesel, that the most important characteristic of the detected stimulus are the contours, i.e., the level sets of the input function with large gradient. J. Petitot [5] gave a precise geometrical formulation of this claim as a statement that simple neurons of V1 cortex detect infinitesimal contours, i.e., 1-jets of contours, considered as non parametrized curves. One of the main task of the higher order visual subsystems is to integrate such infinitesimal contours to global ones.

2.2. Retina

2.2.1. Anatomy of Retina

Retina consists of 5 layers. In human there are in approx. 80 different types of cells. The bottom layer consists of receptors, photoelements which transform light energy into electric signals, see Figure 3. They measure the input function and send information to ganglion cells. In fovea, one cone is connected with 1 ganglion. In periphery, one rode is connected with – ganglions. There are 1 million of ganglions and 125–150 millions of receptors.

Figure 3

Anatomy of retina.

2.2.2. Ganglion Cells as Marr Filters

It was discovered by S. Kuffler that the receptive field of a typical ganglion cell is rotationally invariant (isotropic) and contains central disc and surround ring. It is working as a linear filter with receptive profile which is ether positive in the central disc and negative in the ring or vice versa. In the first case, Kuffler called it ON-cell and in the second one OFF-cell, see Figure 4. D. Marr showed that the filter with Laplacian of the Gauss function as the receptive profile gives a good model of Kuffler cell and proved that image processing by a system of such filters turns a picture into a graphic image, see Figure 5. The purpose of the information processing in retina is to regularize the input function, eliminate the small artifacts of the retina image and to highlight the contours, which are the main objects of perception in early vision.

Figure 4

On and Off Kuffler cells.

Figure 5

Action of Marr filter.

2.2.3. Information Processing in Retina. Two Pathways from Receptors to Ganglion Cells

There are two pathways from receptors to ganglion cells: Direct path: receptor–bipolar–ganglion activates the center of ganglion cells, which work as a linear filter. Antagonistic surround is activated by (linear) negative feed back from horizontal cells via indirect path: receptor–horizontal cell–(amacrine)–bipolar–ganglion. A nonlinear rectifying mechanism (associated with contrast gain control) is related with amacrine cells.

For sufficiently small contrast, ganglion P-cells is working as linear Marr filter. M-cells, responsible for perception of moving objects, are working as essentially non-linear filters. Response depends on stimulus contrast and temporal frequency [36].

2.2.4. Fovea

The fovea was discovered by Leonardo da Vinci. It is a small pit in the retina which contains mostly cones, see Figure 6. The cental part of the fovea, called the foveola, has a diameter 0.35 mm ∼ 1. It consists only from cones packed with maximum density. The fovea occupies 1% of retina, but is projected onto almost 50% of area of the visual cortex. When we fix gaze on a point A, the image of this point on retina moved due to the fixation eye movements (FEM), but remains inside fovea.

Figure 6

Eye, retina and fovea. Adapted from Wikipedia.

2.2.5. Inhomogeneity of the Retina and Magnification. Physiological Metric in Retina

The physical metric in retina (considered as a sphere) is standard metric of the sphere. (The distance is described in mm or in degrees). 1 mm = 3.5∼ 6 cm at a distance of 1.5 m, 1∼ 0.3 mm ∼ 2.5 cm at a distance 135 cm. Apparent diameter of Moon and Sun is 0.5 = 0.15 mm = 150 . Receptive field of neurons of V1 cortex projected to fovea has diameter 0.25–0.7 and the area 0.07 × 0.15∼ 0.12 mm2. The receptive field of neurons projected onto the periphery of the retina has a diameter up to 8, on average this is 30 times more then in fovea and the RF here contains thousands of rods.

Magnification = distance between two points of V1 cortex which corresponds to 1 mm distance in retina. The cortical magnification in the fovea 1mm ∼ (1/6) = 0.05 mm is 20 times. The cortical magnification in the periphery 1 mm = 6 = 1.8 mm is 0.55 times.

Hubel [37] remarked that the structure of retina is very inhomogeneous. He supposed that it is one of the reason, why the information processing in retina is very limited. On the other hand, he emphasized the amazing homogeneity of the cortex V1. It is expressed in the fact that a shift in 2 mm at any point of the cortex corresponds to shift on diameter of the corresponding receptive field in retina. We define the physiological metric in the retina, where the length of a curve is given by the number of receptive fields of neurons along this curve. This metric in the retina is proportional to the physical metric in the cortex. In particular, the diameter of fovea 1 corresponds to 6 mm in V1 cortex. (Hubel).

We will discuss a possible application of this metric to choosing of appropriate diffusion kernel for stochastic model of the drift.

2.2.6. Conformal Retinotopic Map from the Retina to the LGN (Lateral Geniculate Nucleus) and to the Visual Cortex V1

After image processing in the retina, the input function is encoded by the firings of ganglion cells. Then it is sent to the LGN and the V1 cortex by the conformal retinotopic mapping, see [38,39]. There are three main pathways from the retina to the V1 cortex: the P-pathway, which is responsible for the perception of stable objects, the M-pathway, which is important for the perception of moving objects, and the K-pathway, important for the color vision. In static models, only the P-pathway is considered, but for dynamic model the M-pathway is also very important. M-pathway is more complicated then P-pathway, since M-neurons are not linear, see [36].

Let be the standard coordinates of the tangent plane of the eye sphere at the center F of the fovea. We will consider these coordinates as conformal coordinates on the eye sphere due to the stereographic map with center at the nodal point N. It is convenient also to introduce the complex coordinate and the associated polar coordinates where . In physiology, the coordinate r (the geodesic distance to F) is called the eccentricity and the angular coordinate. In appropriate complex coordinate in LGN and the V1 cortex, the retinotopic map is described by a meromorphic function of the form

The module describes the local magnification at a point z of the retina (see E. Schwartz [38]).

2.3. Functional Architecture of the Primary Visual Cortex: Columns, Pinweels, Simple and Complex Cells, Hypercolumns

The primary visual cortex V1 is a surface of depth 1.5–2 mm which consists of 6 layers. Each layer consists of columns of cells which have approximately the same receptive field. Hubel and Wiesel proposed a classification of V1 cells into simple and complex cells. Simple cells act as Gabor filters (defined by the receptive profile, that is the Gauss function modulated by sin or cos). The most important property of the Gabor filter is that it detects orientation of the contour, crossing its receptive field. There are several versions of the Gabor filters, which measure at the same time other parameters of the stimuli, for example, spatial frequency, phase etc. This means that the Gabor filter is activated only if these parameters take (with some precision) certain values. All simple cells from a regular column act as Gabor filters with almost the same center and they detect almost the same orientation of the contour.

A singular column called(pinwheel) contains simple cells which measure any possible orientation of the contour.

One of the purposes of the eyes movement is to produce the shift of the retinal stimulus such that the contour intersects pinwheels and is detected by their neurons.

Hypercolumns of V1 Cortex

Hubel and Wiesel proposed a deep and very productive notion of hypercolumns in V1 cortex. Given a system of local parameters (e.g., orientation, ocular dominance, spatial frequency, temporal frequency, phase etc.). A lhypercolumn (or, module) is defined as a minimal collection of (regular) columns, containing simple cells which measure any possible value of these parameters and which is sufficient to detect the local structure of the stimulus. Applying this notion to orientation and ocular dominance, they proposed a famous ice cube model of V1 cortex. Now this notion is applied also for the V2 cortex. Usually, the area of hypercolumns is 1–2 mm2.

3. Information Processing in Dynamics

3.1. The Eye as a Rotating Rigid Ball

From a mechanical point of view, the eye is a rigid ball which can rotate around its center O. The retina occupies only part of the eye sphere but for simplicity, we identify it with the whole eye sphere . We will assume that the eye nodal point N (or optical center) belongs to the eye sphere and the opposite point F of the sphere at the center of the fovea.

For a fixed position of the head, there is a standard initial position of the eye sphere, described by the canonical orthonormal frame , which determines the standard coordinates of the Euclidean space with center O. We will consider these coordinates as the spatiotopic (or the world-centered) coordinates and at the same time as the head-centered coordinates. Here i indicates the standard frontal direction of the gaze, j is the lateral direction from left to right which is orthogonal to i and k is the vertical direction up.

Any other position of the eye is described by an orthogonal transformation which maps the frame into another frame where is the new direction of the gaze. Recall that any movement is a rotation about some axis through some angle .

Definition of a Straight Line by Helmholz

If the frontal plane (orthogonal to the line of sight) is far enough away compared to the size of the eyeball, then the central projection can be considered as a conformal map.

H. von Helmholtz gave the following physiological definition of a straight line:

A straight line is a curve , which is characterized by the following property: when the gaze moves along the curve ℓ, the retinal image of ℓ does not change.

Indeed, given a straight line , let us denote by the plane through ℓ and the center O of the eye ball and by n its normal vector. Assume that for the standard position of the eye, the gaze is concentrated on the point , i.e., . The retina image of ℓ belongs to the intersection between and the standard position of the eye sphere. When the gaze moves along , the eye rotates with the axis n. Since at each moment t the new position of the eye sphere is , the retina image remains the same for all t.

We will see that saccades correspond to such movements along the straight lines.

3.2. Saccades and Fixation Eye Movements: Tremor, Drift and Macrosaccades

3.2.1. Saccades

Eyes participate in different types of movements [40]. We are interested only in saccades and fixation eye movements (FEMs) when the gaze is “fixed” [41].

Saccades are one of the fastest movements produced by the human body. The angular speed of the eye during a saccade reaches up to 700/s in humans for great saccades ( 25 of visual angle). Saccades to an unexpected stimulus normally take about 200 milliseconds (ms) to initiate, and then last from about 20–200 ms, depending on their amplitude. For amplitudes up to 15 or 20, the velocity of a saccade linearly depends on the amplitude. Head-fixed saccades can have amplitudes of up to 90, but in normal conditions saccades are far smaller, and any shift of gaze larger than about 20 is accompanied by a head movement. Most researchers define microsaccades as a small saccades, i.e., saccades with a small amplitude, such that the during a microsaccade the retina image of the point of fixation belongs to the fovea and even foveola, [23]. However in [42], the authors distinguish the small goal-directed voluntary eye movements from microsaccades. They showed that properties of microsaccades are correlated with precursory drift motion, while amplitudes of goal-directed saccades do not dependent on previous drift epochs. Microsaccades represent one of the three types of fixation eye movements.

3.2.2. Fixation Eye Movements (FEM)

The fixation eye movements are responsible for detection of local image structures and consist of tremor, drifts and microsaccades.

Tremor is an aperiodic, wave-like motion of the eyes of high frequency but very small amplitude. We hypothesize that the role of tremor is to increase the width of the contour on the retina, so that it is perceived by several rows of photoreceptors. This will allow also to estimate the value of the gradient along the contour. A detailed study of tremor and its influence on the retina images was made in [43], see Figure 7.

Figure 7

(A) An example of an eye trace taken from an AOSLO movie. A microsaccade (magenta background) is clearly distinguishable from the ocular drift (blue background). Gray vertical gridlines demarcate frame boundaries from the AOSLO movie. Each frame is acquired over 33 ms as indicated by the scale bar. (B) An example of an image/frame from an AOSLO movie. The cone mosaic can be resolved even at the fovea. (C) An example of the AOSLO raster with a green letter E as it would appear to the subject. The small discontinuities in the eye trace at the boundaries between frames 478–479 and 480–481 are likely the result of tracking errors that occur at the edges of the frame. They are infrequent and an example is included here for full disclosure. Errors like this contribute to the peaks in the amplitude spectrum at the frame rate and higher harmonics. All original eye motion traces are available for download. Adapted from [43].

Drifts occur simultaneously with tremor and are slow motions of eyes, in which the image of the fixation point for each eye remains within the fovea. Drift is an involuntary stochastic process. However, the stochastic characteristics of the drift may depend on the local structure of the stimulus. Drifts occur between the fast, jerk-like, linear microsaccades. The main property of the FEMs is that during FEM the retina image of the point of fixation remains in the fovea and even the foveola [23]. The following Table 1 indicates the main characteristics of the FEM.

Table 1

Characteristics of fixation eye movements (Adapted from [44]) with refined data from [23,43] and Wikipedia.

	Amplitude	Duration	Frequency	Speed
Tremor	11-60 arcsec	-	50–100 Hz	Max 20 arcmin/s
Drift	1.5–4 arcmin	0.2–0.8 s	95–97% of time	50 arcmin/s
Micsac	1–30 arcmin	0.01–0.02 s	0.1–5 Hz	40–220 deg/s

Per 1 s tremor moves on 1–1.5 diameters of the fovea cone, drift moves on 10–15 diameters, microsaccads moves on 15–300 diameters, see Figure 8.

Figure 8

Microsaccades and Ocular Drifts. Adapted from Wikipedia https://commons.wikimedia.org/wiki, CC-BY.

3.2.3. The Role of Fixation Eye Movements

The papers by M. Rucci and his collaborators [21,22,23,24,25] contain very useful information about different characteristics of fixation eye movements and a detailed analysis of the role of FEM in vision. In a survey [23], the authors critically revised three main hypotheses about the role of microsaccades (MS) in vision:

the maintenance of accurate fixation;

the prevention of image fading due to fast adaptation of retinal photoreceptors;

vision of fine spatial detail.

They gave many very convincing arguments in support of the hypotheses (1) and (3) and 10 arguments against the hypothesis (2). We add here only one additional argument against (2). Support that before the MS a retinal photoreceptor in fovea received light signal from stimulus A. After the MS, it will receive a signal from another stimulus B, which can be even brighter. Why this will prevent the photoreceptor from adaptation?

We mention also one geometric argument why FEM are useful for vision. In monocular vision, provided that the position of eye is fixed, the retina gets information only from the 2-dimensional Lagrangian submanifold of the 4-dimensional space of lines consisting of lines incident to the eye nodal point N. The space of lines is naturally identified with the (co)tangent bundle of the unit sphere. It is a symmetric pseudo-Kähler manifold of neutral signature . When the eye moves with a small amplitude, the retina gets information from a neighborhood of this 2-surface in the 4-manifold .

M. Poletti and M. Rucci [23] gave evidence that during natural vision the microsaccades can not be regarded as a random process. Their characteristics depend on the scene. Moreover, the ability to control microsaccades plays an important role in performing different fine work, like reading, threading a needle, playing some sports (e.g., table tennis), etc. However, it seems plausible that in some cases MS can be considered as random processes. For example, when contemplating the sea, the blue sky and similar homogeneous scenes, it can be assumed that microsaccades make a random walk. Perhaps the pleasure that a person feels when contemplating such scenes is due to the fact that the eyes get rid of the difficult work of finding new targets for microsaccades.

3.2.4. Remapping and Shift of the Receptive Fields (RFs)

In a seminal paper, J.-R. Duhamel, C.L. Colby and M.E. Goldberg [45] described the shift of receptive field of many neurons in macaque lateral intraparietal area (LIP), which shows that the visual neurons of these systems get information about the retina images of their future receptive fields. This is one of the most remarkable discoveries of neurophysiology of vision at the end of the 20th century.

Assume that the RF of a neuron before a saccade covers the retina image of a point A and after the saccade the retina image of another point B. Then 100 ms before the saccade, the neuron detects stimuli at the locations . This process constitutes a remapping of the stimulus from the retina coordinates with the initial fixation point A to those of the future fixation point B. The process is governed by a copy of the motor command (corollary discharge).

For a long time, it had been assumed that the presaccadic shift of the receptive field (RF) from to is an anticipation of the retinal consequences of the saccade, which randomly changes the gaze direction and the RF of the neurons to . Since any point of the retina can be a new position of the receptive field, this means that the information about the visual stimulus at the point can be transmitted to neurons with receptive field at the point . This seems very doubtful, since the number of neurons pairs is too big. The solution was proposed by M. Zirnsac and T. Moore [46]. They conjectured that the presaccadic shift of RF is a part of a process of remapping and reflects the selection of the targets for the saccades. Some local area of a higher center of the visual system has information about visual stimulus concentrated at and about other points of the retina. It uses this information to choose a new saccadic target . Just before the saccade, it sends the information about the visual stimulus at the retinal point to neurons with presaccadic receptive field at . After saccades, the real RF of these neurons cover the retina stimulus . Then the visual system use information from these neurons to corrects the presaccadic information. In the last section, we propose a mechanism of realization of such presaccadic remapping.

3.2.5. Oculomotor System, Corollary Discharge and Stability Problem

In dynamic, the retinal photoreceptors are not the only source of visual information. The important part of information about eyes movements is coded in oculomotor system. A copy of motor commands, which control eyes movements, the corollary discharge (CD) or efference copy, is sent from the sensorimotor region through the MD thalamus to the frontal cortex. The mechanism of interaction of CD information with information from retinal receptors processed in the visual cortex is not well known. It is very important for solution of the stability problem, i.e., explanation of the compensation mechanism for shift of stimuli on the retina caused by eye movements, such that a stable stimuli will be perceived as stable, see [26,27,28]. Clearly, it must be very strong synchronization between corollary discharge and the presentation of the retina input function in visual cortex.

The stability problem was first formulated in the eleventh century by the Persian scholar Abu’Ali al-Hasan ibn al-Hasan ibn al-Haytham (latinized, Alhazen) and was discussed by Descartes, Helmholtz, Mach, Sherrington and many others scientists.

3.3. The Geometry of the Quaternions

Now we recall the basic facts about quaternions and the Hopf bundle, which are we need for reformulation of Donders’ and Listing’s laws in terms of Listing’s section of the Hopf bundle.

Let be the algebra of quaternions with the unit 1, where the space of the imaginary quaternions is the standard Euclidean vector space with the orthonormal basis and the product of two elements from E is the sum of their scalar product and the cross-product:

The group of unit quaternions are naturally identified with the three dimensional sphere and its Lie algebra is the algebra of imaginary quaternions with the cross-product as the Lie bracket.

Denote by the (exact) left representation and by the (exact) right representation, which commutes with the left representation. They define the representation with the kernel .

The representation is called the adjoint representation. It has the kernel , acts trivially on the real line and defines the isomorphism which shows that the group is the universal covering of the orthogonal group . The standard scalar product in , where for the is the conjugated quaternion, induces the standard Riemannian metric of the unit 3-sphere , which is invariant with respect to the (transitive) actions of the group . The group preserves the points (which will be considered as poles of ) and acts transitively on the equator , which is the standard Euclidean unite sphere of the Euclidean space . The geodesics of are the great circles (the intersections of with 2-subspaces of ).

The following simple facts are important for us and we state them as

Any point where

Points of

Any orbit

The 1-parameter subgroup

More generally, let be a geodesic of

In other words, the orbit

The adjoint image of the one-parameter subgroup is an one-parameter subgroup of , which preserves the vector , hence the group of rotation w.r.t. v. To calculate the angle of the rotation, we apply to a vector , which anticommutes with v, as follows

This shows that .

follows from and the following calculation

□

3.3.2. The Hopf Bundle and Listing’s Sphere

The Hopf bundle is defined as the natural projection of to the -orbit of the point i.

The base sphere is called the Euclidean 2-sphere. The points will be considered the north and south poles of . We denote by the equator of .

The Hopf bundle is a non trivial bundle and has no global section. However, by removing just one point with the preimage from the base sphere , we will construct the canonical section of the bundle over the punctured sphere .

First of all, we define Listing’s sphere and Listing’s hemisphere, which play a central role in the geometry of saccades. The Listing’s sphere is intersection of the 3-sphere with the subspace spanned by vectors . In other words, it is the equator of the 3-sphere w.r.t. the poles , see Figure 9.

Figure 9

Listing’s sphere.

We consider the point (resp., −1) as north (resp. south) pole of Listing’s sphere and denote by (resp., ) the open north (resp., south) hemisphere and by (resp., ) the closed hemisphere. Note that the equator of Listing’s sphere coincides with the equator of the Euclidean sphere .

3.3.3. Geometry of Listing’s Hemisphere

We consider Listing’s sphere as the Riemannian sphere with the induced metric of curvature 1 equipped with the polar coordinates centered at the north pole . The geodesics of are big circles. Any point of belongs to the unique 1-parameter subgroup of .

Any point , different from , can be canonically represented as where is the polar radius (the geodesic distance to the pole (such that is the geographic latitude) and is the geographic longitude of the point a. The point is the geodesic projection of a to the equator, i.e., the closest to a point of the intersection of with the equator .

Note that the coordinate lines are big circles (meridians), in particular, is zero ("greenwich") meridian and the coordinate lines are parallels. The only geodesic parallel is the zero parallel, i.e., the equator .

The open Listing’s hemisphere is geodesic convex. This means that any two distinct points determine a unique (oriented) geodesic of the sphere and are joined by a unique geodesic segment .

Canonical Parametrization of Geodesics

Let be two distinct points and the oriented geodesic. Denote by p the first point of intersection of with the equator .

If then the geodesic is an 1-parameter subgroup and is its canonical parametrization.

If , the unique top point , with the maximal latitude has the form where is the geodesic projection of m to and , hence .

Then where and , is the canonical parametrization of the geodesic .

The intersection of the geodesic with the Listing hemisphere is called the Listing’s semicircle.

The restriction

It maps the poles

Any different from

The map

More generally, let

It is the orbit and the Hopf mapping χ maps it into the orbit of the 1-parameter group of rotations

The restriction of the map χ to the Listing hemisphere

(i)–(ii) follow from the remark that quaternions commute with i and the quaternions from anticommute with i. Hence for .

(iii) We calculate

(iv) follows from or from Lemma 2. □

3.3.5. Listing Section

According to the Theorem, the Hopf map defines a diffeomorphism

Since the preimage is the equator of Listing’s sphere , the inverse map where is a section of the principal bundle

We call the section s the Listing section.

3.4. The Physiological Interpretation: Donders’ and Listing’s Laws and Geometry of Saccades

We use the developed formalism to give an interpretation of Donders’ and Listing’s laws and to study the saccades and drifts.

We consider the Euclidean sphere as the model of the eye sphere, see Figure 10, (the boundary of the eye ball ) with the center at the origin 0. We assume that the head is fixed and the standard basis determines the standard initial position of the eye, where the first vector i (the gaze vector) indicates the standard frontal direction of the gaze, the second vector j gives the lateral direction from right to left and k is the vertical direction up.

Figure 10

The eye sphere.

The coordinates associated with the standard basis are the head-centered and spatiotopic (or world-centered) coordinates. A general position of the eye, which can rotate around the center 0 is determined by the orthonormal moving (retinatopic) frame , which determine the (moving) retina-centered coordinates .

The configuration space of the rotating sphere is identified with the orthogonal group , an orthogonal transformation R define the frame

It is more convenient to identify the configuration space with the group of unit quaternions, which is the universal cover of . The corresponding -covering is given by the adjoint representation

A unit quaternion gives rise the orthogonal transformation and the frame which defines the new position of the eye. We have to remember that opposite quaternions represent the same frame and the same eye position. Note that a direction of the gaze determines the position of the eye up to a rotation w.r.t. the axe . Such rotation is called the twist.

Donders’ law states that if the head is fixed, then there is no twist. More precisely, the position of the gaze determines the position of the eye, i.e., there is a (local) section of the Hopf bundle

In other words, the admissible configuration space of the eye is two-dimensional. Physiologists were very puzzled by this surprise. Even the great physiologist and physicist Hermann von Helmholtz doubted the justice of this law and recognized it only after their own experiments. However, from the point of view of the modern control theory, it is very natural and sensible. The complexity of motion control in 3-dimensional configuration space compared to control on the surface is similar to the difference between piloting a plane and driving a car.

Listing’s law specifies the section s. In our language, it can be stated as follows.

Listing’s law. The section of Donder’s law is the Listing’s section where , which is the inverse diffeomorphism to the restriction of the Hopf projection to Listing’s hemisphere.

In other words, a gaze direction determines the position of the eye as follows

Saccades

We define a saccade as a geodesic segment of the geodesic semicircle . Recall that the semicircle , (where p is the first point of the intersection of the oriented geodesic with the equator , is the top point of and q is the equatorial point of the meridian of the point m), has the natural parametrization where . We may chose the vector q, defined up to a sign, such that .

The image is the circle (without the point ), obtained by the rotating of the point with respect to the axe , or, in other words, it is the section of the punctured sphere by the plane with the normal vector , where . The segment is the gaze curve, the curve, which describes the evolution of the gaze during the saccade .

The natural question arises. If the gaze circle is not a meridian, it is not a geodesic of and the gaze curve is not the shortest curve of the sphere, joint A and B. Why the eye does not rotate such that the gaze curve is not the geodesic?

The answer is the following. If all gaze curves during saccades would be geodesics, then we get the twist and the configuration space of the eye becomes three-dimensional. Assume that the gaze curve of three consecutive saccades is a geodesic triangle which starts and finishes in the north pole . Since the sphere is a symmetric space, moreover, the space of constant curvature, the movements along a geodesic induce a parallel translation of tangent vectors. This implies that after saccadic movements along the triangle, the initial position of the eye will rotates w.r.t. the normal axe i on the angle which is proportional to the area of the triangle. Hence, a twist will appear.

Fortunately, since the retina image of the fixation point during FEM remain in the fovea with the center at , the gaze curve remains in a small neighborhood of the standard position i. In this case, the deviation of the gaze curve during MS from the geodesic will be very small. This is important for energy minimization, since during wakefulness, 2–3 saccades occur every second. Hence more than 100,000 saccades occur during the day.

Consider the stereographic projection of the sphere onto the tangent plane at the point . It is a conformal diffeomorphism, which maps any gaze circle onto a straight line and any gaze curve of a saccade onto an interval where is the point of the intersection of the tangent plane with the line and similar for . More precisely, where . The spherical n-gon , formed by gaze curves of saccades, maps into the n-gone on the plane, such that the angles between adjacent sides are preserved.

3.5. Listing’s Section and Fixation Eye Movements

Below we propose an approach to description of information processing in dynamics.

3.5.1. Retinotopic Image of a Stable Stimulus during Eye Movements

Recall that the direction of the gaze determines the position of the eye, which determines the frame and associated retinotopic coordinates.

Let the eye look for some time at a stationary surface, for example, at a plane , and the gaze describes a curve and hence is directed to the points of the stimulus . Then the eye position is defined by the curve . We call Listing’s curve.

The retinal image of the points forms the curve .

Moreover, if it the retinal image of a point at , then due to eye movement, the retinal image of the same point B at the moment t will be

Hence the retinal curve is the retinal image of the external point B. Indeed, in retinotopic coordinates, the eye is stable and the external plane is rotating in the opposite direction and at the moment t take the position . The point is the new position of the point .

3.5.2. n-Cycles of Fixation Eye Movements

We define a fixation eye movement as a FEM which starts and finishes at the standard eye position and consists of n drifts and n microsaccades between them. We will assume that MSs are instantaneous movements and occur at times . Then the corresponding Listing’s curve can be written as

We associate with n-cycle the spherical polygon with vertices (-gone)

The sides represent saccades and the sides corresponds to the drifts .

Using the stereographic projection of Listing’s sphere from the south pole to the tangent plane , we can identify P with an -gone on the tangent plane .

In the case of saccade, Listing’s curve is a segment . Hence all saccades of n-cycle are determined by the position of their initial and final points in Listing’s hemisphere, i.t. by points .

For example, a 3-cycle is characterised by the hexagon and consists of 3 drifts and 3 MSs:

An example of 3-cycle and associated hexagon is depicted in Figure 11.

Figure 11

Hexagone.

We suppose that during n-cycle with a Listing’s curve the visual system perceives local information about the stimulus, more precisely, information about points B whose retinal image belong to the fovea. The information needed for such local pattern recognition during a FEM cycle consists of two parts:

The dynamical information about Listing’s curve , coded in oculomotor command signals. A copy of these signals (corollary discharge (CD)) is sent from the superior colliculus through the MD thalamus to the frontal cortex. It is responsible for visual stability, that is the compensation of the eye movements and perception of stable objects as stable.

The visual information about characteristics of a neighborhood of points B of the stimulus which is primary encoded into the chain of photoreceptors along the closed retinal curve , which represents the point B during FEM. Then this information is sent for decoding through LGN to the primary visual cortex and higher order visual structures. In particular, if is the gaze curve with the initial direction to external point , the point of fixation A is represented by the retinal curve with .

3.6. A Model of Fixation Eye Movements

At first, we consider a purely deterministic scheme for processing information encoded in CD and visual cortex.

Then we discuss the problem of extending this model to a stochastic model. We state our main assumptions. If the opposite is not stated, we assume that we are working in spatiotopic coordinates associated to .

1. We assume that CD contains information about the eye position during the beginning and the end of the saccades , (which is equivalent to information about the gaze positions) and about the corresponding time .

2. We assume also that CD has information about Listing’s curve of the drift from the point to the point . (This assumption is not realistic and later we will revise it.)

3. Let B be a point of the stable stimulus and its retina image at the time . Then during the drift the image of B is the retina curve . We denote by the characteristics of this image B, which is recorded in the activation of photoreceptors along the retinal curve during the drift and then in firing of visual neurons in V1 cortex and higher order visual subsystems. Note that the information about the external stable point B is encoded into the dependent on time vector–function . This is a manifestation of a phenomenon that E. Ahissar and A. Arieli [12] aptly named ‘figurating space by time’.

4. We assume that the (most) information about the drift , encoded in Listing’s curve and about the characteristic functions , is encoded in the coordinate system, associated to the end point of the preceding saccade . We remark that if , then associated with coordinate system is obtained from the spatiotopic coordinates by the rotation along the axe p of the Listing plane through the angle . (These coordinates are the retinotopic coordinates at the time ).

5. Let C be another point of the stable stimulus with the retina image at and the characteristic function of the retina image of C during drift . Then the visual system is able to calculate the visual distance between point during drift as an appropriate distance between their characteristic functions .

6. We assume that the change of coordinates (remapping) appear during each saccade. So for example during 3-cycle, the system uses the coordinates associated to the following points of Listing’s hemisphere

Here the interval indicates the time of the drift when the coordinates is used.

7. In particular, this means that the information about the characteristic function of the external point B along the retinal curves during the drift is encoded into the coordinates associated to the end point of the preceding saccade (which are the retinotopic coordinates at the time ).

To recalculate the characteristic function in terms of the spatiotopic coordinates, associated to , it is sufficient to know the point .

8. Following M. Zirnsak and T. Moore [46], we suppose that during the drift , the visual system chooses an external saliency point A as the target for the next gaze position. More precisely, it fixes the retinal image of this point w.r.t. coordinates associated with (which are retinotopic coordinates at the moment ). After the next saccade (at the moment ) the point will become the point F (the center of the fovea) and after the saccade the point A will be the target point of the gaze vector , .

9. This allows to give an explanation of the presaccadic shift or receptive fields.

The above assumption means that before the time of the saccade , the visual system knows the future gaze vector with respect to the coordinates, associated with . Of course, this information may be obtained only due to collaboration of the visual system with the ocular motor system. At some moment 100 ms these subsystems recalculate the characteristic functions from the coordinates into the new coordinates, associated to the future gaze point and send this information to neurons of different visual systems.

This leads to the shift of receptive field, discovered in [45]. The information about the future characteristic functions will contains some mistakes since the real position of the eye at the moment is different from the position . It is observed as dislocation (compression) of the image in space and time [16,17,18,19]. After the saccade, this mistakes are corrected. One of the way to reduce such dislocation is to increase the frequency of microsaccades.

3.6.1. Diffusion Maps and Stochastic Model of FEM

It seems that the assumption 1. that the CD contains information about the eye position at the beginning and the end of each saccade is rather reasonable. However, the assumption 2. must be clarified. Since the drift trajectory (Listing curve) can be arbitrary, it is difficult to believe that the CD stores information about its shape even for a short time. It is naturally to assume that the drift is a random walk and the ocular motor system and CD store information about random trajectory of the drift. Similarly, the characteristic functions , which contain information about the stable stimulus B, recorded by photoreceptors during the drift becomes a random function.

3.6.2. Diffusion Map by R.R. Coifman and S. Lafon

We shortly recall the basis ideas of the diffusion maps (or diffusion geometry) by R.R. Coifman and S. Lafon [34,35], which we will need.

The diffusion geometry on a (compact oriented) manifold M with a volume form such that is determined by a kernel i.e., a non negative and symmetric ( function on . An example of the kernel is the Gauss kernel , of the Euclidean space or the heat kernel of a Riemannian manifold. The normalization of the kernel gives the transition Markov kernel which defines a random walk on M. The value is considered as the probability to jump in one step from the point x to the point y.

The associated diffusion operator P on the space of function is defined by

Then the probability density to move form x to y in steps is described by the kernel associated to the power of the operator P such that

It can be defined for any in terms of the eigenvectors and eigenfunction of the operator P [34]. So any point determines a family of the bump functions on M, which characterize the local structure of a small neighborhood of x. We call the trajectory of random walk (or random trajectory) started from x during time interval

R.R. Coifman and S. Lafon [34] define the diffusion distance between points as the -distance between the bump functions (or random trajectories) and , started form these points:

Let be eigenvalues of the diffusion operator P and associated eigenfunctions. Then for sufficiently big number m, the diffusion distance is approximated by the function given by

In other worlds, the map (called the diffusion map) is closed to the isometric map of the manifold M with the diffusion metric to the Euclidean space . If the manifold M is approximated by a finite systems of points , the diffusion map gives a dimensional reduction of the system X.

3.6.3. Remarks on Stochastic Description of Drift as Random Walk and Possible Application of Diffusion Distance

The idea that FEMs is a stochastic process and may be described as a random walk has a long history, [29,30,31,32,33,42].

1. We assume that the drift is a random walk on the Listing hemisphere defined by some kernel. The question is to chose an appropriate kernel. The first guess is to assume that it is the heat kernel of the round (hemi)sphere. The short-time asymptotic of the heat kernel of the round sphere is known, see [47]. The functional structure of the retina which records light information, is very important for choosing the kernel. Inhomogeneity of the retina shows that the first guess is not very reasonable. It seems that the more natural assumption is that the system uses the heat kernel for the metric on Listing hemisphere, which corresponds to the physiological metric of the retina. Recall that it is the pull back of the physical metric of the V1 cortex with respect to the retinotopic mapping.

2. We assume that the drift is a random walk in Listing’s hemisphere, defined by some kernel. Then by the drift trajectory from the point we may understand the random trajectory on (or the bump function) during the time interval . It has no fixed end point but it allows to calculate the probability that the end point belongs to any neighbourhood of the point . The situation is similar to Feynman’s path integral formulation of quantum mechanics. Moreover, if by a point we will understand not a mathematical point but a small domain, e.g., the domain which corresponds to the receptive field of a visual neuron in V1 cortex or the composite receptive field of a V1 column (which is 2–4 time larger) [37], then we may speak about random drift from the point to the point with the bump function (“the random trajectory”). Roughly speaking, this function gives the probability that the random drift from the point to the point after steps comes to the point .

3. Due to diffeomorphism defined by the Hopf map , we may identify the random walk in with the random walk on the eye sphere . A drift in induces the “drift” of a point given by

Let A be the fixation point of the gaze at the initial moment , such that its retina image is . Then the retina image of the point A during the drift is the curve . More generally, if is the retina image at of any other point B of the stimulus, then the retina image during the drift is

In the stochastic case, the drift is characterized by the random trajectory , and associated “drift” of points in by the random trajectory where and is Listing’s section. Note that the right hand side does not depend on the point .

We conjecture that the ocular motor control system detects information about random trajectories in and and the corollary discharge get a copy of this information.

It seems that the proposed explanation for shifting receptive fields may be generalized to the stochastic case.

4. Let B be a stable stimulus and its retina image at and the retina image at the time . Denote by the characteristic function, which describes the visual information about a stable stimulus point B with the retina image during the drift . If the drift is considered as a random walk, the information about the drift curve is encoded in the function and the characteristic function becomes a random function and is described by the bump function on . We suppose that the visual system calculates the visual distance between external points as the diffusion distance between the associated bump functions.

5. We also conjecture that like in deterministic case, the information about the random trajectory of the drift encoded in CD and the information about characteristic bump function, encoded in different structures of the visual cortex are sufficient for stabilization of visual perception. The problem reduces to recalculation of all information in spatiotopic coordinates, associated with the point .