Texture-dependent motion signals in primate middle temporal area.

Neurons in the middle temporal (MT) area of primate cortex provide an important stage in the analysis of visual motion. For simple stimuli such as bars and plaids some neurons in area MT--pattern cells--seem to signal motion independent of contour orientation, but many neurons--component cells--do not. Why area MT supports both types of receptive field is unclear. To address this we made extracellular recordings from single units in area MT of anaesthetised marmoset monkeys and examined responses to two-dimensional images with a large range of orientations and spatial frequencies. Component and pattern cell response remained distinct during presentation of these complex spatial textures. Direction tuning curves were sharpest in component cells when a texture contained a narrow range of orientations, but were similar across all neurons for textures containing all orientations. Response magnitude of pattern cells, but not component cells, increased with the spatial bandwidth of the texture. In addition, response variability in all neurons was reduced when the stimulus was rich in spatial texture. Fisher information analysis showed that component cells provide more informative responses than pattern cells when a texture contains a narrow range of orientations, but pattern cells had more informative responses for broadband textures. Component cells and pattern cells may therefore coexist because they provide complementary and parallel motion signals.

Key points

The receptive fields of neurons in the middle temporal (MT) area of primate visual cortex are an important stage in motion analysis. Some neurons in MT (pattern cells) can signal motion independent of contour orientation, but others (component cells) cannot; there is no systematic account of how responses in area MT depend on the spatial structure of images.

We measured the extracellular response of neurons in area MT of anaesthetised marmoset monkeys to synthetic textures and natural images.

Direction tuning of pattern cells was broad and largely stable against variation in spatial texture. Direction tuning of component cells was narrower than that of pattern cells when spatial textures contained few orientations, but tuning was not stable against variation in spatial texture.

Response variability in all neurons was lower for rich spatial texture.

Pattern and component cells may provide parallel analyses for motion vision.

Introduction

Visual motion analysis in primates is thought to require two distinct stages of processing. The first stage is captured by direction-selective neurons in primary visual cortex (V1), which have receptive fields tightly tuned for contour orientation and direction of motion (Hubel & Wiesel, 1968; Adelson & Bergen, 1985; Bradley & Goyal, 2008). The second stage has been most extensively studied in cortical area MT/V5. Almost all neurons in the middle temporal (MT) area are selective for direction of visual motion (Dubner & Zeki, 1971; Albright, 1984). Neurons in area MT can be functionally distinguished by their response to plaids formed by superimposed drifting gratings. While many direction-selective neurons in area MT, like those in V1, respond whenever one of the plaids’ components aligns with the preferred direction (‘component cells’), some neurons in area MT (‘pattern cells’) are capable of signalling the overall motion of the plaid and not the individual components (Movshon et al. 1985; Smith et al. 2005; Rust et al. 2006; Majaj et al. 2007; Solomon et al. 2011). Pattern cells are generally thought to draw convergent input (excitatory and inhibitory) from neurons with overlapping receptive fields and component-like response properties (Majaj et al. 2007): this convergence is what allows pattern cell motion signals to be stable against changes in the orientation of contours (Rust et al. 2006).

The emergence of pattern cells in area MT has proved a useful framework in which to understand how association areas of the sensory cortex transform the signals that are provided to them by primary sensory cortices. Yet only a minority of the neurons encountered in area MT show pattern cell response properties, and these appear to lie on the end of a continuum, with component cells at the other extreme. Why area MT supports receptive fields with different capacity to integrate motion remains mysterious. One possibility is that component cells in area MT are a preliminary stage in the computation achieved by pattern cells. This serial model is unlikely, because the receptive fields of component cells are binocular and are as large as those of pattern cells (Solomon et al. 2011), but the motion integration of pattern cells breaks down when the components of a plaid are spatially separated (Majaj et al. 2007) or are presented to different eyes (Tailby et al. 2010). Thus, pattern cells appear to rely on inputs from neurons with receptive fields unlike those of component cells in area MT. An alternative possibility is that component- and pattern-like responses provide complementary streams of motion analysis. In line with this hypothesis the response profiles of pattern and component cells remain distinct if the stimulus is a bar, cross or spot (Rodman et al. 1989; Okamoto et al. 1999), but we do not know if these responses generalise to more complex surfaces and objects.

The possible functional roles of different types of neurons in area MT would be better constrained if we had a systematic account of how their signals depend on the spatial structure of images. Here we examine response to synthetic images that lie on a continuum in Fourier space. At one end of this continuum are drifting gratings that contain a single orientation and spatial frequency; at the other end are complex textured stimuli with a large range of orientations and spatial frequencies. The aim of this work is to characterise the response of neurons in area MT across this continuum. We show that the motion sensitivity and information capacity of the different classes of neuron in area MT depend on the spatial structure of images. Our analyses suggest complementary roles for pattern and component cells in the analysis of visual motion.

Methods

Ethical approval

Adult male marmosets (Callithrix jacchus, n= 8, weighing between 350 and 430 g) were obtained from the Australian National Health and Medical Research Council (NHMRC) combined breeding facility (Churchill, VIC, Australia). Procedures were approved by the University of Sydney Animal Ethics Committee, and conform to the Society for Neuroscience and NHMRC policies on the use of animals in neuroscience research. The authors have read, and the experiments comply with, the policies and regulations of The Journal of Physiology given by Drummond (2009).

Experimental preparation

The neurons that are described here are a subset of those reported previously (Solomon et al. 2011). Each animal was initially sedated with an intramuscular (i.m.) injection of 12 mg kg−1 Alfaxan (Jurox, NSW, Australia) and 3 mg kg−1 diazepam (Roche, NSW, Australia). We then gave preoperative i.m. injections of 0.2 mg kg−1 atropine (Pfizer, New York, USA), to reduce lung secretions, and dexamethasone (0.3 mg kg−1; Maine Pharmaceuticals, VIC, Australia) to reduce inflammation. Subsequent surgery was performed under supplemental local anaesthesia (lignocaine 2%; Astra Zeneca, NSW, Australia). A femoral vein was cannulated, the trachea exposed and an endotracheal tube inserted, and the head was placed in a stereotaxic frame.

Post-surgical anaesthesia was maintained by continuous i.v. infusion of sufentanil citrate (4–12 μg kg−1 h−1; Sufenta Forte, Janssen Cilag, Beerse, Belgium) in physiological solution (sodium lactate; Baxter International, NSW, Australia) with added dexamethasone (0.4 mg kg−1 h−1; Maine Pharma) and Synthamin 17 (225 mg kg−1 h−1; Baxter International). The electrocardiogram (ECG), electroencephalogram (EEG) and were monitored continuously. Neuromuscular blockade was then induced and maintained by continuous infusion of pancuronium bromide (0.3 mg kg−1 h−1; Astra Zeneca). The animal was artificially ventilated, with a 70:30 mix of N2O and Carbogen, so as to keep end-tidal CO2 near 33 mmHg. EEG and ECG signals were monitored to ensure adequate depth of anaesthesia. Dominance of low frequencies (1–5 Hz) in the EEG recording and absence of EEG changes under noxious stimulus (tail-pinch) were taken as the chief sign of an adequate level of anaesthesia. At any sign of the anaesthesia becoming less effective, the dose of sufentanil citrate was increased. Rectal temperature was kept near 38°C with the use of a heating blanket. Additional antibiotic and anti-inflammatory cover was given daily by i.m. injection of 25 mg Noricillin (Norbrook, Corby, UK), and 0.1 mg dexamethasone. The pupils were dilated with atropine sulphate and the corneas were protected with high-permeability contact lenses that remained in place for the duration of the experiment. Artificial pupils were not used. At the end of the experiment the animal was killed via i.v. 500 mg kg−1 sodium pentobarbitone (Lethobarb; Verbac Australia, NSW, Australia).

Visual stimulation and recording

Stimuli were presented on a calibrated cathode ray tube monitor (ViewSonic G810, 100 Hz refresh rate, width 40 cm and height 30 cm), viewed from 114 cm via a front-silvered mirror or directly from a distance of 45 cm. The position of the monitor was adjusted to bring the receptive field of a neuron onto the centre of the screen. Visual stimuli were generated by a G5 Power Macintosh computer using custom software (EXPO); they were drawn with 8-bit resolution using commands to OpenGL. All stimuli were modulated around the mean luminance (45–55 cd m−2) and were presented within a circular window with hard edges; outside this window the screen was held at the mean luminance. Refraction was optimised using the first encountered neurons, by measuring responses to drifting gratings and selecting supplementary lenses that maximised the neurons’ spatial resolution. The power of the lenses determined in this way was similar to that typically needed to maximise the spatial resolution of neurons in the dorsal lateral geniculate nucleus of the thalamus (Camp et al. 2009). The non-dominant eye was occluded.

A craniotomy (diameter 8–10 mm) was made over the MT region and the posterior tip of the lateral sulcus was identified. A small slit was made in the dura and guide tubes containing one or two tetrodes, which were held vertically, were positioned above the cortical surface 2–3 mm posterior and 2 mm lateral to the posterior tip of the lateral sulcus (Rosa & Elston, 1998). The craniotomy was sealed with silicone elastomer. The extracellular recordings reported here were made using tetrodes (2–5 MΩ; Thomas Recordings). The analog signals from the electrodes were amplified, band-pass filtered (0.3–10 kHz) and sampled at 48 kHz by the same computer that generated the visual stimulus. Multiple neurons recorded simultaneously were isolated using real time principal components analysis. Off-line analysis was used to confirm and refine the identification of spike waveforms. The timing of waveforms was recorded with an accuracy of 0.1 ms. Subsequent analysis was performed using Matlab (MathWorks, Natick, MA, USA). The sample of neurons reported here is a subset of that reported in Solomon et al. (2011).

Visual stimuli

For each neuron we obtained response to gratings and plaids, or textures and images of varying orientation bandwidth, as described below. The procedure we used to collect responses was the same as that described by Smith et al. (2005; see also Solomon et al. 2011). In each case the set of stimuli, which always included two presentations of a blank screen, was presented in a pseudo-random sequence for 0.32 s, with no inter-stimulus interval. The response latency of neurons in area MT of marmoset is ∼0.08 s. This response latency needs to be accounted for in the analysis because there was no inter-stimulus interval. Following Smith et al. (2005) we did this by systematically changing the starting point of a 0.32 s analysis window, and for each of these analysis windows we calculated the variance in response (the average spike rate over the analysis window) across stimuli. For subsequent analyses we used the analysis window that maximised the response variance over all stimulus conditions in the set. For each set of stimuli, neurons were included for analyses if mean response to at least one of the stimuli in the set was more than 10 impulses s−1 above the maintained discharge rate.

Gratings and plaids

For each neuron we measured direction selectivity for sine-wave gratings of optimal spatial and temporal frequency, and size, presented at a Michelson contrast of 0.5. Plaids were the sum of two of these sine-wave gratings drifting at directions 120 deg apart. Both gratings and plaids were presented at each of 12 directions (0–330 deg at 30 deg intervals).

Synthetic textures

We used one set of synthetic textures (Goddard et al. 2008) to measure responses of all neurons. Each texture in the set was 256 × 256 pixels. To generate the textures we started by making two coordinate matrices of the same size. In one matrix we specified the distribution of Fourier amplitude over spatial frequency (distance of each pixel from the centre pixel, which was index [129,129]). This distribution was a log-Gaussian with centre spatial frequency 8 cycles per image and bandwidth 0.5 octaves. In the second matrix we specified the distribution of Fourier amplitude over orientation (angle of each pixel relative to the centre pixel). This distribution was a wrapped Gaussian. The standard deviation of the wrapped Gaussian was 6.3, 12.5, 25, 37 or 50 deg; an additional texture had uniform (isotropic) distribution of Fourier amplitude over orientation. For standard deviations less than about 30 deg the wrapped Gaussian is equivalent to a Gaussian. For larger standard deviations, the tails of a Gaussian distribution are not zero at orientations orthogonal to the peak, so the distribution must be wrapped, and this was done (Dakin et al. 2005).

The distributions over spatial frequency and orientation were multiplied together to provide a distribution of amplitude over spatial frequency and orientation. The resultant was then shifted using the function fftshift in Matlab. This amplitude distribution was used to filter uniformly distributed random noise. We further normalised the distribution of pixel intensities in the textures to zero mean, and standard deviation of 0.125 (where 1 is the maximum achievable luminance of the monitor, and 0 is the mean luminance). We chose this normalisation to minimise the impact of contrast gain controls early in the visual system – in the retina and thalamus – which are sensitive to the contrast of images but are largely insensitive to the distribution of that contrast over orientation and spatial frequency (Solomon et al. 2002, 2006; Bonin et al. 2005, 2006). This measure of contrast has also been shown to correlate well with the detectability of visual stimuli (Bex & Makous, 2002). The consequence of this normalisation is that as more orientations are included in each texture, the Fourier amplitude along the dominant orientation axis declines.

Textures were viewed through a circular aperture, the diameter of which was also the width of the texture. The texture therefore scaled with aperture size. We used linear interpolation when the chosen aperture was not the size of the original image. The appropriate size of the aperture was constrained by two factors. First, the components of the texture needed to be within the passband of the neuron under study. Second, to ensure that the measurements were not contaminated by edge effects the edge of the aperture needed to be outside the classical receptive field of the neuron. Average aperture diameter was 18.0 deg (n= 66), providing an average centre spatial frequency of 0.44 cycles deg−1. Neurons in area MT of marmoset usually resolve spatial frequencies of above 1 cycle deg−1 (Solomon et al. 2011), and the preferred spatial frequency of the neurons here was on average 0.25 cycles deg−1. The size of the aperture was larger than the classical receptive field: preferred size of a patch of drifting grating was on average 11.5 deg for the neurons under study. Translation speed was normally set near 20 deg s−1 and was adjusted if this was far from the preferred speed, as determined from preliminary measurements using drifting dot fields. Preferred speed of the neurons in this sample was on average 25.4 deg s−1. Translation speed of the textures was on average 24.7 deg s−1. Responses were obtained for a median 20 repetitions of each stimulus (mean 17.1, SD 4.2, range 10–20) for the standard set of measurements and a median 25 (mean 28.3, SD 8.3, range 15–37) for those where we varied motion direction and orientation independently.

Images of natural scenes

Image segments, whose amplitude spectrum showed strong linear correlation with a specified amplitude spectra (6, 25 and 50 deg orientation bandwidth), were selected from images in the van Hateren image database (van Hateren & van der Schaaf, 1998; Goddard et al. 2008). Like the synthetic images above, the target amplitude spectra consisted of a wrapped Gaussian distribution over orientation; unlike the synthetic images, the distribution over spatial frequency was 1/f, where f is frequency in cycles per image. One hundred 256 × 256 pixel segments were selected from random portions of each of the first 1000 images in the database. Three of the 16 image segments with highest linear correlation coefficient were selected for each orientation bandwidth. We normalised the distribution of pixel values in the textures to zero mean, and standard deviation of 0.125. Images were translated behind a circular aperture with hard edges, the diameter of which was fixed at 18.0 deg. Translation speed was usually 20 deg s−1. Presentation time was 0.32 s, and the edge of the image did not cross the receptive field. Images were rigidly translated along a direction orthogonal to the dominant orientation. Responses were obtained for a median 17 repetitions of each image (mean 15.5, SD 4.9, range 6–20).

Classification of pattern and component cells

We used the response of neurons to gratings and plaids to classify them as component or pattern cells. For each neuron we computed the partial correlation of response to a plaid with that predicted for an ideal component cell or an ideal pattern cell. An ideal pattern cell would respond to the plaid as if it were a grating moving in the average direction of the plaid. An ideal component cell would respond to the individual components of the plaid, so the response to a plaid should be the sum of response to the two component gratings. The partial correlations between the observed and ideal responses (rcomponent and rpattern) were then transformed to z-scores (zc and zp) using Fisher's r-to-Z transformation (Movshon et al. 1985; Smith et al. 2005). Cells were classified as component if the correlation with the component prediction (zc) exceeded 1.28 and the difference between the component and pattern predictions (zc–zp) also exceeded 1.28; the reverse criterion was applied to classify pattern cells (Smith et al. 2005). The sample of cells here included seven that were weakly responsive to gratings but robustly responsive to plaids (see Solomon et al. 2011). The direction tuning curves of these neurons for plaids were unimodal, they showed strong response to textures and we include them in our sample of pattern cells.

Direction tuning

To characterise direction tuning of each neuron we found the best prediction of a modified von Mises function. Some of the direction tuning curves that we obtained were bimodal, with strong response to one motion direction and weaker response to the opposite motion direction. We therefore characterised all tuning curves with a bimodal von Mises function, V(θ):

where θ is stimulus motion direction in degrees, A and B are respectively the gain along the preferred and anti-preferred direction (B was constrained to be less than A), κ is the concentration parameter of the von Mises function and θp is the neuron's preferred direction in degrees; m allows the entire curve to be offset and was constrained to be greater than or equal to zero. In the text we characterise the bandwidth of these tuning curves as the standard deviation of a circular Gaussian that corresponds to the von Mises concentration parameter κ (Jammaladak & Sengupta, 2001), and the amplitude of these tuning curves as that along the preferred direction, A. Parameters providing the best-fitting predictions of this model were found using the lsqcurvefit function in the Matlab environment, which minimised the square error between the model predictions and the response. We found the best predictions of this model for all tuning curves, but estimates of response bandwidth are not well constrained when response amplitude is low. Neurons were included in analyses if response to at least one of the stimuli in the set was more than 10 impulses s−1 above the maintained discharge rate. For analyses of response bandwidth we imposed an additional criterion, including only those tuning curves where the maximum response was greater than 5 impulses s−1 above baseline.

Variance-to-mean

To establish the variability of spike response to different sets of stimuli we measured spike count for each trial and from these calculated the ratio of variance-to-mean, or Fano factor (F; Fano, 1947). The Fano factor is defined as:

where σ2 is the variance, and μ is the mean, of spike count over time window W. We first calculated the Fano factor separately for each neuron, each stimulus bandwidth and each motion direction. Analysis of responses to any particular stimulus orientation bandwidth showed no dependence of Fano factor on the motion direction of the stimulus. For each stimulus orientation bandwidth we therefore present average Fano factor over all the neurons and motion directions included in the relevant data set. To ensure that changes in Fano factor were not simply related to changes in mean rate across textures we also calculated the mean-matched Fano factor, as described in Churchland et al. (2010). Briefly, for each stimulus bandwidth we established the distribution of the mean spike count over all neurons and all motion directions, with bin width of 1 spike. We then defined a global distribution where the height of each bin in the histogram was the minimum for the relevant bin across all textures. For each stimulus we then removed randomly chosen observations until the height of each bin in the distribution matched that of the global distribution. The observations that remained were used to calculate the Fano factor for each stimulus.

Fisher information

We used Fisher information to estimate the capacity of each neuron to discriminate nearby motion directions. We calculated Fisher information for each neuron, and each stimulus, by combining the derivative of the fitted von Mises tuning curve (evaluated at steps of 1 deg) and the measured Fano factor (Butts & Goldman, 2006; Durant et al. 2007). The Fisher information over direction, f(θ) is

where V(θ) is the direction tuning curve (eqn 1) for the stimulus and neuron under study; F is the Fano factor (eqn 2) for the stimulus and neuron under study, averaged across motion directions. T is the time window over which spike count was measured. We present mean Fisher information across the neurons in the sample.

Statistics

We used a mixed design two-way analysis of variance (ANOVA), in the SPSS environment. We used a ‘within-subject’ analysis to test the effect of stimulus bandwidth on the relevant measurements from each neuron. We used a ‘between-subject’ analysis to compare the effect of stimulus bandwidth on each different class of neuron. We also tested the interaction between cell class and stimulus bandwidth. As our data for the synthetic textures violated the sphericity assumption, we used the Greenhouse–Geisser model to test the significance levels. For natural images sphericity could be assumed.

Modelling

There is no single accepted model of the functional properties of neurons in area MT. To understand the neuronal responses, however, we wanted to know the expected effect of stimulus manipulations on the direction tuning of component and pattern cells. A freely available and validated framework is that of Simoncelli & Heeger (1998; http://www.cns.nyu.edu/∼lcv/MTmodel/), implemented in the Matlab environment. Model predictions were generated for the default parameter values provided in the download. We modified the simulation such that the output of each V1 direction selective complex (DS) cell was subject to two types of normalisation (Rust et al. 2006): by the summed response of the population of V1 DS cells, and by their own response. The latter is equivalent to a static compressive non-linearity, which may arise in the action of ‘end-stopping’ or ‘suppressive surrounds’ (Rust et al. 2006; Tsui et al. 2010). The strength of normalisation was the same for both types. Stimuli were translated at a speed of 1 pixel per frame. Direction tuning curves were established for each simulation by selecting model neurons with appropriate preferred directions. Unless specified otherwise, the preferred speed of the model neurons was the same speed as the stimulus. Direction tuning curves were established for each simulation by selecting model neurons with appropriate preferred directions. Predictions were averaged across 100 simulations, each performed on textures generated by a different random sample of noise.

Results

Pattern and component cells in area MT are distinguished by their capacity to signal motion independent of stimulus orientation, but this has only been established for simple stimuli such as bars, gratings and plaids. In the following we show that component and pattern cell response remains distinct for textures that comprise a range of orientations and spatial frequencies. Each texture was generated by filtering random noise. As described in Methods, a Gaussian was used to specify the filter Fourier amplitude at each orientation, and different textures were generated by changing the average orientation or standard deviation of that Gaussian (the distribution over spatial frequency was held constant).

Figure 1 shows examples of textures with standard deviation 6 and 25 deg, and an additional texture that contained all orientations. Figure 1 illustrates the Fourier spectra of these textures when they move orthogonal to the average orientation. Each plot indicates for one texture the distribution of amplitude in three-dimensional Fourier space, which is defined by two axes of spatial frequency and one axis of temporal frequency. The translation speed of each texture is the same, and the Fourier spectra of all textures are therefore strongest near the same tilted plane, the tilt of which is set by the translation speed. How much of the plane is occupied depends on the orientation bandwidth of the texture – as more orientations are added, more of the plane is filled in.

Figure 1

Examples of synthetic textures

A, textures were generated by parametrically varying bandwidth of a filter applied to random spatial noise. In all cases the filter was band-pass over spatial frequency; for the example textures the orientation bandwidth of the filter was a standard deviation of 6.25 deg or 25 deg. An additional texture had uniform distribution over orientation. Textures were moved in a direction orthogonal to the dominant orientation. B, representation of textures in three-dimensional Fourier space. The x- and y-axes show, respectively, the spatial frequency parallel to the direction of motion and orthogonal to it. The z-axis shows the temporal frequency, and the centre of the space is zero spatial- and temporal frequency. The surfaces are defined by the locations in this space where the Fourier amplitude had decreased to 30% of the maximum.

We first illustrate how neurons in area MT can be classified as component or pattern cells from their response to gratings and plaids, which we obtained for all neurons included in the following analyses. We then show how response depends on the orientation of the texture relative to its motion direction, and then concentrate our quantitative analyses on the response to textures that drift orthogonal to the average orientation of the texture. It was not possible to obtain the complete set of measurements from all neurons, and Table 1 shows the number of neurons included in each of the following analyses.

Table 1

Numbers of neurons included in different analyses

	Total	Pattern	Component	Unclassified
Textures (parallel and orthogonal)	18	5	9	4
Textures (orthogonal)	66	19	16	31
Natural images	42	10	18	14

Classification of pattern and component cells using gratings and plaids

Figure 2 shows the direction tuning of four representative neurons to gratings, and to plaids that are the sum of two gratings drifting in directions 120 deg apart. We used these responses to define a neuron as a component, pattern or unclassifiable cell for our subsequent analyses, as described in Methods under ‘Classification of pattern and component cells’. Figure 2 shows the response of a typical component cell. The response to plaids is bimodal, because the neuron responds when either component of the plaid is aligned to its preferred orientation. Figure 2 shows the response of two pattern cells in area MT. These neurons show similar response to gratings and plaids – they signal the overall motion of the plaid and not that of its individual components. Figure 2 shows the response of one unclassifiable cell in area MT. This neuron is unusual in that it shows a single response peak for plaids, but the motion direction corresponding to the response peak is different from that obtained for gratings, as if the neuron responded to one component of the plaid but not the other.

Figure 2

Response of neurons in area MT to gratings and plaids

A, grating (filled symbols); B, plaid (open symbols). Each panel in C–F shows mean discharge rate for moving gratings and plaids. For each neuron responses were aligned to the preferred direction for a drifting grating. C, component cell. The response to plaids is bimodal. D, pattern cell. The response to plaids is unimodal. E, pattern cell. The response to plaids is unimodal. F, unclassifiable cell. This neuron is unusual in that it shows a single response peak for plaids, but the peak is different from that obtained for gratings, as if the neuron responded to one component of the plaid but not the other. Error bars in C–F show ±1 SEM.

Response of pattern and component cells to moving textures

In the following sections we generate predictions for how component and pattern cells should respond to the moving textures we used as stimuli, using the model of primate motion processing provided by Simoncelli and Heeger (1998). We then compare these predictions to the response of neurons recorded in area MT. Figure 3 shows examples of some of the visual stimuli used. Textures contained a broad range of orientations (in Fig. 3, the standard deviation over orientation was 25 deg), distributed around the average orientation. The texture moved either orthogonal or parallel to its average orientation.

Figure 3

Response of model neurons and neurons in area MT to moving textures

A, examples of the stimuli used. Textures contained a broad range of orientations (standard deviation over orientation, 25 deg). The texture moved either orthogonal (left) or parallel (right) to its average orientation. B, predicted response of direction-selective complex cells in V1 and MT pattern cells to motion orthogonal (left panel) and parallel to (right panel) the dominant orientation. C, average response of component (n= 9) and pattern (n= 5) cells in area MT to motion orthogonal and parallel to the dominant orientation. Response of each neuron was aligned to the preferred direction for motion orthogonal to a texture of bandwidth 25 deg, normalised to the maximum response across stimuli and then averaged. Responses are plotted as a function of the direction in which each texture is translated. Error bars show ±1 SEM.

Figure 3 shows the response predicted by the model for DS cells in V1, and pattern cells in area MT, to motion of the textures along the average orientation of the texture or orthogonal to it. For the types of stimuli used here, component cells in area MT should respond in the same way as DS cells in V1; we therefore use the model's predictions for V1 DS cells as a surrogate for component cells in area MT (we return to this in Discussion). The left panel in Fig. 3 shows responses when the average orientation was orthogonal to the motion direction; the right panel in Fig. 3 shows responses when the average orientation was parallel to the motion direction. For model V1 DS cells, and therefore component cells in area MT, response depends on the motion of components in the texture that are aligned to the preferred orientation of the cell. Direction tuning therefore depends on the average orientation of the texture: tuning is unimodal for textures drifting orthogonal to the dominant orientation (Fig. 3, left panel), and is bimodal when the textures drift parallel to the dominant orientation (Fig. 3, right panel).

The predicted response of pattern cells is different from that of V1 DS cells and component cells. Model pattern cells combine inputs from a large range of V1 cells, each selective for a different component motion: each model pattern cell receives excitatory input from a particular set of V1 neurons with preferred spatio-temporal frequencies lying on a plane in the Fourier domain, and inhibitory input from V1 neurons with preferred spatio-temporal frequencies lying off that plane. It is this combination of excitatory and inhibitory inputs that makes the direction tuning of model pattern cells more stable against changes in the average orientation of the texture. Preferred direction is the same whether the texture moves orthogonal or parallel to its dominant orientation (Fig. 3).

We measured response from 18 neurons in area MT of marmoset (9 component, 4 unclassified, 5 pattern) to textures with average orientation orthogonal or parallel to the motion direction. For analysis of neuronal response, we aligned responses of each neuron to the preferred direction during presentation of a texture moving orthogonal to its average orientation. Figure 3 shows the average direction tuning curves of component and pattern cells. As expected, when average orientation was orthogonal to motion direction, tuning curves were unimodal in both pattern and component cells. Both pattern and component cells show robust response when the texture moved parallel to its average orientation, but distinct response profile – pattern cells have unimodal tuning curves aligned to the same preferred direction; component cells show bimodal tuning curves, with preferred direction 60–90 deg away from that preferred for motion orthogonal to average orientation. Two unclassifiable cells showed response profiles like those of pattern cells; the other two showed response like that of component cells (not shown). There is some discrepancy between the predicted response of pattern cells (Fig. 3) and the observed response (Fig. 3). That is, when textures move parallel to their dominant orientation model pattern cells show narrowed direction tuning curves and robust response along anti-preferred motion direction, but the neurons in our sample show broad tuning curves and no response along anti-preferred motion direction. We return to this in Discussion. For our current purposes we emphasise that the response of component and pattern cells to moving textures is qualitatively similar to that predicted for model V1 complex cells and MT pattern cells. The response of component and pattern cells is distinct even when the spatial texture has a single dominant orientation.

Response as a function of texture orientation bandwidth

The primary aim of this work is to understand how response of neurons in area MT depends on the orientation content of moving two-dimensional textures. A full description would include both changes in the average orientation of a texture (relative to its motion direction) and changes in the distribution over orientation. This was too large a space for us to explore quantitatively in individual neurons. In the following we therefore focus on the response modulation brought about by changes in distribution over orientation, because this describes a natural continuum between simple stimuli used in the laboratory, such as bars and gratings, and the richer spatial structure of more natural images. The average orientation of the texture was always orthogonal to the motion direction.

Figure 4 shows the tuning curves predicted by the Simoncelli and Heeger (1998) model for DS cells in V1 and pattern cells in area MT. Figure 4 shows the response bandwidth and response amplitude of these model neurons, obtained by finding the best predictions of a von Mises function fit to the tuning curves. For model V1 DS cells, and therefore component cells, response depends on the components in the texture that are aligned to the preferred orientation of the receptive field. Response amplitude and response bandwidth therefore depend on the distribution of orientations in the texture. The response amplitude of the model V1 DS cell declines slightly with the orientation bandwidth of the texture. This is because Fourier amplitude along the average orientation declines at large orientation bandwidths (see Methods). We note that the reduction in response amplitude is much less than the reduction in Fourier amplitude. This is because both tuned and un-tuned normalisation among neurons in V1 effectively suppresses response to narrow bandwidth textures. In addition, the response bandwidth of the model V1 DS cell increases with the orientation bandwidth of the texture. This is because for textures with large orientation bandwidths there is a wider range of translation directions where the texture contains components that are aligned to the orientation that is preferred by the neuron.

Figure 4

Response of model neurons to moving textures of different orientation bandwidth

A, predicted response of direction-selective complex cells in V1 to the synthetic textures shown in Fig. 1. Response bandwidth depends on texture bandwidth, but response amplitude is less affected. B, predicted response of pattern cells in area MT that receive weighted input from many direction-selective complex cells in V1. The response bandwidth is relatively stable against changes in the orientation bandwidth of the texture. The response amplitude increases markedly with orientation bandwidth. C, response bandwidth of the model neurons shown in A and B, as a function of texture orientation bandwidth. D, as in C, but showing response amplitude of the model neurons. Response amplitude and bandwidth were obtained by finding the best predictions of a von Mises function fit to the tuning curves. Data points in C and D show mean over 32 model neurons, measured in each of five simulations. Each simulation was performed with a different sample of noise.

The predicted response of pattern cells is different from that of V1 cells. The response bandwidth of model pattern cells does not increase with the orientation bandwidth of the texture, because increased response bandwidths in both excitatory and inhibitory inputs counteract. The response amplitude of model pattern cells increases markedly at large orientation bandwidths. This reflects the summation of multiple fundamental motion detectors by the receptive fields of model pattern cells.

Response of neurons in area MT

Figure 5 shows the direction tuning of four example neurons to each of four textures with varying orientation bandwidth. These are the same neurons as in Fig. 2. Figure 5 shows the response of a component cell in area MT. As for model V1 DS cells in Fig. 4, response bandwidth increases with the orientation bandwidth of the texture. Unlike the model V1 cells, tuning curve amplitude is little affected by orientation bandwidth.

Figure 5

Response of neurons in area MT to moving textures of different orientation bandwidth

Same neurons as in Fig. 2. Each panel shows mean discharge rate for four textures of increasing orientation bandwidth (6, 25 and 50 deg bandwidth, and isotropic). For each neuron responses were aligned to the preferred direction for a drifting grating. A, component cell. Response bandwidth increases with bandwidth of the texture, but response amplitude does not. B, pattern cell. Response amplitude and bandwidth are invariant against texture bandwidth. C, pattern cell. Response bandwidth is independent of texture bandwidth, but response amplitude increases markedly. D, unclassifiable cell. Increasing orientation bandwidth produces monotonic shift in preferred direction. Error bars show ±1 SEM

Figure 5 shows the response of two pattern cells in area MT. The response of these neurons to gratings and plaids was similar (Fig. 2). Unlike the component cell, the response bandwidth of both pattern cells is stable against changes in the orientation bandwidth of the texture. For the neuron in Fig. 5, tuning curve amplitude does not depend on the distribution of orientations in the spatial texture, unlike the model pattern cells in Fig. 4. For the neuron in Fig. 5, amplitude increases markedly when textures have large orientation bandwidths. Most pattern cells in our sample showed behaviour like that in Fig. 5: in 14/19 (74%) pattern cells the best response to isotropic textures was at least twice as great as that to textures of the narrowest orientation bandwidth.

Figure 5 shows the response of one unclassifiable cell in area MT. Increasing orientation bandwidth of the stimulus produced monotonic shifts in preferred direction of this cell. The shift in preferred direction is similar to that seen when comparing response to gratings and plaids (Fig. 2).

Our analyses will concentrate on the impact of texture on mean response amplitude. We were initially concerned that changes in the orientation bandwidth of textures may bring about changes in response dynamics, and that this would confound measures of response amplitude. Figure 6 shows response dynamics changed little with texture orientation bandwidth. Figure 6 plots the average peri-stimulus time histogram for four textures, for all the neurons in our sample (Fig. 6), and separately for each of component, unclassifiable and pattern cell populations (Fig. 6). Across all cells, and within each subtype, the shape of the peri-stimulus time histogram is similar across textures. Figure 6 shows that across all cells response amplitude increases with orientation bandwidth, and Fig. 6 shows that this is primarily due to a large increase in the response amplitude of pattern cells. For component cells orientation bandwidth did have a small impact on response timing, with response to high bandwidth textures lagging that for narrow bandwidth textures by about 10 ms; the effect on response timing is mild and so we ignore it in subsequent analyses.

Figure 6

Population response time course for moving textures

Each panel shows average response rate along the most effective motion direction for each of the textures shown in Fig. 1, after accounting for response latency, which was estimated independently for each neuron. A, average across all neurons in the sample (n= 66). B, average across all component cells (n= 16). C, average across all unclassifiable cells (n= 31). D, average across all pattern cells (n= 19). Vertical lines to the right show ±1 SEM, calculated separately for each bin and then averaged. Bin width 10 ms.

Impact of orientation bandwidth on direction tuning curves

Figure 7 summarises the impact of texture orientation bandwidth on the direction tuning curves of neurons in area MT. Figure 7 shows the average response of component and pattern cells, respectively. Responses were normalised to the preferred direction and maximum response obtained with narrowband textures (bandwidth 6 deg). For component cells, tuning curve bandwidth, but not amplitude, depends on orientation bandwidth. Pattern cells show the opposite: tuning curve amplitude, but not bandwidth, depends on orientation bandwidth.

Figure 7

Impact of texture orientation bandwidth on direction tuning of neurons in area MT

A, mean response of component cells (n= 16) to each of four textures. Responses were aligned and normalised to the preferred motion direction for textures of bandwidth 6 deg. B, mean response of pattern cells (n= 19). Conventions as in A. C, response amplitude of all neurons as a function of texture orientation bandwidth. Response amplitude is the parameter A in eqn (1), obtained by finding the best predictions of the model for the relevant tuning curve of each neuron. D, response amplitude of component (n= 16), unclassifiable (n= 31) and pattern (n= 19) cells. Conventions as in C. The data points adjacent to the y-axis show same measure of response amplitude, but for drifting gratings of preferred spatial and temporal frequency, and contrast 0.5. E, tuning curve bandwidth of all neurons as a function of texture orientation bandwidth. Bandwidth is derived from the parameter K in eqn (1). F, tuning curve bandwidth of pattern, component and unclassified cells. Conventions as in E. The data points adjacent to the y-axis show same measure of tuning curve bandwidth, but for drifting gratings of preferred spatial and temporal frequency, and contrast 0.5. All error bars are ±1 SEM.

To quantify these observations, we found the best predictions of a von Mises function for each of the direction tuning curves, and from this extracted the parameters for tuning curve amplitude and tuning curve bandwidth. Figure 7 shows these parameters for the population as a whole (Fig. 7), and for each of the subtypes (Fig. 7). Figure 7 shows tuning curve amplitude. Across all cells the amplitude increases with orientation bandwidth. This primarily reflects a large increase in the amplitude of pattern cells: for pattern cells, amplitude is more than twice as great for isotropic textures than it is for narrowband textures. Figure 7 also shows response amplitude for gratings of preferred spatial and temporal frequency, and contrast 0.5, obtained during the measurements illustrated in Fig. 2. The response amplitude of each cell class is close to that for textures of the narrowest orientation bandwidth.

Figure 7 shows tuning curve bandwidth. Across all cells tuning curve bandwidth increases with stimulus bandwidth. This increase is driven largely by increases in the tuning curve bandwidth of component cells; that of pattern cells is more stable. For narrow bandwidth textures the tuning width of component cells is about half that of pattern cells, but for large bandwidth textures component and pattern cells have similar tuning width. Tuning curve bandwidth for gratings is close to that for textures of the narrowest orientation bandwidth. ANOVA showed significant effects of orientation bandwidth for both tuning curve amplitude and bandwidth (F2.86,148.98= 16.2 and F1.35,82.4= 28.5; P < 0.001 in both cases); tuning curve bandwidth but not amplitude showed a significant interaction between orientation bandwidth and functional subtype (F2.7,82.4= 5.173, P= 0.003).

Changes in texture orientation bandwidth were not generally accompanied by a change in preferred motion direction. To illustrate this, we calculated the absolute difference in preferred motion direction for an isotropic texture, and each of the textures with narrower orientation bandwidth. This difference in preferred direction did not depend on orientation bandwidth of the comparison texture (not shown), so we present data averaged across all textures, including only those tuning curves where maximum response exceeded 5 impulses s−1. Across the population of neurons and textures, average difference in preferred motion direction was 13.4 deg (SD 16.5; 312 comparisons between tuning curves). Instability in preferred motion direction was slightly greater in component cells (μ 18.8, SD 20.2; n= 74) than in unclassified cells (μ 12.1, SD 12.9; n= 148) and pattern cells (μ 11.0, SD 12.9; n= 90), but we note that these shifts are less than the sampling interval (30 deg). Six neurons with unimodal direction tuning curves for plaids showed large (>45 deg) differences in preferred motion direction for gratings and plaids (1/16 component cells, 5/31 unclassifiable cells). The difference in preferred motion direction is why these neurons are not classified as pattern cells by the conventional metric. In three of these neurons, increasing texture orientation bandwidth led to the same monotonic shift in preferred direction, but in the other three it did not.

While all component cells showed strong direction selectivity for large bandwidth textures, response to the narrowest bandwidth textures was more variable. In 5 of 16 component cells response to the anti-preferred direction was more than one-third that to the preferred direction (for these neurons response to the anti-preferred direction was on average 0.45 (SD 0.08, n= 5) that to the preferred direction). In none of these neurons was there strong response to anti-preferred motion direction when the texture was a drifting grating of preferred temporal frequency (not shown). In these cells response to anti-preferred directions diminished rapidly with bandwidth of the texture, and for isotropic textures all component cells showed strong direction selectivity (response to anti-preferred direction was on average 0.05 (SD 0.08, n= 16) that to the preferred direction). Pattern and unclassifiable cells did not show strong response to anti-preferred directions of motion of any texture. Because the textures contain a range of spatial frequencies and orientations, some of the Fourier components are modulated at high temporal frequency. Although we have not explored it further, we expect that component cells that show this type of response have receptive fields that encompass components of the texture modulated at high temporal frequency. In some direction-selective neurons in V1 of cat and ferret, direction selectivity is diminished at high temporal frequency (Saul & Humphrey, 1992; Moore et al. 2005). Lack of response to anti-preferred directions for textures of broader bandwidth may arise because those textures engage appropriate suppressive mechanisms, or because the contrast of the effective components is reduced.

In summary, the responses of component and pattern cells in area MT show very different dependencies on the orientation bandwidth of moving textures. Component cells have much sharper tuning curves than pattern cells when the stimulus contains a narrow range of orientations: component cells may therefore be better able to support motion analysis for these types of stimuli. The tuning width of pattern cells is more stable against orientation bandwidth, and response amplitude increases markedly when textures have broader bandwidth: pattern cells may therefore be better able to support motion analysis for these types of stimuli.

Response variability and motion discrimination

The mean response of a neuron is a robust measure of functional properties. There is mounting evidence, however, that response variability is also modulated by sensory stimuli. Repeated presentations of the same stimulus give rise to variable response from cortical neurons (Schiller et al. 1976; Dean, 1981; Tolhurst et al. 1983; Vogels et al. 1989; Snowden et al. 1992; Britten et al. 1993). This trial-by-trial variability of neural response may be a source of information about the state of neurons (Shadlen & Newsome, 1998). Trial-by-trial variability necessarily places limits on the stimuli that can be reliably discriminated by an individual neuron – when variability is high, two stimuli can be discriminated only when they elicit very different mean response. In the following we show that spike count variability depends on texture orientation bandwidth, and follow the consequences of this for the directional information provided by MT neurons.

Figure 8 shows how variability in spike count depends on texture orientation bandwidth, across the population of cells, expressed as the ratio of variance to mean. This ratio is often called the Fano factor. We calculated Fano factor separately for each cell and each stimulus; Fano factor was not dependent on motion direction, so we combined data across all directions. Figure 8 shows counterpart plots for each of the three groups of neurons. Across the population of neurons, and within each subtype, Fano factor declines with increasing orientation bandwidth. Pattern cells and component cells show similar Fano factors; unclassifiable cells show slightly higher Fano factor for all textures. ANOVA showed significant effects of orientation bandwidth (F3.7,225.15= 8.128, P < 0.001) but no interaction between bandwidth and functional subclass. To ensure that the change in Fano factor does not simply reflect a change in the distribution of mean responses, we also examined the Fano factor after matching the distribution of mean spike count across all textures (Churchland et al. 2010; see Methods). Figure 8 shows that the dependence of Fano factors on texture bandwidth is preserved if we restrict the analysis in this way. In the following we use Fano factors calculated across all responses, and not the mean-matched response, as it allowed us to perform parametric analyses of neuronal variability across orientation bandwidth.

Figure 8

Response variability and capacity of neurons to discriminate motion direction of textures

Same neurons as in Fig. 7. A, variance-to-mean ratio (Fano factor; open symbols) of spike count, as a function of texture orientation bandwidth, across all neurons in the population. The ratio was calculated separately for each neuron, motion direction and texture. Data points show geometric mean and ±1 geometric SEM, having collapsed across neurons and motion direction. In separate analyses the variance-to-mean ratios were drawn from distributions matched for mean spike count (‘mean matched’; filled symbols; see Results). B, variance-to-mean ratio for component, unclassifiable and pattern cells. Conventions as in A. C, maximum Fisher information across motion directions, for all neurons, as a function of texture orientation bandwidth. Fisher information was estimated as described in eqn (3). Error bars show ±1 SEM. D, maximum Fisher information across motion direction, for component, unclassifiable and pattern cells. Conventions as in C. E, average Fisher information for component (dashed line) and pattern (continuous line) cell populations as a function of the direction of motion of a narrowband texture (bandwidth 6 deg). Each pair of lines shows mean ±1 SEM. F, average Fisher information for broadband (isotropic) textures. Conventions as in E.

To quantify the capacity of individual neurons to discriminate between nearby motion directions we estimated Fisher information, which is influenced by the slope of a tuning curve and the variability in spike count. For any given tuning curve, Fisher information is highest at the point of maximal change, on the flanks of a tuning curve, where nearby directions give very different responses. Fisher information is also higher if there is less variability in spike count. For neurons in area MT the tuning of Fisher information shows two large peaks, one on each flank of the direction tuning curve. Where these peaks are depends on the width of the tuning curve. The height of the peaks depends on the slope of the tuning curve (which depends on the tuning width and response amplitude) and response variability.

Figure 8 shows average profiles of Fisher information for component and pattern cell populations; Fig. 8 shows Fisher information for narrowband textures and Fig. 8 shows Fisher information for broadband (isotropic) textures. For each neuron Fisher information was estimated by combining the fitted von Mises tuning curve and the measured variance-to-mean ratio (eqn 3). The direction tuning curve was rotated so that preferred direction of the all tuning curves was 0 deg and average Fisher information was then calculated across neurons. Figure 8 shows that for narrowband textures, component cells out-perform pattern cells. This is because the tuning curve amplitude and response variability of pattern and component cells is similar, but tuning curve bandwidth of component cells is much narrower than that of pattern cells. Figure 8 shows minor peaks for component cells near 180 deg. This arises because in some component cells narrowband textures produced reliable responses at directions opposite to that preferred by the neuron, as described above. Figure 8 shows that for broadband textures pattern cells out-perform component cells. This is because the tuning curve amplitude of pattern cells increases substantially for larger orientation bandwidths, but tuning curve bandwidth is little changed by orientation bandwidth. Large orientation bandwidths also lead to a reduction in the maximum information carried by component cells, because their tuning curves broaden (and hence have lower slope) for broadband textures.

Figure 8 compares peak Fisher information (the maximum obtained across all directions) for all neurons in our sample (Fig. 8) and for each of three subgroups (Fig. 8). Across the population of cells, peak Fisher information decreased slightly, from 0.0013 deg2 (at an average of 33.9 deg) for narrowband textures to 0.0010 deg2 (at 46.3 deg) for broadband textures. Across the population of neurons, Fisher information shows non-monotonic dependence on orientation bandwidth: Fig. 8 shows that this is because peak Fisher information is greater in component cells when the texture has narrow orientation bandwidth, but greater in pattern cells when the texture has broad bandwidth, with a switch-over near orientation bandwidths of 37 deg. ANOVA showed significant effects of orientation bandwidth (F1.69,109.821= 3.807, P= 0.032), and a significant interaction between bandwidth and functional subclass (F3.38,109.821= 3.282, P= 0.019).

In summary, variance-to-mean ratio declined with increasing orientation bandwidth for both component and pattern cells. Pattern and component cells nevertheless provide complementary precision across orientation bandwidth. For narrowband textures, component cells provide more precise motion signals than pattern cells. For broadband textures, pattern cells provide more precise motion signals.

Response to images of natural scenes

The synthetic images used above lack strong contours, because the phases of their various frequency components are random. This raises the question of whether the differences between component and pattern cells will be preserved during the motion of real scenes, in which there are often strong contours. To begin to establish this we measured direction tuning curves for a small subset of natural images selected from van Hateren's database (van Hateren & van der Schaaf, 1998). Images were selected because the distribution of Fourier amplitude over orientation was highly correlated to that of the synthetic images we have used so far (see Methods; Goddard et al. 2008). We therefore label the images by the standard deviation of the orientation spectra with which they were well correlated. Note, however, that the true distribution over orientation may vary across spatial frequency within an image, and across images. We measured response to the same set of images for every neuron, thereby establishing the variability of responses as well as tuning curves. Figure 9 shows 4 of the 9 images used. Figure 9 shows one where the Fourier spectra correlated strongly to that used to generate synthetic textures of bandwidth 6 deg; Fig. 9 shows two of the images that were well correlated with a bandwidth of 25 deg; Fig. 9 shows one of the images well correlated with a bandwidth of 50 deg. Direction tuning curves for each image were measured as for the synthetic textures, by moving the image orthogonal to its dominant orientation. The neurons studied here include 10 pattern cells, 18 component cells and 14 unclassifiable cells; 28 of the 42 neurons were also studied with synthetic images, as above (Table 1).

Figure 9

Example images of natural scenes

Images were selected because the distribution of amplitude over orientation showed strong linear correlation with a wrapped Gaussian of bandwidth 6 deg (A), bandwidth 25 deg (B and C), and bandwidth 50 deg (D). Conventions as in Fig. 1, except that the surfaces are defined by the locations in this space where the Fourier amplitude had decreased to 10% of the maximum.

Figure 10 shows, for component and pattern cells, respectively, average direction tuning curves for each of the images shown in Fig. 9. Direction tuning curves are unimodal and generally symmetric around the peak; for pattern cells, motion in the anti-preferred direction clearly suppresses response below the maintained rate. Some aspects of the direction tuning curves depend on the image used to obtain it: direction tuning curves for the image in Fig. 9 are similar for both pattern and component cells; direction tuning curves for the image in Fig. 9 are lower in amplitude and tighter in bandwidth for component cells than for pattern cells. Because we are interested in the overall trends in the following we nevertheless collapse across images with the same nominal bandwidth. Figure 10 shows, in the same format as Fig. 7, tuning curve amplitude and bandwidth for each image. There are some similarities with the data shown in Fig. 7. First, for component cells response bandwidth is narrower for narrowband images than for broadband images. Second, tuning curve bandwidth of pattern and unclassifiable cells is generally stable against changes in stimulus bandwidth. Third, tuning curve amplitude of pattern cells is higher for broadband images than for narrowband images. ANOVA showed significant dependence of tuning curve amplitude and bandwidth on image orientation bandwidth (respectively: F2,78= 17.782, P < 0.001; F2,54= 4.596, P= 0.014); tuning curve amplitude, but not bandwidth, showed an interaction between subgroups (F4,78= 2.370, P= 0.06).

Figure 10

Response of neurons in area MT to moving images of natural scenes

A, average response of component cells (n= 18) to each image of Fig. 9, as indicated in the legend. Conventions otherwise as in Fig. 7. B, average response of all pattern cells (n= 10) to each image. Conventions as in A. C, response amplitude of all neurons as a function of nominal orientation bandwidth of the image. Conventions as in Fig. 7. D, response amplitude of component (n= 18), unclassifiable (n= 14) and pattern (n= 10) cells. E, tuning curve bandwidth of all neurons as a function of nominal orientation bandwidth. Conventions as in Fig. 7. F, tuning curve bandwidth of pattern, component and unclassified cells. Error bars show ±1 SEM.

Figure 11 shows, in the same format as Fig. 8, how spike count variability and Fisher information depend on the bandwidth of these images. As for synthetic textures, response variability is reduced during presentation of high bandwidth images (Fig. 11). ANOVA showed significant dependence of response variability on texture bandwidth (F2,78= 9.652, P < 0.001) but not functional subgroup. Across the population of neurons Fisher information is not obviously dependent on orientation bandwidth (Fig. 11). Component cells out-perform pattern cells for images of the narrowest bandwidth (Fig. 11; component cells: 0.0016 deg2 at 39.8 deg; pattern cells: 0.0006 deg2 at 47.4 deg). This advantage is retained, but is less, for broadband images (Fig. 11; component cells: 0.0011 deg2 at 44.2 deg; pattern cells: 0.0008 deg2 at 44.8 deg). ANOVA showed no significant dependence of Fisher information on orientation bandwidth or functional subgroup.

Figure 11

Response variability and capacity of neurons to discriminate motion direction of images

Same neurons as in Fig. 10. A, variance-to-mean ratio (Fano Factor; open symbols) of spike count, as a function of nominal orientation bandwidth, across all neurons in the population. Conventions as in Fig. 8. B, variance-to-mean ratio for component, unclassifiable and pattern cells. Conventions as in A. C, maximum Fisher information across motion directions, for all neurons, as a function of texture orientation bandwidth. Conventions as in Fig. 8. D, maximum Fisher information across motion direction, for component, unclassifiable and pattern cells. Conventions as in C. E, average Fisher information for component (dashed line) and pattern (continuous line) cell populations as a function of the direction of motion of a narrowband image (nominal bandwidth 6 deg). Conventions as in Fig. 8. F, average Fisher information for broadband (nominal bandwidth 50 deg) image. Conventions as in E.

In summary, the motion selectivity of pattern cells seems largely invariant to the spatial structure of natural images. The motion selectivity of component cells is less stable. Despite the fact that we have used a small subset of images whose spectra are only partially controlled, most aspects of response depend on the nominal orientation bandwidth of those images in the same way they do for synthetic images.

Discussion

Our measurements show how the direction signals of neurons in area MT depend on the spatial texture of a rigidly moving surface. How spatial structure influences the shape and amplitude of a neuron's direction tuning curve depends on which functional category the neuron belongs to: in component cells tuning width increases with the range of orientations present in the stimulus, but response amplitude is unchanged; in pattern cells response amplitude increases but tuning width is little affected. In addition to these changes, response variability in all neurons depends on spatial texture, and is lowest when the visual stimulus is rich in texture.

Stability of motion signals against changes in spatial texture

By definition, pattern cells in area MT are capable of extracting orientation-independent motion from the orientation-dependent signals provided by fundamental motion detectors in area V1 (Movshon et al. 1985). Consistent with this we have shown that (i) the width of the direction tuning curve of pattern cells is broad and largely stable with variation in range of orientations included in the texture, and (ii) the preferred direction is the same when the texture moves parallel, or orthogonal, to its dominant orientation. Such stability of pattern cell direction tuning against orientation content is reminiscent of the contrast independence of spatial tuning in retina, lateral geniculate nucleus and V1 (Tolhurst & Movshon, 1975; Shapley & Victor, 1979; Dean, 1981; Albrecht & Hamilton, 1982; Sclar & Freeman, 1982; Solomon & Lennie, 2005; Scholl et al. 2012).

The direction tuning of component cells was much more dependent on the spatial components of the stimulus. Like pattern cells, component cells must pool the signals of many direction-selective cells in V1, because their receptive fields are much larger than those found in V1. Component cells are nevertheless incapable of providing motion signals that are stable in the face of variation in the spatial texture of surfaces: the preferred motion direction of component cells depended on the dominant orientation of the stimulus, and tuning bandwidth increased with the range of orientations present in the stimulus. That is, component cells are responding to the motion of elements in the texture that align to the orientation axis of the neuron. These dependences of component cells are those expected for DS cells in V1, as is shown by a comparison of the observed component cell responses with the responses of V1 DS cells that are predicted by the model of Simoncelli and Heeger (1998). These observations are therefore consistent with the idea that component cells get input from many DS cells in V1, whose receptive fields are spread over the retina, but all have similar direction tuning (Rust et al. 2006). Component cells do not show the stability inherent in pattern cell responses because those computations require neurons to appropriately weigh the signals of fundamental motion detectors that span a large range of preferred directions, and spatial and temporal frequencies.

Response of real neurons and model neurons

The predictions of the Simoncelli and Heeger (1998) model show strong similarities to the observed neural responses, but there are some discrepancies. First, tuning bandwidth of model pattern cells decreases at large bandwidths. This discrepancy may arise if the stimulus is not tailored to the receptive field of the neuron under study (it could be precisely tailored for the simulations). In additional simulations not shown we either halved or doubled the stimulus speed. For model V1 neurons, response amplitude and bandwidth showed similar dependence on stimulus bandwidth to that seen in Fig. 4. The response of model pattern cells was more dependent on stimulus speed, as follows. Reducing stimulus speed brought about bimodal direction tuning curves (with peaks up to 120 deg apart) for stimulus orientation bandwidths of less than 50 deg (see related discussion in Simoncelli & Heeger, 1998, p. 757). We did not see this type of bimodal response in our sample of pattern cells. Increasing speed made tuning bandwidth of model pattern cells less dependent on stimulus orientation bandwidth, and markedly increased response amplitude for textures of large bandwidth. Thus, systematic manipulation of texture speed may bring model predictions and pattern cell response into better quantitative agreement.

The impact of orientation bandwidth on the response of component and pattern cells was similar for synthetic textures and images of natural scenes, and these were generally captured well by a modified version of the Simoncelli and Heeger (1998) model (Rust et al. 2006). These observations are consistent with recent work showing that responses during viewing of nearly natural movies are generally well captured by models like that used in our simulations (Nishimoto & Gallant, 2011). That study, however, did not obtain responses to gratings and plaids, and relied on model simulations to characterise the receptive fields of neurons as pattern- or component-like; they report a lack of clear pattern-like responses in area MT. The reason for the lack of clear (simulated) pattern-like responses in Nishimoto and Gallant (2011) is unclear; it may be related to our observation that pattern cells in area MT show markedly increased response to textures of large orientation bandwidth. In model simulations not shown we found that the response amplitude of both model V1 DS cells and model MT pattern cells was substantially reduced at large orientation bandwidths when tuned normalisation was omitted from the V1 receptive fields. Tuned normalisation was not included in the model of Nishimoto and Gallant (2011). Resolution of this issue requires measurements from V1 neurons, and recent work shows potentially analogous effects of stimulus orientation bandwidth among neurons in area V1 of macaque (Goris et al. 2013).

The second discrepancy between our data and the model predictions is that when textures drifted parallel to their dominant orientation, pattern cells in area MT showed broad tuning curves and little response to motion along the anti-preferred motion direction. This is different from the model predictions, where tuning curves are narrow and there is robust response along the anti-preferred motion direction. In the model, response to anti-preferred motion arises because inputs to pattern cells include fundamental motion detectors that prefer static or very low temporal frequencies and are aligned to the dominant orientation of the texture. That pattern cells in area MT of marmoset do not respond in the manner predicted by the model is consistent with the idea that the receptive fields of pattern cells only partially cover the velocity plane, similar to observations in macaque (Nishimoto and Gallant, 2011). Broad tuning curves for parallel motion may arise if there is also less inhibitory input from fundamental motion detectors that prefer low temporal frequencies.

Finally, some neurons in area MT showed changes in preferred direction with texture bandwidth, or when comparing response to gratings and plaids. We do not know the reason for this shift in preferred direction. It may arise if the distribution of inhibitory or suppressive inputs to the receptive field is not aligned with the distribution of excitatory inputs, if excitatory inputs are not distributed along a single velocity plane, or through a combination of both factors.

Influence of spatial texture on response variability of neurons in area MT

One aim of this work was to establish if response variability depends on the spatial structure of images. Across the population of neurons, and within each subtype, variance-to-mean ratio declined with increasing texture bandwidth. The stimulus dependence of variability is consistent with recent work showing that, in primary visual cortex of cat, variability of spiking activity is lower for natural movies than for gratings (Herikstad et al. 2011). In superior colliculus of cat, and medial superior temporal area of macaque, variability of spiking activity depends on the velocity of visual stimuli (Mochol et al. 2010; Brostek et al. 2012), and is lower when the stimulus moves quickly. The stimulus dependence of variability seen here does not obviously depend on the mean spike count – it persisted even when we matched the distribution of mean spike count across stimuli. Additionally, component cells, whose overall response amplitude does not depend on stimulus bandwidth, showed the same effect.

In area V1 of cat and macaque, response variability is reduced, and neuronal response becomes sparser, when a stimulus is made large (Vinje & Gallant, 2000, 2002; Haider et al. 2010). These changes arise because large images recruit greater extra-classical inhibition, or suppression, from regions surrounding the classical receptive field (Haider et al. 2010). Increases in texture orientation bandwidth and increases in stimulus size might be expected to induce similar changes in V1 response, because both stimulus manipulations should increase the proportion of the cortical network that is active. Neurons in area V1, however, have been described as more responsive to natural images than they are to gratings or random noise of the same size (Rainer et al. 2001; Kayser et al. 2003; Felsen et al. 2005; but see Herikstad et al. 2011). Where it has been directly compared, in area V1 of macaque, response sparseness is similar for gratings and natural images (Vinje & Gallant, 2000). In area MT, response amplitude increased with texture orientation bandwidth (for pattern cells), or response bandwidth increased (component cells). Neither of these observations is consistent with the idea that rich textures or natural images increase response sparseness in area MT. Neurons in area MT draw inputs from many afferents, and this may override the impact of increased sparseness in V1 afferents, but resolution of this requires greater knowledge of the impact of spatial texture on the responses of neurons in area V1.

Where does this stimulus-dependent reduction in variability arise? We cannot rule out the possibility that response variability is reduced by large texture bandwidths even in neurons early in the visual pathway (in the retina, or lateral geniculate nucleus). We think this is unlikely, however, because those neurons are generally insensitive to the orientation of contours, and the standard deviation of luminance values was the same for all textures. A more attractive explanation is that textures with large orientation bandwidths drive a greater number of orientation-sensitive afferents, and therefore a larger fraction of the inputs, to direction-sensitive neurons. Some of the response variability of orientation-sensitive neurons may be correlated, but these correlations are higher if the neurons have similar orientation preferences (Smith & Kohn, 2008). Activation of a larger number of afferent inputs may allow some of the variability, and co-variability, to be ‘averaged out’, and thus reduce response variability in direction-sensitive neurons. As explained above, additional reduction in variability may be achieved if textures with large orientation bandwidth recruit more inhibition as well as more excitation. These computations may be done in area V1, but similar arguments can be made if direction-selective neurons in V1 share response variability, and converge onto neurons in area MT.

Recent work shows that increasing the spatial richness of textures produces ocular following responses that are faster and more reliable (Simoncini et al. 2012). These dependencies may have a corollary in the responses of neurons studied here: richer textures were associated with increased response amplitude (primarily in pattern cells) and lower response variability (in all neurons). The different impact of spatial texture on direction tuning curves in pattern and component cells may be useful in elucidating their contribution to perceptual judgments and eye movements.

Complementary signals of component and pattern cells

The mechanisms underlying pattern and component cell computations are thought to be similar: pattern cell computations appear to require access to signals from a wider range of fundamental motion detectors (Rust et al. 2006), and may also rely on receptive fields that are particularly susceptible to end stopping (Tsui et al. 2010). The differences between component and pattern cells may therefore be explained by quantitative variation in receptive field properties along a continuum, and component cells may simply exist because they draw differently on inputs from thalamic, V1 or extra-striate areas, or from distinct cell types within these regions. But while the importance of the pattern cell computation to motion vision is clear, the contribution of component cell computations is not. It may be that pattern and component cells have different projections, but there is as yet no evidence for this.

Among pattern cells, the increase in response amplitude at large stimulus bandwidths is accompanied by a large increase in Fisher information: for rich spatial textures more informative responses are found among pattern cells. Across component cells, however, Fisher information decreased with increasing orientation content of the stimulus: for simple contours more informative responses are found among component cells. These observations are consistent with the idea that pattern cells pool inputs over many component motions: when a stimulus has narrower bandwidth than a receptive field, the receptive field effectively blurs the stimulus. The consequence of this is that for contours moving approximately orthogonal to their dominant orientation, component cells provide more precise motion signals than pattern cells. Pattern cells, by contrast, provide more precise motion signals when the image is rich in spatial texture, regardless of the relationship between orientation and motion direction. Component and pattern cells may therefore coexist in area MT because they provide complementary and parallel computations. Future work may show whether and how their signals are combined to exploit this.