Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain

A critical step in the interpretation of the visual world is the integration of the various local motion signals generated by moving objects. This process is complicated by the fact that local velocity measurements can differ depending on contour orientation and spatial position. Specifically, any local motion detector can measure only the component of motion perpendicular to a contour that extends beyond its field of view. This "aperture problem" is particularly relevant to direction-selective neurons early in the visual pathways, where small receptive fields permit only a limited view of a moving object. Here we show that neurons in the middle temporal visual area (known as MT or V5) of the macaque brain reveal a dynamic solution to the aperture problem. MT neurons initially respond primarily to the component of motion perpendicular to a contour's orientation, but over a period of approximately 60 ms the responses gradually shift to encode the true stimulus direction, regardless of orientation. We also report a behavioural correlate of these neural responses: the initial velocity of pursuit eye movements deviates in a direction perpendicular to local contour orientation, suggesting that the earliest neural responses influence the oculomotor response.     

Dynamic coding of behaviourally relevant stimuli in parietal cortex.

A general function of cerebral cortex is to allow the flexible association of sensory stimuli with specific behaviours. Many neurons in parietal, prefrontal and motor cortical areas are activated both by particular movements and by sensory cues that trigger these movements, suggesting a role in linking sensation to action. For example, neurons in the lateral intraparietal area (LIP) encode both the location of visual stimuli and the direction of saccadic eye movements. LIP is not believed to encode non-spatial stimulus attributes such as colour. Here we investigated whether LIP would encode colour if colour was behaviourally linked to the eye movement. We trained monkeys to make an eye movement in one of two directions based alternately on the colour or location of a visual cue. When cue colour was relevant for directing eye movement, we found a substantial fraction of LIP neurons selective for cue colour. However, when cue location was relevant, colour selectivity was virtually absent in LIP. These results demonstrate that selectivity of cortical neurons can change as a function of the required behaviour.     

Feature-based attention influences motion processing gain in macaque visual cortex.

Changes in neural responses based on spatial attention have been demonstrated in many areas of visual cortex, indicating that the neural correlate of attention is an enhanced response to stimuli at an attended location and reduced responses to stimuli elsewhere. Here we demonstrate non-spatial, feature-based attentional modulation of visual motion processing, and show that attention increases the gain of direction-selective neurons in visual cortical area MT without narrowing the direction-tuning curves. These findings place important constraints on the neural mechanisms of attention and we propose to unify the effects of spatial location, direction of motion and other features of the attended stimuli in a 'feature similarity gain model' of attention.     

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.     

Cortical microstimulation influences perceptual judgements of motion direction

Neurons in the visual cortex respond selectively to perceptually salient features of the visual scene, such as the direction and speed of moving objects, the orientation of local contours, or the colour or relative depth of a visual pattern. It is commonly assumed that the brain constructs its percept of the visual scene from information encoded in the selective responses of such neurons. We have now tested this hypothesis directly by measuring the effect on psychophysical performance of modifying the firing rates of physiologically characterized neurons. We required rhesus monkeys to report the direction of motion in a visual display while we electrically stimulated clusters of directionally selective neurons in the middle temporal visual area (MT, or V5), an extrastriate area that plays a prominent role in the analysis of visual motion information. Microstimulation biased the animals' judgements towards the direction of motion encoded by the stimulated neurons. This result indicates that physiological properties measured at the neuronal level can be causally related to a specific aspect of perceptual performance.     

Neuronal switching of sensorimotor transformations for antisaccades

The influence of cognitive context on orienting behaviour can be explored using the mixed memory-prosaccade, memory-antisaccade task. A symbolic cue, such as the colour of a visual stimulus, instructs the subject to make a brief, rapid eye movement (a saccade) either towards the stimulus (prosaccade) or in the opposite direction (antisaccade). Thus, the appropriate sensorimotor transformation must be switched on to execute the instructed task. Despite advances in our understanding of the neuronal processing of antisaccades, it remains unclear how the brain selects and computes the sensorimotor transformation leading to an antisaccade. Here we show that area LIP of the posterior parietal cortex is involved in these processes. LIP's population activity turns from the visual direction to the motor direction during memory-antisaccade trials. About one-third of the visual neurons in LIP produce a brisk, transient discharge in certain memory-antisaccade trials. We call this discharge 'paradoxical' because its timing is visual-like but its direction is motor. The paradoxical discharge shows, first, that switching occurs already at the level of visual cells, as previously proposed by Schlag-Rey and colleagues; and second, that this switching is accomplished very rapidly, within 50 ms from the arrival of the visual signals in LIP.     

Motion direction, speed and orientation in binocular matching

The spatial differences between the images seen by the two eyes, called binocular disparities, can be used to recover the volumetric (three-dimensional) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this matching problem is one of the main issues in stereoscopic vision. Many cortical neurons in visual areas V1 (ref. 2), MT (refs 3, 4) and MST (refs 5, 6) that are tuned to binocular disparity are also tuned to orientation, motion direction and speed. Although psychophysical work has shown that motion direction can facilitate binocular matching, the psychophysical literature on the role of orientation is mixed, and it has been argued that speed differences are ineffective in aiding correspondence. Here we use a different psychophysical paradigm to show that the visual system uses similarities in orientation, motion direction and speed to achieve binocular correspondence. These results indicate that cells that multiplex orientation, motion direction, speed and binocular disparity may help to solve the binocular matching problem.     

Neural synchrony correlates with surface segregation rules

To analyse an image, the visual system must decompose the scene into its relevant parts. Identifying distinct surfaces is a basic operation in such analysis, and is believed to precede object recognition. Two superimposed gratings moving in different directions (plaid stimuli) may be perceived either as two surfaces, one being transparent and sliding on top of the other (component motion) or as a single pattern whose direction of motion is intermediate to the component vectors (pattern motion). The degree of transparency, and hence the perception, can be manipulated by changing only the luminance of the grating intersections. Here we show that neurons in two visual cortical areas--A18 and PMLS--synchronize their discharges when responding to contours of the same surface but not when responding to contours belonging to different surfaces. The amplitudes of responses correspond to previously described rate predictions for component and pattern motion, but, in contrast to synchrony, failed to reflect the transition from component to pattern motion induced by manipulating the degree of transparency. Thus, dynamic changes in synchronization could encode, in a context-dependent way, relations among simultaneous responses to spatially superimposed contours and thereby bias their association with distinct surfaces.     

Neural mechanisms of unconscious visual motion priming

The neural correlates of the motion priming were examined in normal young subjects using event-related brain potentials (ERPs) and functional magnetic resonance imaging (fMRI). Visual motion perception can be uncon-sciously biased in favor of a particular direction by a pre-ceding motion in that direction. Motion priming first in-volved an enhancement of ERP amplitude about 100 ms fol-lowing the onset of motion. The amplitudes of ERP compo-nents after 350 ms were also increased. The fMRI results suggest that the early-latency effect reflects modulation of neural responses in extrastriate cortex. Higher-level visual processing areas, including cortical regions MT/MST and the intraparietal cortices were also activated. The findings provide direct evidence that unconscious priming of motion perception is the result of interaction of direction-selective neural responses to motion stimuli. The results cannot be accounted for by refractoriness of neural responses, but in-stead support a theory of motion priming based on motion opponency, as proposed in computational models.     

Dynamic properties of neurons in cortical area MT in alert and anaesthetized macaque monkeys.

In order to see the world with high spatial acuity, an animal must sample the visual image with many detectors that restrict their analyses to extremely small regions of space. The visual cortex must then integrate the information from these localized receptive fields to obtain a more global picture of the surrounding environment. We studied this process in single neurons within the middle temporal visual area (MT) of macaques using stimuli that produced conflicting local and global information about stimulus motion. Neuronal responses in alert animals initially reflected predominantly the ambiguous local motion features, but gradually converged to an unambiguous global representation. When the same animals were anaesthetized, the integration of local motion signals was markedly impaired even though neuronal responses remained vigorous and directional tuning characteristics were intact. Our results suggest that anaesthesia preferentially affects the visual processing responsible for integrating local signals into a global visual representation.     

Motion streaks provide a spatial code for motion direction.

Although many neurons in the primary visual cortex (V1) of primates are direction selective, they provide ambiguous information about the direction of motion of a stimulus. There is evidence that one of the ways in which the visual system resolves this ambiguity is by computing, from the responses of V1 neurons, velocity components in two or more spatial orientations and then combining these velocity components. Here I consider another potential neural mechanism for determining motion direction. When a localized image feature moves fast enough, it should become smeared in space owing to temporal integration in the visual system, creating a spatial signal-a 'motion streak'-oriented in the direction of the motion. The orientation masking and adaptation experiments reported here show that these spatial signals for motion direction exist in the human visual system for feature speeds above about 1 feature width per 100 ms. Computer simulations show that this psychophysical finding is consistent with the known response properties of V1 neurons, and that these spatial signals, when appropriately processed, are sufficient to determine motion direction in natural images.     

A new perceptual illusion reveals mechanisms of sensory decoding

Perceptual illusions are usually thought to arise from the way sensory signals are encoded by the brain, and indeed are often used to infer the mechanisms of sensory encoding. But perceptual illusions might also result from the way the brain decodes sensory information, reflecting the strategies that optimize performance in particular tasks. In a fine discrimination task, the most accurate information comes from neurons tuned away from the discrimination boundary, and observers seem to use signals from these 'displaced' neurons to optimize their performance. We wondered whether using signals from these neurons might also bias perception. In a fine direction discrimination task using moving random-dot stimuli, we found that observers' perception of the direction of motion is indeed biased away from the boundary. This misperception can be accurately described by a decoding model that preferentially weights signals from neurons whose responses best discriminate those directions. In a coarse discrimination task, to which a different decoding rule applies, the same stimulus is not misperceived, suggesting that the illusion is a direct consequence of the decoding strategy that observers use to make fine perceptual judgments. The subjective experience of motion is therefore not mediated directly by the responses of sensory neurons, but is only developed after the responses of these neurons are decoded.     

Selective deficit of visual search in moving displays after extrastriate damage

A visual cue that is often associated with significant stimuli, such as those provided by prey and predators, is movement relative to the observer. An efficient visual system should be able to direct attention to those parts of the visual field that contain such stimuli. What is needed is a system that can filter by movement difference. This could direct attention to a moving item among stationary items, or an item moving in one direction against a background moving in a different direction. Visual search experiments have shown that people are indeed able to filter by movement; that is, they can attend to just the moving items in arrays of moving and stationary stimuli. Single-cell recordings from monkey visual cortex show that the medial temporal cortical area (MT) has some of the properties required to filter by movement. We have now linked these two observations by showing that a patient with bilateral lesions to the presumed human homologue of MT cannot restrict visual attention to the moving items in arrays of both moving and stationary items. This suggests that MT is the site of a movement filter used in normal visual processing.     

Hearing visual motion in depth

Auditory spatial perception is strongly affected by visual cues. For example, if auditory and visual stimuli are presented synchronously but from different positions, the auditory event is mislocated towards the locus of the visual stimulus-the ventriloquism effect. This 'visual capture' also occurs in motion perception in which a static auditory stimulus appears to move with the visual moving object. We investigated how the human perceptual system coordinates complementary inputs from auditory and visual senses. Here we show that an auditory aftereffect occurs from adaptation to visual motion in depth. After a few minutes of viewing a square moving in depth, a steady sound was perceived as changing loudness in the opposite direction. Adaptation to a combination of auditory and visual stimuli changing in a compatible direction increased the aftereffect and the effect of visual adaptation almost disappeared when the directions were opposite. On the other hand, listening to a sound changing in intensity did not affect the visual changing-size aftereffect. The results provide psychophysical evidence that, for processing of motion in depth, the auditory system responds to both auditory changing intensity and visual motion in depth.     

Top-down gain control of the auditory space map by gaze control circuitry in the barn owl

High-level circuits in the brain that control the direction of gaze are intimately linked with the control of visual spatial attention. Immediately before an animal directs its gaze towards a stimulus, both psychophysical sensitivity to that visual stimulus and the responsiveness of high-order neurons in the cerebral cortex that represent the stimulus increase dramatically. Equivalent effects on behavioural sensitivity and neuronal responsiveness to visual stimuli result from focal electrical microstimulation of gaze control centres in monkeys. Whether the gaze control system modulates neuronal responsiveness in sensory modalities other than vision is unknown. Here we show that electrical microstimulation applied to gaze control circuitry in the forebrain of barn owls regulates the gain of midbrain auditory responses in an attention-like manner. When the forebrain circuit was activated, midbrain responses to auditory stimuli at the location encoded by the forebrain site were enhanced and spatial selectivity was sharpened. The same stimulation suppressed responses to auditory stimuli represented at other locations in the midbrain map. Such space-specific, top-down regulation of auditory responses by gaze control circuitry in the barn owl suggests that the central nervous system uses a common strategy for dynamically regulating sensory gain that applies across modalities, brain areas and classes of vertebrate species. This approach provides a path for discovering mechanisms that underlie top-down gain control in the central nervous system.     

Directionally selective calcium signals in dendrites of starburst amacrine cells

The detection of image motion is fundamental to vision. In many species, unique classes of retinal ganglion cells selectively respond to visual stimuli that move in specific directions. It is not known which retinal cell first performs the neural computations that give rise to directional selectivity in the ganglion cell. A prominent candidate has been an interneuron called the 'starburst amacrine cell'. Using two-photon optical recordings of intracellular calcium concentration, here we find that individual dendritic branches of starburst cells act as independent computation modules. Dendritic calcium signals, but not somatic membrane voltage, are directionally selective for stimuli that move centrifugally from the cell soma. This demonstrates that direction selectivity is computed locally in dendritic branches at a stage before ganglion cells.     

Experience with moving visual stimuli drives the early development of cortical direction selectivity

The onset of vision occurs when neural circuits in the visual cortex are immature, lacking both the full complement of connections and the response selectivity that defines functional maturity. Direction-selective responses are particularly vulnerable to the effects of early visual deprivation, but it remains unclear how stimulus-driven neural activity guides the emergence of cortical direction selectivity. Here we report observations from a motion training protocol that allowed us to monitor the impact of experience on the development of direction-selective responses in visually naive ferrets. Using intrinsic signal imaging techniques, we found that training with a single axis of motion induced the rapid emergence of direction columns that were confined to cortical regions preferentially activated by the training stimulus. Using two-photon calcium imaging techniques, we found that single neurons in visually naive animals exhibited weak directional biases and lacked the strong local coherence in the spatial organization of direction preference that was evident in mature animals. Training with a moving stimulus, but not with a flashed stimulus, strengthened the direction-selective responses of individual neurons and preferentially reversed the direction biases of neurons that deviated from their neighbours. Both effects contributed to an increase in local coherence. We conclude that early experience with moving visual stimuli drives the rapid emergence of direction-selective responses in the visual cortex.     

Microstimulation of inferotemporal cortex influences face categorization

The inferior temporal cortex (IT) of primates is thought to be the final visual area in the ventral stream of cortical areas responsible for object recognition. Consistent with this hypothesis, single IT neurons respond selectively to highly complex visual stimuli such as faces. However, a direct causal link between the activity of face-selective neurons and face perception has not been demonstrated. In the present study of macaque monkeys, we artificially activated small clusters of IT neurons by means of electrical microstimulation while the monkeys performed a categorization task, judging whether noisy visual images belonged to 'face' or 'non-face' categories. Here we show that microstimulation of face-selective sites, but not other sites, strongly biased the monkeys' decisions towards the face category. The magnitude of the effect depended upon the degree of face selectivity of the stimulation site, the size of the stimulated cluster of face-selective neurons, and the exact timing of microstimulation. Our results establish a causal relationship between the activity of face-selective neurons and face perception.     

Gamma-band synchronization in visual cortex predicts speed of change detection

Our capacity to process and respond behaviourally to multiple incoming stimuli is very limited. To optimize the use of this limited capacity, attentional mechanisms give priority to behaviourally relevant stimuli at the expense of irrelevant distractors. In visual areas, attended stimuli induce enhanced responses and an improved synchronization of rhythmic neuronal activity in the gamma frequency band (40-70 Hz). Both effects probably improve the neuronal signalling of attended stimuli within and among brain areas. Attention also results in improved behavioural performance and shortened reaction times. However, it is not known how reaction times are related to either response strength or gamma-band synchronization in visual areas. Here we show that behavioural response times to a stimulus change can be predicted specifically by the degree of gamma-band synchronization among those neurons in monkey visual area V4 that are activated by the behaviourally relevant stimulus. When there are two visual stimuli and monkeys have to detect a change in one stimulus while ignoring the other, their reactions are fastest when the relevant stimulus induces strong gamma-band synchronization before and after the change in stimulus. This enhanced gamma-band synchronization is also followed by shorter neuronal response latencies on the fast trials. Conversely, the monkeys' reactions are slowest when gamma-band synchronization is high in response to the irrelevant distractor. Thus, enhanced neuronal gamma-band synchronization and shortened neuronal response latencies to an attended stimulus seem to have direct effects on visually triggered behaviour, reflecting an early neuronal correlate of efficient visuo-motor integration.     

Molecular identification of a retinal cell type that responds to upward motion

The retina contains complex circuits of neurons that extract salient information from visual inputs. Signals from photoreceptors are processed by retinal interneurons, integrated by retinal ganglion cells (RGCs) and sent to the brain by RGC axons. Distinct types of RGC respond to different visual features, such as increases or decreases in light intensity (ON and OFF cells, respectively), colour or moving objects. Thus, RGCs comprise a set of parallel pathways from the eye to the brain. The identification of molecular markers for RGC subsets will facilitate attempts to correlate their structure with their function, assess their synaptic inputs and targets, and study their diversification. Here we show, by means of a transgenic marking method, that junctional adhesion molecule B (JAM-B) marks a previously unrecognized class of OFF RGCs in mice. These cells have asymmetric dendritic arbors aligned in a dorsal-to-ventral direction across the retina. Their receptive fields are also asymmetric and respond selectively to stimuli moving in a soma-to-dendrite direction; because the lens reverses the image of the world on the retina, these cells detect upward motion in the visual field. Thus, JAM-B identifies a unique population of RGCs in which structure corresponds remarkably to function.