Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain

A critical step in the interpretation of the visual world is the integration of the various local motion signals generated by moving objects. This process is complicated by the fact that local velocity measurements can differ depending on contour orientation and spatial position. Specifically, any local motion detector can measure only the component of motion perpendicular to a contour that extends beyond its field of view. This "aperture problem" is particularly relevant to direction-selective neurons early in the visual pathways, where small receptive fields permit only a limited view of a moving object. Here we show that neurons in the middle temporal visual area (known as MT or V5) of the macaque brain reveal a dynamic solution to the aperture problem. MT neurons initially respond primarily to the component of motion perpendicular to a contour's orientation, but over a period of approximately 60 ms the responses gradually shift to encode the true stimulus direction, regardless of orientation. We also report a behavioural correlate of these neural responses: the initial velocity of pursuit eye movements deviates in a direction perpendicular to local contour orientation, suggesting that the earliest neural responses influence the oculomotor response.     

Perceptual consequences of centre-surround antagonism in visual motion processing

Centre-surround receptive field organization is a ubiquitous property in mammalian visual systems, presumably tailored for extracting image features that are differentially distributed over space. In visual motion, this is evident as antagonistic interactions between centre and surround regions of the receptive fields of many direction-selective neurons in visual cortex. In a series of psychophysical experiments we make the counterintuitive observation that increasing the size of a high-contrast moving pattern renders its direction of motion more difficult to perceive and reduces its effectiveness as an adaptation stimulus. We propose that this is a perceptual correlate of centre-surround antagonism, possibly within a population of neurons in the middle temporal visual area. The spatial antagonism of motion signals observed at high contrast gives way to spatial summation as contrast decreases. Evidently, integration of motion signals over space depends crucially on the visibility of those signals, thereby allowing the visual system to register motion information efficiently and adaptively.     

Mapping multiple features in the population response of visual cortex

Stimulus features such as edge orientation, motion direction and spatial frequency are thought to be encoded in the primary visual cortex by overlapping feature maps arranged so that the location of neurons activated by a particular combination of stimulus features can be predicted from the intersections of these maps. This view is based on the use of grating stimuli, which limit the range of stimulus combinations that can be examined. We used optical imaging of intrinsic signals in ferrets to assess patterns of population activity evoked by the motion of a texture (a field of iso-oriented bars). Here we show that the same neural population can be activated by multiple combinations of orientation, length, motion axis and speed. Rather than reflecting the intersection of multiple maps, our results indicate that population activity in primary visual cortex is better described as a single map of spatiotemporal energy.     

Motion direction, speed and orientation in binocular matching

The spatial differences between the images seen by the two eyes, called binocular disparities, can be used to recover the volumetric (three-dimensional) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this matching problem is one of the main issues in stereoscopic vision. Many cortical neurons in visual areas V1 (ref. 2), MT (refs 3, 4) and MST (refs 5, 6) that are tuned to binocular disparity are also tuned to orientation, motion direction and speed. Although psychophysical work has shown that motion direction can facilitate binocular matching, the psychophysical literature on the role of orientation is mixed, and it has been argued that speed differences are ineffective in aiding correspondence. Here we use a different psychophysical paradigm to show that the visual system uses similarities in orientation, motion direction and speed to achieve binocular correspondence. These results indicate that cells that multiplex orientation, motion direction, speed and binocular disparity may help to solve the binocular matching problem.     

Gaze direction controls response gain in primary visual-cortex neurons

To localize objects in space, the brain needs to combine information about the position of the stimulus on the retinae with information about the location of the eyes in their orbits. Interaction between these two types of information occurs in several cortical areas, but the role of the primary visual cortex (area V1) in this process has remained unclear. Here we show that, for half the cells recorded in area V1 of behaving monkeys, the classically described visual responses are strongly modulated by gaze direction. Specifically, we find that selectivity for horizontal retinal disparity-the difference in the position of a stimulus on each retina which relates to relative object distance-and for stimulus orientation may be present at a given gaze direction, but be absent or poorly expressed at another direction. Shifts in preferred disparity also occurred in several neurons. These neural changes were most often present at the beginning of the visual response, suggesting a feedforward gain control by eye position signals. Cortical neural processes for encoding information about the three-dimensional position of a stimulus in space therefore start as early as area V1.     

Dynamic properties of neurons in cortical area MT in alert and anaesthetized macaque monkeys.

In order to see the world with high spatial acuity, an animal must sample the visual image with many detectors that restrict their analyses to extremely small regions of space. The visual cortex must then integrate the information from these localized receptive fields to obtain a more global picture of the surrounding environment. We studied this process in single neurons within the middle temporal visual area (MT) of macaques using stimuli that produced conflicting local and global information about stimulus motion. Neuronal responses in alert animals initially reflected predominantly the ambiguous local motion features, but gradually converged to an unambiguous global representation. When the same animals were anaesthetized, the integration of local motion signals was markedly impaired even though neuronal responses remained vigorous and directional tuning characteristics were intact. Our results suggest that anaesthesia preferentially affects the visual processing responsible for integrating local signals into a global visual representation.     

Cortical microstimulation influences perceptual judgements of motion direction

Neurons in the visual cortex respond selectively to perceptually salient features of the visual scene, such as the direction and speed of moving objects, the orientation of local contours, or the colour or relative depth of a visual pattern. It is commonly assumed that the brain constructs its percept of the visual scene from information encoded in the selective responses of such neurons. We have now tested this hypothesis directly by measuring the effect on psychophysical performance of modifying the firing rates of physiologically characterized neurons. We required rhesus monkeys to report the direction of motion in a visual display while we electrically stimulated clusters of directionally selective neurons in the middle temporal visual area (MT, or V5), an extrastriate area that plays a prominent role in the analysis of visual motion information. Microstimulation biased the animals' judgements towards the direction of motion encoded by the stimulated neurons. This result indicates that physiological properties measured at the neuronal level can be causally related to a specific aspect of perceptual performance.     

Control of visual cortical signals by prefrontal dopamine

The prefrontal cortex is thought to modulate sensory signals in posterior cortices during top-down attention, but little is known about the underlying neural circuitry. Experimental and clinical evidence indicate that prefrontal dopamine has an important role in cognitive functions, acting predominantly through D1 receptors. Here we show that dopamine D1 receptors mediate prefrontal control of signals in the visual cortex of macaques (Macaca mulatta). We pharmacologically altered D1-receptor-mediated activity in the frontal eye field of the prefrontal cortex and measured the effect on the responses of neurons in area V4 of the visual cortex. This manipulation was sufficient to enhance the magnitude, the orientation selectivity and the reliability of V4 visual responses to an extent comparable with the known effects of top-down attention. The enhancement of V4 signals was restricted to neurons with response fields overlapping the part of visual space affected by the D1 receptor manipulation. Altering either D1- or D2-receptor-mediated frontal eye field activity increased saccadic target selection but the D2 receptor manipulation did not enhance V4 signals. Our results identify a role for D1 receptors in mediating the control of visual cortical signals by the prefrontal cortex and suggest how processing in sensory areas could be altered in mental disorders involving prefrontal dopamine.     

Induction of visual orientation modules in auditory cortex

Modules of neurons sharing a common property are a basic organizational feature of mammalian sensory cortex. Primary visual cortex (V1) is characterized by orientation modules--groups of cells that share a preferred stimulus orientation--which are organized into a highly ordered orientation map. Here we show that in ferrets in which retinal projections are routed into the auditory pathway, visually responsive neurons in 'rewired' primary auditory cortex are also organized into orientation modules. The orientation tuning of neurons within these modules is comparable to the tuning of cells in V1 but the orientation map is less orderly. Horizontal connections in rewired cortex are more patchy and periodic than connections in normal auditory cortex, but less so than connections in V1. These data show that afferent activity has a profound influence on diverse components of cortical circuitry, including thalamocortical and local intracortical connections, which are involved in the generation of orientation tuning, and long-range horizontal connections, which are important in creating an orientation map.     

Activation of specific interneurons improves V1 feature selectivity and visual perception

Inhibitory interneurons are essential components of the neural circuits underlying various brain functions. In the neocortex, a large diversity of GABA (γ-aminobutyric acid) interneurons has been identified on the basis of their morphology, molecular markers, biophysical properties and innervation pattern. However, how the activity of each subtype of interneurons contributes to sensory processing remains unclear. Here we show that optogenetic activation of parvalbumin-positive (PV+) interneurons in the mouse primary visual cortex (V1) sharpens neuronal feature selectivity and improves perceptual discrimination. Using multichannel recording with silicon probes and channelrhodopsin-2 (ChR2)-mediated optical activation, we found that increased spiking of PV+ interneurons markedly sharpened orientation tuning and enhanced direction selectivity of nearby neurons. These effects were caused by the activation of inhibitory neurons rather than a decreased spiking of excitatory neurons, as archaerhodopsin-3 (Arch)-mediated optical silencing of calcium/calmodulin-dependent protein kinase IIα (CAMKIIα)-positive excitatory neurons caused no significant change in V1 stimulus selectivity. Moreover, the improved selectivity specifically required PV+ neuron activation, as activating somatostatin or vasointestinal peptide interneurons had no significant effect. Notably, PV+ neuron activation in awake mice caused a significant improvement in their orientation discrimination, mirroring the sharpened V1 orientation tuning. Together, these results provide the first demonstration that visual coding and perception can be improved by increased spiking of a specific subtype of cortical inhibitory interneurons.     

Representation of a perceptual decision in developing oculomotor commands

Behaviour often depends on the ability to make categorical judgements about sensory information acquired over time. Such judgements require a comparison of the evidence favouring the alternatives, but how the brain forms these comparisons is unknown. Here we show that in a visual discrimination task, the accumulating balance of sensory evidence favouring one interpretation over another is evident in the neural circuits that generate the behavioural response. We trained monkeys to make a direction judgement about dynamic random-dot motions and to indicate their judgement with an eye movement to a visual target. We interrupted motion viewing with electrical microstimulation of the frontal eye field and analysed the resulting, evoked eye movements for evidence of ongoing activity associated with the oculomotor response. Evoked eye movements deviated in the direction of the monkey's judgement. The magnitude of the deviation depended on motion strength and viewing time. The oculomotor signals responsible for these deviations reflected the accumulated motion information that informed the monkey's choices on the discrimination task. Thus, for this task, decision formation and motor preparation appear to share a common level of neural organization.     

Experience-dependent representation of visual categories in parietal cortex

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as 'chair', 'table' and 'vehicle', which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360 degrees of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP--an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making-robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.     

Feature-based attention influences motion processing gain in macaque visual cortex.

Changes in neural responses based on spatial attention have been demonstrated in many areas of visual cortex, indicating that the neural correlate of attention is an enhanced response to stimuli at an attended location and reduced responses to stimuli elsewhere. Here we demonstrate non-spatial, feature-based attentional modulation of visual motion processing, and show that attention increases the gain of direction-selective neurons in visual cortical area MT without narrowing the direction-tuning curves. These findings place important constraints on the neural mechanisms of attention and we propose to unify the effects of spatial location, direction of motion and other features of the attended stimuli in a 'feature similarity gain model' of attention.     

Perceptual learning without perception.

The brain is able to adapt rapidly and continually to the surrounding environment, becoming increasingly sensitive to important and frequently encountered stimuli. It is often claimed that this adaptive learning is highly task-specific, that is, we become more sensitive to the critical signals in the tasks we attend to. Here, we show a new type of perceptual learning, which occurs without attention, without awareness and without any task relevance. Subjects were repeatedly presented with a background motion signal so weak that its direction was not visible; the invisible motion was an irrelevant background to the central task that engaged the subject's attention. Despite being below the threshold of visibility and being irrelevant to the central task, the repetitive exposure improved performance specifically for the direction of the exposed motion when tested in a subsequent suprathreshold test. These results suggest that a frequently presented feature sensitizes the visual system merely owing to its frequency, not its relevance or salience.     

感受野视觉特征整合模型及应用

 采用基本ICA模拟视觉感知机制对自然图像分解得到的图像基函数在空间排列上是混乱的,这与视觉生理机制相互矛盾.模拟视皮层感受野间的信息整合机制,建立了新的计算模型.针对基于内容的图像故障区域检测问题,提出了相应的高效率少样本检测算法. 首先,以列车正常和故障图像序列作为训练数据,利用拓扑ICA方法学习图像基函数,由此得到的独立分量系数作为神经元响应,然后模拟同步振荡机制选择响应强烈的神经元,输出其对应的内容,最后通过自动对比实现图像故障区域的快速定位.实验结果表明,与传统方法相比较,引入视觉信息整合机制的新模型及其算法能够提高故障检测率.     

An evolutionary scaling law for the primate visual system and its basis in cortical function

A hallmark of mammalian brain evolution is the disproportionate increase in neocortical size as compared with subcortical structures. Because primary visual cortex (V1) is the most thoroughly understood cortical region, the visual system provides an excellent model in which to investigate the evolutionary expansion of neocortex. I have compared the numbers of neurons in the visual thalamus (lateral geniculate nucleus; LGN) and area V1 across primate species. Here I find that the number of V1 neurons increases as the 3/2 power of the number of LGN neurons. As a consequence of this scaling law, the human, for example, uses four times as many V1 neurons per LGN neuron (356) to process visual information as does a tarsier (87). I argue that the 3/2 power relationship is a natural consequence of the organization of V1, together with the requirement that spatial resolution in V1 should parallel the maximum resolution provided by the LGN. The additional observation that thalamus/neocortex follows the same evolutionary scaling law as LGN/V1 may suggest that neocortex generally conforms to the same organizational principle as V1.     

Influence of motion signals on the perceived position of spatial pattern

After adaptation of the visual system to motion of a pattern in a particular direction, a static pattern appears to move in the opposite direction-the motion aftereffect (MAE). It is thought that the MAE is not accompanied by a shift in perceived spatial position of the pattern being viewed, providing psychophysical evidence for a dissociation of the neural processing of motion and position that complements anatomical and physiological evidence of functional specialization in primate and human visual cortex. However, here we measure the perceived orientation of a static windmill pattern after adaptation to rotary motion and find a gradual shift in orientation in the direction of the illusory rotation, though at a rate much lower than the apparent rotation speed. The orientation shift, which started to decline within a few seconds, could persist longer than the MAE, and disappeared when the MAE was nulled by physical motion of the windmill pattern. Our results indicate that the representation of the position of spatial pattern is dynamically updated by neurons involved in the analysis of motion.     

Neural correlates of implied motion

Current views of the visual system assume that the primate brain analyses form and motion along largely independent pathways; they provide no insight into why form is sometimes interpreted as motion. In a series of psychophysical and electrophysiological experiments in humans and macaques, here we show that some form information is processed in the prototypical motion areas of the superior temporal sulcus (STS). First, we show that STS cells respond to dynamic Glass patterns, which contain no coherent motion but suggest a path of motion. Second, we show that when motion signals conflict with form signals suggesting a different path of motion, both humans and monkeys perceive motion in a compromised direction. This compromise also has a correlate in the responses of STS cells, which alter their direction preferences in the presence of conflicting implied motion information. We conclude that cells in the prototypical motion areas in the dorsal visual cortex process form that implies motion. Estimating motion by combining motion cues with form cues may be a strategy to deal with the complexities of motion perception in our natural environment.     

Selective gating of visual signals by microstimulation of frontal cortex

Several decades of psychophysical and neurophysiological studies have established that visual signals are enhanced at the locus of attention. What remains a mystery is the mechanism that initiates biases in the strength of visual representations. Recent evidence argues that, during spatial attention, these biases reflect nascent saccadic eye movement commands. We examined the functional interaction of saccade preparation and visual coding by electrically stimulating sites within the frontal eye fields (FEF) and measuring its effect on the activity of neurons in extrastriate visual cortex. Here we show that visual responses in area V4 could be enhanced after brief stimulation of retinotopically corresponding sites within the FEF using currents below that needed to evoke saccades. The magnitude of the enhancement depended on the effectiveness of receptive field stimuli as well as on the presence of competing stimuli outside the receptive field. Stimulation of non-corresponding FEF representations could suppress V4 responses. The results suggest that the gain of visual signals is modified according to the strength of spatially corresponding eye movement commands.     

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.