An unexpected specialization for horizontal disparity in primate primary visual cortex

The horizontal separation of the eyes means that objects nearer or farther than the fixation point project to different locations on the two retinae, differing principally in their horizontal coordinates (horizontal binocular disparity). Disparity-selective neurons have generally been studied with disparities applied in only one direction (often horizontal), which cannot determine whether the encoding is specialized for processing disparities along the horizontal axis. It is therefore unclear if disparity selectivity represents a specialization for naturally occurring disparities. I used random dot stereograms to study disparity-selective neurons from the primary visual cortex (V1) of awake fixating monkeys. Many combinations of vertical and horizontal disparity were used, characterizing the surface of responses as a function of two-dimensional disparity. Here I report that the response surface usually showed elongation along the horizontal disparity axis, despite the isotropic stimulus. Thus these neurons modulated their firing rate over a wider range of horizontal disparity than vertical disparity. This demonstrates that disparity-selective cells are specialized for processing horizontal disparity, and that existing models of disparity selectivity require substantial revision.     

Probing the human stereoscopic system with reverse correlation.

Our two eyes obtain slightly different views of the world. The resulting differences in the two retinal images, called binocular disparities, provide us with a stereoscopic sense of depth. The primary visual cortex (V1) contains neurons that are selective for the disparity of individual elements in an image, but this information must be further analysed to complete the stereoscopic process. Here we apply the psychophysical technique of reverse correlation to investigate disparity processing in human vision. Observers viewed binocular random-dot patterns, with 'signal' dots in a specific depth plane plus 'noise' dots with randomly assigned disparities. By examining the correlation between the observers' ability to detect the plane and the particular sample of 'noise' disparities presented on each trial, we revealed detection 'filters', whose disparity selectivity was remarkably similar to that of individual neurons in monkey V1. Moreover, if the noise dots were of opposite contrast in the two eyes, the tuning inverted, just like the response patterns of V1 neurons. Reverse correlation appears to probe disparity processing at the earliest stages of binocular combination, prior to the generation of a full stereoscopic depth percept.     

Motion direction, speed and orientation in binocular matching

The spatial differences between the images seen by the two eyes, called binocular disparities, can be used to recover the volumetric (three-dimensional) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this matching problem is one of the main issues in stereoscopic vision. Many cortical neurons in visual areas V1 (ref. 2), MT (refs 3, 4) and MST (refs 5, 6) that are tuned to binocular disparity are also tuned to orientation, motion direction and speed. Although psychophysical work has shown that motion direction can facilitate binocular matching, the psychophysical literature on the role of orientation is mixed, and it has been argued that speed differences are ineffective in aiding correspondence. Here we use a different psychophysical paradigm to show that the visual system uses similarities in orientation, motion direction and speed to achieve binocular correspondence. These results indicate that cells that multiplex orientation, motion direction, speed and binocular disparity may help to solve the binocular matching problem.     

Sensitivity for vertical retinal image differences

The detection of horizontal disparities between the right and left retinal images is accepted as an important component in the visual perception of depth. It has occasionally been pointed out that a disparity will also occur in the vertical when a target is nearer to one eye than the other and could be used as a second component. Here it is shown that while it is possible to detect size differences between the retinal images in the vertical direction, the sensitivity is at least one order of magnitude less than the horizontal differences. It had been generally accepted that vertical disparities cannot be detected (literature summarized by Tschermak), but there are clear reports of the ability of subjects to respond to relative differences in the vertical magnification of the images of the two eyes, the so-called induced size effect. It manifests itself in the appearance of a visual scene as if there had been an opposite horizontal magnification and is, therefore, in the correct direction if it is to be utilized in asymmetrical convergence.     

Disparity curvature and the perception of three-dimensional surfaces

BINOCULAR stereopsis provides information about the relative distance of objects from the differences in the horizontal position of their images on the two retinas. Because the size of the disparity between two points is inversely related to the square of the viewing distance, it is usually assumed that disparities have to be scaled according to distance using the vergence angle of the eyes, or by using the small vertical disparities that also exist between corresponding points of the two images. Here we present evidence that the visual system could extract information about the shapes of surfaces (without the need for scaling) by using the second spatial derivative of disparity--disparity curvature--which remains invariant with viewing distance. Rather than computing the second derivative, we suggest that an approximation to disparity curvature could be derived from the differences in curvature of corresponding line elements in the two eyes.     

光轴外扩式布置的双目相机图像立体校正

摘要： 针对宽视野光轴外扩式布置双目图像采集和显示系统的图像立体校正问题,利用三维重建误差,提出了一种基于双目图像视差特性的立体校正方法.在分析光轴外扩式布置双目相机图像视差特性的基础上,针对相机光心与显示投影中心间的位置误差和相机光轴与显示视轴间的角度误差,建立图像仿射变换、水平视差畸变和垂直视差的校正函数,利用一组空间采样点的三维重投影误差作为目标函数通过优化方法求解校正参数.仿真实验表明：在视轴外扩夹角为30°时,本方法可以有效地消除光心位置误差和光轴角度误差对图像视差造成的影响.真实图像校正实验表明：校正后的图像水平畸变小,在显示宽视野双目图像时,能保持正确的三维立体感.
关键词：中图分类号：文献标志码： A     

Vertical disparities and perception of three-dimensional shape

The information about depth and three-dimensional shape available from the horizontal component of the stereo disparity field requires interpretation in conjunction with information about egocentric viewing distance (D). A novel computational approach for estimating D was proposed by Mayhew and Longuet-Higgins, who demonstrated that the horizontal gradient of vertical disparities uniquely specifies the viewing distance. We have now used random dot stereograms in a shape judgement task to show that changes in vertical disparities have no effect on perceived three-dimensional shape. Changes in ocular convergence do alter perceived shape, suggesting substantial changes in the subjects' scaling of horizontal disparities. We conclude that vertical disparities are not used to scale disparities for viewing distance, and that extraretinal signals must be considered when analysing human three-dimensional shape perception.     

Parallel processing of motion and colour information

When the two eyes are confronted with sufficiently different versions of the visual environment, one or the other eye dominates perception in alternation. A similar situation may be created in the laboratory by presenting images to the left and right eyes which differ in orientation or colour. Although perception is dominated by one eye during rivalry, there are a number of instances in which visual processes nevertheless continue to integrate information from the suppressed eye. For example the interocular transfer of the motion after-effect is undiminished when induced during binocular rivalry. Thus motion information processing may occur in parallel with the rivalry process. Here we describe a novel example in which the visual system simultaneously exhibits binocular rivalry and vision that integrates signals from both eyes. This apparent contradiction is resolved by postulating parallel visual processes devoted to the analyses of colour and motion information. Counterphased gratings are viewed dichoptically such that for one eye the grating is composed of alternating yellow and black stripes (luminance) while for the other it is composed of alternating red and green stripes (chrominance). When the gratings are fused, a moving grating is perceived. A consistent direction of motion can only be achieved if left and right monocular signals are integrated by the nervous system. Yet the apparent colour of the binocular percept alternates between red-green and yellow-black. These observations demonstrate the segregation of processing by the early motion system from that affording the perception of colour. Although, in this stimulus, colour information in itself can play no part in the cyclopean perception of motion direction, colour is carried along perceptually (filled in) by the moving pattern which is integrated from both eyes.     

对立体视觉中对称性假设的检验

指出了基于立体视觉研究中的隐性对称性假设,即认为匹配的选取与哪只眼观察哪一图像无关,而仅取决于两眼映像特征的相似性,因此交换左眼和右眼图像这样一种实验操作,被认为提供了一种非常简便有效的技术,用于鉴定双眼性视差和其他不同于双眼性视差的因素对立体深度的贡献.采用2:2直线型常规立体视构型,对比了左右眼图像交换前后的匹配结果,发现匹配融合产生中间第3根直线的两根直线在图像交换前后会发生改变,并导致中间直线的深度在图像交换前后不发生颠倒.实验结果证明:立体视觉研究中的隐性对称性假设不成立;交换左眼和右眼图像后深度是否颠倒,并不是判断深度是来源于匹配视差还是其他因素的依据.     

Coding of smooth eye movements in three-dimensional space by frontal cortex

Through the development of a high-acuity fovea, primates with frontal eyes have acquired the ability to use binocular eye movements to track small objects moving in space. The smooth-pursuit system moves both eyes in the same direction to track movement in the frontal plane (frontal pursuit), whereas the vergence system moves left and right eyes in opposite directions to track targets moving towards or away from the observer (vergence tracking). In the cerebral cortex and brainstem, signals related to vergence eye movements--and the retinal disparity and blur signals that elicit them--are coded independently of signals related to frontal pursuit. Here we show that these types of signal are represented in a completely different way in the smooth-pursuit region of the frontal eye fields. Neurons of the frontal eye field modulate strongly during both frontal pursuit and vergence tracking, which results in three-dimensional cartesian representations of eye movements. We propose that the brain creates this distinctly different intermediate representation to allow these neurons to function as part of a system that enables primates to track and manipulate objects moving in three-dimensional space.     

Monocular aniseikonia: a motion parallax analogue of the disparity-induced effect

Mayhew and Longuet-Higgins have recently outlined a computational model of binocular depth perception in which the small vertical disparities between the two eyes' views of a three-dimensional scene are used to determine the 'viewing parameters' of fixation distance (d) and the angle of asymmetric convergence of the eyes (g). The d/g hypothesis, as it has been called, correctly predicts that a fronto-parallel surface, viewed with a vertically magnifying lens over one eye, should appear to be rotated in depth about a vertical axis. We report here a comparable illusion for surfaces specified by monocular motion parallax information, which can be explained more simply by considering the differential invariants of the optic flow field. In addition, our observations suggest that the disparity-induced effect is not a 'whole field' phenomenon nor one limited to small magnification differences between the eyes.     

Correlated binocular activity guides recovery from monocular deprivation

Monocular deprivation (MD) has much more rapid and severe effects on the ocular dominance of neurons in the primary visual cortex (V1) than does binocular deprivation. This finding underlies the widely held hypothesis that the developmental plasticity of ocular dominance reflects competitive interactions for synaptic space between inputs from the two eyes. According to this view, the relative levels of evoked activity in afferents representing the two eyes determine functional changes in response to altered visual experience. However, if the deprived eye of a monocularly deprived kitten is simply reopened, there is substantial physiological and behavioural recovery, leading to the suggestion that absolute activity levels, or some other non-competitive mechanisms, determine the degree of recovery from MD. Here we provide evidence that correlated binocular input is essential for such recovery. Recovery is far less complete if the two eyes are misaligned after a period of MD. This is a powerful demonstration of the importance of cooperative, associative mechanisms in the developing visual cortex.     

基于BP神经网络自学习的双目视觉系统的研发

在摄像机标定的过程中,深度信息的丢失,摄像机镜头的畸变以及图像处理时误差等因素都影响标定的精度．本论文采用BP神经网络的自学习的性能,开发出一套双目视觉系统．以匹配点在左、右图像的坐标为网络的四路输入,通过网络得到三路输出,性能指标为该对应点在世界坐标系的坐标和网络输出的差值的平方和,根据梯度下降法来调整各神经元之间的连接权值,求得网络达到给定的误差时的各节点问权值．这样,双目视觉系统两个摄像机的投影矩阵可以用神经网络的权值与激发函数来代替,完成系统的标定．最后对系统进行精度分析．     

Gain control by layer six in cortical circuits of vision

After entering the cerebral cortex, sensory information spreads through six different horizontal neuronal layers that are interconnected by vertical axonal projections. It is believed that through these projections layers can influence each other's response to sensory stimuli, but the specific role that each layer has in cortical processing is still poorly understood. Here we show that layer six in the primary visual cortex of the mouse has a crucial role in controlling the gain of visually evoked activity in neurons of the upper layers without changing their tuning to orientation. This gain modulation results from the coordinated action of layer six intracortical projections to superficial layers and deep projections to the thalamus, with a substantial role of the intracortical circuit. This study establishes layer six as a major mediator of cortical gain modulation and suggests that it could be a node through which convergent inputs from several brain areas can regulate the earliest steps of cortical visual processing.     

Gaze direction controls response gain in primary visual-cortex neurons

To localize objects in space, the brain needs to combine information about the position of the stimulus on the retinae with information about the location of the eyes in their orbits. Interaction between these two types of information occurs in several cortical areas, but the role of the primary visual cortex (area V1) in this process has remained unclear. Here we show that, for half the cells recorded in area V1 of behaving monkeys, the classically described visual responses are strongly modulated by gaze direction. Specifically, we find that selectivity for horizontal retinal disparity-the difference in the position of a stimulus on each retina which relates to relative object distance-and for stimulus orientation may be present at a given gaze direction, but be absent or poorly expressed at another direction. Shifts in preferred disparity also occurred in several neurons. These neural changes were most often present at the beginning of the visual response, suggesting a feedforward gain control by eye position signals. Cortical neural processes for encoding information about the three-dimensional position of a stimulus in space therefore start as early as area V1.     

Subtractive and divisive adaptation in the human visual system.

Sensory systems can adapt to the conditions imposed on them. In the visual system, adapting to a pattern increases the threshold of the ability to see that pattern, and reduces the perceived contrast of the pattern above threshold. Most neurons of the striate cortex reduce their responsiveness after being stimulated for some time by a high-contrast pattern. Such an effect may lie behind these psychophysical adaptation phenomena. These adaptation effects have been reported to be confined to patterns of similar orientation, which is understandable in that the visual neurons that adapt are only excited by a small range of orientations. Neurophysiological evidence suggests that neurons with different orientation preferences have inhibitory interconnections. It is therefore of interest to explore the possible effects of these connections on perception. Here we show that adapting to a horizontal pattern can reduce the perceived contrast of a vertical test pattern more than a horizontal test pattern. These 'cross-orientation' effects are modelled by a division-like process, whereas the more normal 'similar-orientation' effects are modelled by a subtractive process.     

Interocular rivalry revealed in the human cortical blind-spot representation

To understand conscious vision, scientists must elucidate how the brain selects specific visual signals for awareness. When different monocular patterns are presented to the two eyes, they rival for conscious expression such that only one monocular image is perceived at a time. Controversy surrounds whether this binocular rivalry reflects neural competition among pattern representations or monocular channels. Here we show that rivalry arises from interocular competition, using functional magnetic resonance imaging of activity in a monocular region of primary visual cortex corresponding to the blind spot. This cortical region greatly prefers stimulation of the ipsilateral eye to that of the blind-spot eye. Subjects reported their dominant percept while viewing rivalrous orthogonal gratings in the visual location corresponding to the blind spot and its surround. As predicted by interocular rivalry, the monocular blind-spot representation was activated when the ipsilateral grating became perceptually dominant and suppressed when the blind-spot grating became dominant. These responses were as large as those observed during actual alternations between the gratings, indicating that rivalry may be fully resolved in monocular visual cortex. Our findings provide the first physiological evidence, to our knowledge, that interocular competition mediates binocular rivalry, and indicate that V1 may be important in the selection and expression of conscious visual information.     

Functional mapping of single spines in cortical neurons in vivo

The individual functional properties and spatial arrangement of afferent synaptic inputs on dendrites have a critical role in the processing of information by neurons in the mammalian brain. Although recent work has identified visually-evoked local dendritic calcium signals in the rodent visual cortex, sensory-evoked signalling on the level of dendritic spines, corresponding to individual afferent excitatory synapses, remains unexplored. Here we used a new variant of high-resolution two-photon imaging to detect sensory-evoked calcium transients in single dendritic spines of mouse cortical neurons in vivo. Calcium signals evoked by sound stimulation required the activation of NMDA (N-methyl-D-aspartate) receptors. Active spines are widely distributed on basal and apical dendrites and pure-tone stimulation at different frequencies revealed both narrowly and widely tuned spines. Notably, spines tuned for different frequencies were highly interspersed on the same dendrites: even neighbouring spines were mostly tuned to different frequencies. Thus, our results demonstrate that NMDA-receptor-dependent single-spine synaptic inputs to the same dendrite are highly heterogeneous. Furthermore, our study opens the way for in vivo mapping of functionally defined afferent sensory inputs with single-synapse resolution.     

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.     

Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex

Neurons in the cerebral cortex are organized into anatomical columns, with ensembles of cells arranged from the surface to the white matter. Within a column, neurons often share functional properties, such as selectivity for stimulus orientation; columns with distinct properties, such as different preferred orientations, tile the cortical surface in orderly patterns. This functional architecture was discovered with the relatively sparse sampling of microelectrode recordings. Optical imaging of membrane voltage or metabolic activity elucidated the overall geometry of functional maps, but is averaged over many cells (resolution >100 microm). Consequently, the purity of functional domains and the precision of the borders between them could not be resolved. Here, we labelled thousands of neurons of the visual cortex with a calcium-sensitive indicator in vivo. We then imaged the activity of neuronal populations at single-cell resolution with two-photon microscopy up to a depth of 400 microm. In rat primary visual cortex, neurons had robust orientation selectivity but there was no discernible local structure; neighbouring neurons often responded to different orientations. In area 18 of cat visual cortex, functional maps were organized at a fine scale. Neurons with opposite preferences for stimulus direction were segregated with extraordinary spatial precision in three dimensions, with columnar borders one to two cells wide. These results indicate that cortical maps can be built with single-cell precision.