Three-dimensional illusory contours and surfaces.

Under general viewing conditions, objects are often partially camouflaged, obscured or occluded, thereby limiting information about their three-dimensional position, orientation and shape to incomplete and variable image cues. When presented with such partial cues, observers report perceiving 'illusory' contours and surfaces (forms) in regions having no physical image contrast. Here we report that three-dimensional illusory forms share three fundamental properties with 'real' forms: (1) the same forms are perceived using either stereo or motion parallax cues (cue invariance); (2) they retain their shape over changes in position and orientation relative to an observer (view stability); and (3) they can take the shape of general contours and surfaces in three dimensions (morphic generality). We hypothesize that illusory contours and surfaces are manifestations of a previously unnoticed visual process which constructs a representation of three-dimensional position, orientation and shape of objects from available image cues.     

Motion direction, speed and orientation in binocular matching

The spatial differences between the images seen by the two eyes, called binocular disparities, can be used to recover the volumetric (three-dimensional) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this matching problem is one of the main issues in stereoscopic vision. Many cortical neurons in visual areas V1 (ref. 2), MT (refs 3, 4) and MST (refs 5, 6) that are tuned to binocular disparity are also tuned to orientation, motion direction and speed. Although psychophysical work has shown that motion direction can facilitate binocular matching, the psychophysical literature on the role of orientation is mixed, and it has been argued that speed differences are ineffective in aiding correspondence. Here we use a different psychophysical paradigm to show that the visual system uses similarities in orientation, motion direction and speed to achieve binocular correspondence. These results indicate that cells that multiplex orientation, motion direction, speed and binocular disparity may help to solve the binocular matching problem.     

Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain

A critical step in the interpretation of the visual world is the integration of the various local motion signals generated by moving objects. This process is complicated by the fact that local velocity measurements can differ depending on contour orientation and spatial position. Specifically, any local motion detector can measure only the component of motion perpendicular to a contour that extends beyond its field of view. This "aperture problem" is particularly relevant to direction-selective neurons early in the visual pathways, where small receptive fields permit only a limited view of a moving object. Here we show that neurons in the middle temporal visual area (known as MT or V5) of the macaque brain reveal a dynamic solution to the aperture problem. MT neurons initially respond primarily to the component of motion perpendicular to a contour's orientation, but over a period of approximately 60 ms the responses gradually shift to encode the true stimulus direction, regardless of orientation. We also report a behavioural correlate of these neural responses: the initial velocity of pursuit eye movements deviates in a direction perpendicular to local contour orientation, suggesting that the earliest neural responses influence the oculomotor response.     

虚拟现实的技术瓶颈

 在技术变革和资本力量的双重推动下,虚拟现实（virtual reality）技术在近几年发展迅速,初步达到了可商业化的程度。虚拟现实和3D 电影院都是通过双目视差实现三维成像,但虚拟现实提供了3D 电影院所不具备的移动视差并提供了强烈的沉浸感。现阶段虚拟现实技术仍面临着一系列技术难题,其中眩晕和人眼疲劳尤其明显,是虚拟现实的技术禁地。本文从介绍三维视觉感知开始,分析了虚拟现实造成眩晕和人眼疲劳的根本原因。给出了解决这一技术瓶颈的答案--动态光场,并从光场采集和显示两方面分析了多种光场技术的优缺点。列举了增强现实（augmented reality）技术的3 种实现形式,并从人与人交互和通信的角度对比了虚拟现实与增强现实在未来的发展趋势。     

Neuronal synchrony does not correlate with motion coherence in cortical area MT

Natural visual scenes are cluttered with multiple objects whose individual features must somehow be selectively linked (or 'bound') if perception is to coincide with reality. Recent neurophysiological evidence supports a 'binding-by-synchrony' hypothesis: neurons excited by features of the same object fire synchronously, while neurons excited by features of different objects do not. Moving plaid patterns offer a straightforward means to test this idea. By appropriate manipulations of apparent transparency, the component gratings of a plaid pattern can be seen as parts of a single coherently moving surface or as two non-coherently moving surfaces. We examined directional tuning and synchrony of area-MT neurons in awake, fixating primates in response to perceptually coherent and non-coherent plaid patterns. Here we show that directional tuning correlated highly with perceptual coherence, which is consistent with an earlier study. Although we found stimulus-dependent synchrony, coherent plaids elicited significantly less synchrony than did non-coherent plaids. Our data therefore do not support the binding-by-synchrony hypothesis as applied to this class of motion stimuli in area MT.     

Self-motion and the perception of stationary objects

One of the ways that we perceive shape is through seeing motion. Visual motion may be actively generated (for example, in locomotion), or passively observed. In the study of the perception of three-dimensional structure from motion, the non-moving, passive observer in an environment of moving rigid objects has been used as a substitute for an active observer moving in an environment of stationary objects; this 'rigidity hypothesis' has played a central role in computational and experimental studies of structure from motion. Here we show that this is not an adequate substitution because active and passive observers can perceive three-dimensional structure differently, despite experiencing the same visual stimulus: active observers' perception of three-dimensional structure depends on extraretinal information about their own movements. The visual system thus treats objects that are stationary (in an allocentric, earth-fixed reference frame) differently from objects that are merely rigid. These results show that action makes an important contribution to depth perception, and argue for a revision of the rigidity hypothesis to incorporate the special case of stationary objects.     

Cortical microstimulation influences perceptual judgements of motion direction

Neurons in the visual cortex respond selectively to perceptually salient features of the visual scene, such as the direction and speed of moving objects, the orientation of local contours, or the colour or relative depth of a visual pattern. It is commonly assumed that the brain constructs its percept of the visual scene from information encoded in the selective responses of such neurons. We have now tested this hypothesis directly by measuring the effect on psychophysical performance of modifying the firing rates of physiologically characterized neurons. We required rhesus monkeys to report the direction of motion in a visual display while we electrically stimulated clusters of directionally selective neurons in the middle temporal visual area (MT, or V5), an extrastriate area that plays a prominent role in the analysis of visual motion information. Microstimulation biased the animals' judgements towards the direction of motion encoded by the stimulated neurons. This result indicates that physiological properties measured at the neuronal level can be causally related to a specific aspect of perceptual performance.     

fMRI evidence for objects as the units of attentional selection.

Contrasting theories of visual attention emphasize selection by spatial location, visual features (such as motion or colour) or whole objects. Here we used functional magnetic resonance imaging (fMRI) to test key predictions of the object-based theory, which proposes that pre-attentive mechanisms segment the visual array into discrete objects, groups, or surfaces, which serve as targets for visual attention. Subjects viewed stimuli consisting of a face transparently superimposed on a house, with one moving and the other stationary. In different conditions, subjects attended to the face, the house or the motion. The magnetic resonance signal from each subject's fusiform face area, parahippocampal place area and area MT/MST provided a measure of the processing of faces, houses and visual motion, respectively. Although all three attributes occupied the same location, attending to one attribute of an object (such as the motion of a moving face) enhanced the neural representation not only of that attribute but also of the other attribute of the same object (for example, the face), compared with attributes of the other object (for example, the house). These results cannot be explained by models in which attention selects locations or features, and provide physiological evidence that whole objects are selected even when only one visual attribute is relevant.     

Monocular aniseikonia: a motion parallax analogue of the disparity-induced effect

Mayhew and Longuet-Higgins have recently outlined a computational model of binocular depth perception in which the small vertical disparities between the two eyes' views of a three-dimensional scene are used to determine the 'viewing parameters' of fixation distance (d) and the angle of asymmetric convergence of the eyes (g). The d/g hypothesis, as it has been called, correctly predicts that a fronto-parallel surface, viewed with a vertically magnifying lens over one eye, should appear to be rotated in depth about a vertical axis. We report here a comparable illusion for surfaces specified by monocular motion parallax information, which can be explained more simply by considering the differential invariants of the optic flow field. In addition, our observations suggest that the disparity-induced effect is not a 'whole field' phenomenon nor one limited to small magnification differences between the eyes.     

3D立体显示技术的现况与展望

3D立体显示器利用人类双眼视差以及移动视差的特性,分别提供给双眼不同影像,经大脑合成后形成立体感影像。立体显示技术分为戴眼镜式和裸眼式,在戴眼镜式中,着重比较快门眼镜式和偏光眼镜式;在裸眼式中,着重介绍柱状透镜式、视差遮屏式以及方向性背光源式,本文就其原理和各种技术的优缺点进行分析。     

Feature-based attention influences motion processing gain in macaque visual cortex.

Changes in neural responses based on spatial attention have been demonstrated in many areas of visual cortex, indicating that the neural correlate of attention is an enhanced response to stimuli at an attended location and reduced responses to stimuli elsewhere. Here we demonstrate non-spatial, feature-based attentional modulation of visual motion processing, and show that attention increases the gain of direction-selective neurons in visual cortical area MT without narrowing the direction-tuning curves. These findings place important constraints on the neural mechanisms of attention and we propose to unify the effects of spatial location, direction of motion and other features of the attended stimuli in a 'feature similarity gain model' of attention.     

Experience-dependent representation of visual categories in parietal cortex

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as 'chair', 'table' and 'vehicle', which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360 degrees of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP--an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making-robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.     

Neural correlates of perceptual motion coherence.

The motions of overlapping contours in a visual scene may arise from the physical motion(s) of either a single or multiple surface(s). A central problem facing the visual motion system is that of assigning the most likely interpretation. The rules underlying this perceptual decision can be explored using a visual stimulus formed by superimposing two moving gratings. The resultant percept is either that of a single coherently moving 'plaid pattern' (coherent motion) or of the two component gratings sliding noncoherently across one another (noncoherent motion). When plaid patterns are configured to mimic one transparent grating overlying another, the percept of noncoherent motion dominates. We now report that neurons in the visual cortex of rhesus monkeys exhibit changes in direction tuning that parallel this perceptual phenomenon: sensitivity to the motions of the component gratings is enhanced under conditions that favour the perception of noncoherent motion. These results challenge models of cortical visual processing that fail to take into account the contribution of figural image segmentation cues to the analysis of visual motion.     

Hearing visual motion in depth

Auditory spatial perception is strongly affected by visual cues. For example, if auditory and visual stimuli are presented synchronously but from different positions, the auditory event is mislocated towards the locus of the visual stimulus-the ventriloquism effect. This 'visual capture' also occurs in motion perception in which a static auditory stimulus appears to move with the visual moving object. We investigated how the human perceptual system coordinates complementary inputs from auditory and visual senses. Here we show that an auditory aftereffect occurs from adaptation to visual motion in depth. After a few minutes of viewing a square moving in depth, a steady sound was perceived as changing loudness in the opposite direction. Adaptation to a combination of auditory and visual stimuli changing in a compatible direction increased the aftereffect and the effect of visual adaptation almost disappeared when the directions were opposite. On the other hand, listening to a sound changing in intensity did not affect the visual changing-size aftereffect. The results provide psychophysical evidence that, for processing of motion in depth, the auditory system responds to both auditory changing intensity and visual motion in depth.     

Neural mechanisms of unconscious visual motion priming

The neural correlates of the motion priming were examined in normal young subjects using event-related brain potentials (ERPs) and functional magnetic resonance imaging (fMRI). Visual motion perception can be uncon-sciously biased in favor of a particular direction by a pre-ceding motion in that direction. Motion priming first in-volved an enhancement of ERP amplitude about 100 ms fol-lowing the onset of motion. The amplitudes of ERP compo-nents after 350 ms were also increased. The fMRI results suggest that the early-latency effect reflects modulation of neural responses in extrastriate cortex. Higher-level visual processing areas, including cortical regions MT/MST and the intraparietal cortices were also activated. The findings provide direct evidence that unconscious priming of motion perception is the result of interaction of direction-selective neural responses to motion stimuli. The results cannot be accounted for by refractoriness of neural responses, but in-stead support a theory of motion priming based on motion opponency, as proposed in computational models.     

Parallel processing of motion and colour information

When the two eyes are confronted with sufficiently different versions of the visual environment, one or the other eye dominates perception in alternation. A similar situation may be created in the laboratory by presenting images to the left and right eyes which differ in orientation or colour. Although perception is dominated by one eye during rivalry, there are a number of instances in which visual processes nevertheless continue to integrate information from the suppressed eye. For example the interocular transfer of the motion after-effect is undiminished when induced during binocular rivalry. Thus motion information processing may occur in parallel with the rivalry process. Here we describe a novel example in which the visual system simultaneously exhibits binocular rivalry and vision that integrates signals from both eyes. This apparent contradiction is resolved by postulating parallel visual processes devoted to the analyses of colour and motion information. Counterphased gratings are viewed dichoptically such that for one eye the grating is composed of alternating yellow and black stripes (luminance) while for the other it is composed of alternating red and green stripes (chrominance). When the gratings are fused, a moving grating is perceived. A consistent direction of motion can only be achieved if left and right monocular signals are integrated by the nervous system. Yet the apparent colour of the binocular percept alternates between red-green and yellow-black. These observations demonstrate the segregation of processing by the early motion system from that affording the perception of colour. Although, in this stimulus, colour information in itself can play no part in the cyclopean perception of motion direction, colour is carried along perceptually (filled in) by the moving pattern which is integrated from both eyes.     

Dynamic coding of behaviourally relevant stimuli in parietal cortex.

A general function of cerebral cortex is to allow the flexible association of sensory stimuli with specific behaviours. Many neurons in parietal, prefrontal and motor cortical areas are activated both by particular movements and by sensory cues that trigger these movements, suggesting a role in linking sensation to action. For example, neurons in the lateral intraparietal area (LIP) encode both the location of visual stimuli and the direction of saccadic eye movements. LIP is not believed to encode non-spatial stimulus attributes such as colour. Here we investigated whether LIP would encode colour if colour was behaviourally linked to the eye movement. We trained monkeys to make an eye movement in one of two directions based alternately on the colour or location of a visual cue. When cue colour was relevant for directing eye movement, we found a substantial fraction of LIP neurons selective for cue colour. However, when cue location was relevant, colour selectivity was virtually absent in LIP. These results demonstrate that selectivity of cortical neurons can change as a function of the required behaviour.     

Perception of shape from shading

The human visual system can rapidly and accurately derive the three-dimensional orientation of surfaces by using variations in image intensity alone. This ability to perceive shape from shading is one of the most important yet poorly understood aspects of human vision. Here we present several findings which may help reveal computational mechanisms underlying this ability. First, we find that perception of shape from shading is a global operation which assumes that there is only one light source illuminating the entire visual image. This implies that if two identical objects are viewed simultaneously and illuminated from different angles, then we would be able to perceive three-dimensional shape accurately in only one of them at a time. Second, three-dimensional shapes that are defined exclusively by shading can provide tokens for the perception of apparent motion, suggesting that the motion mechanism is remarkably versatile in the kinds of inputs it can use. Lastly, the occluding edges which delineate an object from its background can also powerfully influence the perception of three-dimensional shape from shading.     

Neurons compute internal models of the physical laws of motion

A critical step in self-motion perception and spatial awareness is the integration of motion cues from multiple sensory organs that individually do not provide an accurate representation of the physical world. One of the best-studied sensory ambiguities is found in visual processing, and arises because of the inherent uncertainty in detecting the motion direction of an untextured contour moving within a small aperture. A similar sensory ambiguity arises in identifying the actual motion associated with linear accelerations sensed by the otolith organs in the inner ear. These internal linear accelerometers respond identically during translational motion (for example, running forward) and gravitational accelerations experienced as we reorient the head relative to gravity (that is, head tilt). Using new stimulus combinations, we identify here cerebellar and brainstem motion-sensitive neurons that compute a solution to the inertial motion detection problem. We show that the firing rates of these populations of neurons reflect the computations necessary to construct an internal model representation of the physical equations of motion.     

Perceptual stimulus――A Bayesian-based integration of multi-visual-cue approach and its application

With the view that visual cue could be taken as a kind of stimulus, the study of the mechanism in the visual perception process by using visual cues in their probabilistic representation eventually leads to a class of statistical Integration of multiple visual cues （IMVC） methods which have been applied widely in perceptual grouping, video analysis, and other basic problems in computer vision. In this paper, a survey on the basic ideas and recent advances of IMVC methods is presented, and much focus is on the models and algorithms of IMVC for video analysis within the framework of Bayesian estimation. Furthermore, two typical problems in video analysis, robust visual tracking and “switching problem” in muIti-targèt tracking （MTT） are taken as test beds to verify a series of Bayesian-based IMVC methods proposed by the authors. Furthermore, the relations between the statistical IMVC and the visual perception process, as well as potential future research work for IMVC, are discussed.