Visual behaviour mediated by retinal projections directed to the auditory pathway

An unresolved issue in cortical development concerns the relative contributions of intrinsic and extrinsic factors to the functional specification of different cortical areas. Ferrets in which retinal projections are redirected neonatally to the auditory thalamus have visually responsive cells in auditory thalamus and cortex, form a retinotopic map in auditory cortex and have visual receptive field properties in auditory cortex that are typical of cells in visual cortex. Here we report that this cross-modal projection and its representation in auditory cortex can mediate visual behaviour. When light stimuli are presented in the portion of the visual field that is 'seen' only by this projection, 'rewired' ferrets respond as though they perceive the stimuli to be visual rather than auditory. Thus the perceptual modality of a neocortical region is instructed to a significant extent by its extrinsic inputs. In addition, gratings of different spatial frequencies can be discriminated by the rewired pathway, although the grating acuity is lower than that of the normal visual pathway.     

Functional imaging with cellular resolution reveals precise micro-architecture in visual cortex

Neurons in the cerebral cortex are organized into anatomical columns, with ensembles of cells arranged from the surface to the white matter. Within a column, neurons often share functional properties, such as selectivity for stimulus orientation; columns with distinct properties, such as different preferred orientations, tile the cortical surface in orderly patterns. This functional architecture was discovered with the relatively sparse sampling of microelectrode recordings. Optical imaging of membrane voltage or metabolic activity elucidated the overall geometry of functional maps, but is averaged over many cells (resolution >100 microm). Consequently, the purity of functional domains and the precision of the borders between them could not be resolved. Here, we labelled thousands of neurons of the visual cortex with a calcium-sensitive indicator in vivo. We then imaged the activity of neuronal populations at single-cell resolution with two-photon microscopy up to a depth of 400 microm. In rat primary visual cortex, neurons had robust orientation selectivity but there was no discernible local structure; neighbouring neurons often responded to different orientations. In area 18 of cat visual cortex, functional maps were organized at a fine scale. Neurons with opposite preferences for stimulus direction were segregated with extraordinary spatial precision in three dimensions, with columnar borders one to two cells wide. These results indicate that cortical maps can be built with single-cell precision.     

Mapping multiple features in the population response of visual cortex

Stimulus features such as edge orientation, motion direction and spatial frequency are thought to be encoded in the primary visual cortex by overlapping feature maps arranged so that the location of neurons activated by a particular combination of stimulus features can be predicted from the intersections of these maps. This view is based on the use of grating stimuli, which limit the range of stimulus combinations that can be examined. We used optical imaging of intrinsic signals in ferrets to assess patterns of population activity evoked by the motion of a texture (a field of iso-oriented bars). Here we show that the same neural population can be activated by multiple combinations of orientation, length, motion axis and speed. Rather than reflecting the intersection of multiple maps, our results indicate that population activity in primary visual cortex is better described as a single map of spatiotemporal energy.     

Highly ordered arrangement of single neurons in orientation pinwheels

In the visual cortex of higher mammals, neurons are arranged across the cortical surface in an orderly map of preferred stimulus orientations. This map contains 'orientation pinwheels', structures that are arranged like the spokes of a wheel such that orientation changes continuously around a centre. Conventional optical imaging first demonstrated these pinwheels, but the technique lacked the spatial resolution to determine the response properties and arrangement of cells near pinwheel centres. Electrophysiological recordings later demonstrated sharply selective neurons near pinwheel centres, but it remained unclear whether they were arranged randomly or in an orderly fashion. Here we use two-photon calcium imaging in vivo to determine the microstructure of pinwheel centres in cat visual cortex with single-cell resolution. We find that pinwheel centres are highly ordered: neurons selective to different orientations are clearly segregated even in the very centre. Thus, pinwheel centres truly represent singularities in the cortical map. This highly ordered arrangement at the level of single cells suggests great precision in the development of cortical circuits underlying orientation selectivity.     

Vernier acuity of neurones in cat visual cortex

The ability of human observers to detect Vernier breaks of as little as 5 s arc has been termed hyperacuity as this distance is substantially less than the angular separation of the bars of the highest spatial frequency of grating (approximately 1 arc min) that can be detected. Although the visual cortex is a likely candidate for the location of detectors involved in this performance, it is not known whether there are cells sensitive enough to detect deviations from co-linearity that are small compared with their spatial resolution (defined in terms of the highest spatial frequency that the cell can detect). We report here the results of physiological experiments on single units in area 17 of the cat visual cortex in which we studied the effect of introducing a Vernier break into a bar stimulus moved across the receptive field of the cell at a constant velocity. Our results show that the responses of most simple and complex cells are significantly reduced by the introduction of a Vernier break that is substantially smaller than the spatial resolution of the cell. The most sensitive cells in our sample could discriminate Vernier offsets of 3-6 arc min with a reliability of approximately 70%. This was much smaller than their spatial resolution, which was in the range 25-30 arc min. We interpret these results in terms of mechanisms that could underly the orientation selectivity of cortical neurones and suggest how our results relate to human Vernier acuity.     

Identifying natural images from human brain activity

A challenging goal in neuroscience is to be able to read out, or decode, mental content from brain activity. Recent functional magnetic resonance imaging (fMRI) studies have decoded orientation, position and object category from activity in visual cortex. However, these studies typically used relatively simple stimuli (for example, gratings) or images drawn from fixed categories (for example, faces, houses), and decoding was based on previous measurements of brain activity evoked by those same stimuli or categories. To overcome these limitations, here we develop a decoding method based on quantitative receptive-field models that characterize the relationship between visual stimuli and fMRI activity in early visual areas. These models describe the tuning of individual voxels for space, orientation and spatial frequency, and are estimated directly from responses evoked by natural images. We show that these receptive-field models make it possible to identify, from a large set of completely novel natural images, which specific image was seen by an observer. Identification is not a mere consequence of the retinotopic organization of visual areas; simpler receptive-field models that describe only spatial tuning yield much poorer identification performance. Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone.     

Orientation-specific cortical responses develop in early infancy

Neurones in the visual cortex of higher mammals differ from those elsewhere in the visual pathway in that the majority respond selectively to particular edge or bar orientations in the stimulus. We have developed a visually evoked potential (VEP) technique which isolates the response of orientation-selective mechanisms from that of cortical or sub-cortical neurones which lack orientation selectivity. We are unable to find such orientation-selective responses in newborn human infants within the sensitivity of our method, but repeated longitudinal testing of individual infants shows that measurable responses emerge around 6 weeks of age. This result is consistent with the idea that human cortical visual function is very immature at birth, but develops rapidly in the first two postnatal months.     

Endstopped neurons in the visual cortex as a substrate for calculating curvature

Neurons in the visual cortex typically respond selectively to the orientation, and velocity and direction of movement, of moving-bar stimuli. These responses are generally thought to provide information about the orientation and position of lines and edges in the visual field. Some cells are also endstopped, that is selective for bars of specific lengths. Hubel and Wiesel first observed that endstopped hypercomplex cells could respond to curved stimuli and suggested they might be involved in detection of curvature, but the exact relationship between endstopping and curvature has never been determined. We present here a mathematical model relating endstopping to curvature in which the difference in response of two simple cells gives rise to endstopping and varies in proportion to curvature. We also provide physiological evidence that endstopped cells in area 17 of the cat visual cortex are selective for curvature, whereas non-endstopped cells are not, and that some are selective for the sign of curvature. The prevailing view of edge and curve determination is that orientations are selected locally by the class of simple cortical cells and then integrated to form global curves. We have developed a computational theory of orientation selection which shows that measurements of orientation obtained by simple cells are not sufficient because there will be strong, incorrect responses from cells whose receptive fields (RFs) span distinct curves (Fig. 1). If estimates of curvature are available, however, these inappropriate responses can be eliminated. Curvature provides the key to structuring the network that underlies our theory and distinguishes it from previous lateral inhibition schemes.     

The contribution of sensory experience to the maturation of orientation selectivity in ferret visual cortex.

Sensory experience begins when neural circuits in the cerebral cortex are still immature; however, the contribution of experience to cortical maturation remains unclear. In the visual cortex, the selectivity of neurons for oriented stimuli at the time of eye opening is poor and increases dramatically after the onset of visual experience. Here we investigate whether visual experience has a significant role in the maturation of orientation selectivity and underlying cortical circuits using two forms of deprivation: dark rearing, which completely eliminates experience, and binocular lid suture, which alters the pattern of sensory driven activity. Orientation maps were present in dark-reared ferrets, but fully mature levels of tuning were never attained. In contrast, only rudimentary levels of orientation selectivity were observed in lid-sutured ferrets. Despite these differences, horizontal connections in both groups were less extensive and less clustered than normal, suggesting that long-range cortical processing is not essential for the expression of orientation selectivity, but may be needed for the full maturation of tuning. Thus, experience is beneficial or highly detrimental to cortical maturation, depending on the pattern of sensory driven activity.     

Subtractive and divisive adaptation in the human visual system.

Sensory systems can adapt to the conditions imposed on them. In the visual system, adapting to a pattern increases the threshold of the ability to see that pattern, and reduces the perceived contrast of the pattern above threshold. Most neurons of the striate cortex reduce their responsiveness after being stimulated for some time by a high-contrast pattern. Such an effect may lie behind these psychophysical adaptation phenomena. These adaptation effects have been reported to be confined to patterns of similar orientation, which is understandable in that the visual neurons that adapt are only excited by a small range of orientations. Neurophysiological evidence suggests that neurons with different orientation preferences have inhibitory interconnections. It is therefore of interest to explore the possible effects of these connections on perception. Here we show that adapting to a horizontal pattern can reduce the perceived contrast of a vertical test pattern more than a horizontal test pattern. These 'cross-orientation' effects are modelled by a division-like process, whereas the more normal 'similar-orientation' effects are modelled by a subtractive process.     

Iso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns

The mammalian cortex is organized in a columnar fashion: neurons lying below each other from the pia to the white matter usually share many functional properties. Across the cortical surface, cells with similar response properties are also clustered together, forming elongated bands or patches. Some response properties, such as orientation preference in the visual cortex, change gradually across the cortical surface forming 'orientation maps'. To determine the precise layout of iso-orientation domains, knowledge of responses not only to one but to many stimulus orientations is essential. Therefore, the exact depiction of orientation maps has been hampered by technical difficulties and remained controversial for almost thirty years. Here we use in vivo optical imaging based on intrinsic signals to gather information on the responses of a piece of cortex to gratings in many different orientations. This complete set of responses then provides detailed information on the structure of the orientation map in a large patch of cortex from area 18 of the cat. We find that cortical regions that respond best to one orientation form highly ordered patches rather than elongated bands. These iso-orientation patches are organized around 'orientation centres', producing pinwheel-like patterns in which the orientation preference of cells is changing continuously across the cortex. We have also analysed our data for fast changes in orientation preference and find that these 'fractures' are limited to the orientation centres. The pinwheels and orientation centres are such a prominent organizational feature that it should be important to understand their development as well as their function in the processing of visual information.     

Two types of orientation-sensitive responses of amacrine cells in the mammalian retina.

Neurons sensitive to the orientation of light stimuli exist throughout the mammalian visual system, suggesting that this spatial feature is a fundamental cue used by the brain to decipher visual information. The most peripheral neurons known to show orientation sensitivity are the retinal ganglion cells. Considerable morphological and pharmacological data suggest that the orientation sensitivity of ganglion cells is formed, at least partly, by the amacrine cells, which are laterally oriented interneurons presynaptic to the ganglion cells in the inner plexiform layer. So far there have been few studies of the responses of amacrine cells to oriented visual stimuli and their role in forming orientation-sensitive responses in the retina remains unclear. Here I report the novel finding of a population of amacrine cells in the rabbit retina which are orientation-sensitive. These amacrine cells can be divided into two subtypes, whose orientation sensitivity is manufactured by two distinct mechanisms. The orientation sensitivity of the first subtype of amacrine cell is formed from the interactions of excitatory, centre-receptive field synaptic inputs and inhibitory inputs of opposite polarity, whereas that for cells of the second subtype seems to be the product of a marked asymmetry in their dendritic arbors.     

Miniature eye movements enhance fine spatial detail

Our eyes are constantly in motion. Even during visual fixation, small eye movements continually jitter the location of gaze. It is known that visual percepts tend to fade when retinal image motion is eliminated in the laboratory. However, it has long been debated whether, during natural viewing, fixational eye movements have functions in addition to preventing the visual scene from fading. In this study, we analysed the influence in humans of fixational eye movements on the discrimination of gratings masked by noise that has a power spectrum similar to that of natural images. Using a new method of retinal image stabilization, we selectively eliminated the motion of the retinal image that normally occurs during the intersaccadic intervals of visual fixation. Here we show that fixational eye movements improve discrimination of high spatial frequency stimuli, but not of low spatial frequency stimuli. This improvement originates from the temporal modulations introduced by fixational eye movements in the visual input to the retina, which emphasize the high spatial frequency harmonics of the stimulus. In a natural visual world dominated by low spatial frequencies, fixational eye movements appear to constitute an effective sampling strategy by which the visual system enhances the processing of spatial detail.     

Practising orientation identification improves orientation coding in V1 neurons

The adult brain shows remarkable plasticity, as demonstrated by the improvement in fine sensorial discriminations after intensive practice. The behavioural aspects of such perceptual learning are well documented, especially in the visual system. Specificity for stimulus attributes clearly implicates an early cortical site, where receptive fields retain fine selectivity for these attributes; however, the neuronal correlates of a simple visual discrimination task remained unidentified. Here we report electrophysiological correlates in the primary visual cortex (V1) of monkeys for learning orientation identification. We link the behavioural improvement in this type of learning to an improved neuronal performance of trained compared to naive neurons. Improved long-term neuronal performance resulted from changes in the characteristics of orientation tuning of individual neurons. More particularly, the slope of the orientation tuning curve that was measured at the trained orientation increased only for the subgroup of trained neurons most likely to code the orientation identified by the monkey. No modifications of the tuning curve were observed for orientations for which the monkey had not been trained. Thus training induces a specific and efficient increase in neuronal sensitivity in V1.     

Motion streaks provide a spatial code for motion direction.

Although many neurons in the primary visual cortex (V1) of primates are direction selective, they provide ambiguous information about the direction of motion of a stimulus. There is evidence that one of the ways in which the visual system resolves this ambiguity is by computing, from the responses of V1 neurons, velocity components in two or more spatial orientations and then combining these velocity components. Here I consider another potential neural mechanism for determining motion direction. When a localized image feature moves fast enough, it should become smeared in space owing to temporal integration in the visual system, creating a spatial signal-a 'motion streak'-oriented in the direction of the motion. The orientation masking and adaptation experiments reported here show that these spatial signals for motion direction exist in the human visual system for feature speeds above about 1 feature width per 100 ms. Computer simulations show that this psychophysical finding is consistent with the known response properties of V1 neurons, and that these spatial signals, when appropriately processed, are sufficient to determine motion direction in natural images.     

Visual discrimination of target displacement remains after damage to the striate cortex in humans

Damage to the striate cortex usually causes blindness in those regions of the visual field which map to the area of neural damage. Nonetheless, there are reports that some patients with such damage can localize and perform certain visual discriminations between light stimuli presented within the 'blind' area of the visual field. Experiments on animals with different brain areas ablated suggest that visual function is served by two principal projection pathways from the retina. That to the striate cortex is primarily responsible for fine discrimination between stimulus parameters such as colour and spatial pattern, whereas that to the superior colliculus in the midbrain is responsible for visual localization of stimuli. The residual visual functions in patients with cortical damage are usually attributed to the non-striate retinal projection to the superior colliculus. We now present measurements of spatial discrimination in two observers with large visual field defects (scotomata) caused by damage to the striate cortical region. Both exhibit a near normal ability to discriminate displacements of targets when two lights are flashed sequentially in their defective visual field, but they are unable to discriminate spatial pattern or size. We argue that these results are consistent with the 'two visual systems' interpretation of ablation studies on non-human species.     

Cortical remodelling induced by activity of ventral tegmental dopamine neurons.

Representations of sensory stimuli in the cerebral cortex can undergo progressive remodelling according to the behavioural importance of the stimuli. The cortex receives widespread projections from dopamine neurons in the ventral tegmental area (VTA), which are activated by new stimuli or unpredicted rewards, and are believed to provide a reinforcement signal for such learning-related cortical reorganization. In the primary auditory cortex (AI) dopamine release has been observed during auditory learning that remodels the sound-frequency representations. Furthermore, dopamine modulates long-term potentiation, a putative cellular mechanism underlying plasticity. Here we show that stimulating the VTA together with an auditory stimulus of a particular tone increases the cortical area and selectivity of the neural responses to that sound stimulus in AI. Conversely, the AI representations of nearby sound frequencies are selectively decreased. Strong, sharply tuned responses to the paired tones also emerge in a second cortical area, whereas the same stimuli evoke only poor or non-selective responses in this second cortical field in naive animals. In addition, we found that strong long-range coherence of neuronal discharge emerges between AI and this secondary auditory cortical area.     

Network anatomy and in vivo physiology of visual cortical neurons

In the cerebral cortex, local circuits consist of tens of thousands of neurons, each of which makes thousands of synaptic connections. Perhaps the biggest impediment to understanding these networks is that we have no wiring diagrams of their interconnections. Even if we had a partial or complete wiring diagram, however, understanding the network would also require information about each neuron's function. Here we show that the relationship between structure and function can be studied in the cortex with a combination of in vivo physiology and network anatomy. We used two-photon calcium imaging to characterize a functional property--the preferred stimulus orientation--of a group of neurons in the mouse primary visual cortex. Large-scale electron microscopy of serial thin sections was then used to trace a portion of these neurons' local network. Consistent with a prediction from recent physiological experiments, inhibitory interneurons received convergent anatomical input from nearby excitatory neurons with a broad range of preferred orientations, although weak biases could not be rejected.     

Selective responses of visual cortical cells do not depend on shunting inhibition

Theoretical analyses of the electrical behaviour of the highly branched processes of nerve cells has focused attention on the possibility that single cells perform complex logical operations rather than simply summing their synaptic inputs. In particular, it has been suggested that the orientation and direction selectivity of cells in the visual cortex results from the action of a nonlinear 'shunting' inhibition that emulates an AND-NOT logical operation. The characteristic biophysical feature of this proposed inhibitory mechanism is that it evokes a large and relatively sustained increase in the conductance of the neuronal membrane while leaving the membrane potential unaffected. This shunting mechanism contrasts with linear 'summative' inhibition in which conductance changes are less prominent, and inhibition is achieved by hyperpolarization of the membrane potential. In a direct experimental test of the hypothesis that the selectivity of visual cortical neurons depends on shunting inhibition we found no evidence for the large conductance changes predicted by the theory.     

Experience with moving visual stimuli drives the early development of cortical direction selectivity

The onset of vision occurs when neural circuits in the visual cortex are immature, lacking both the full complement of connections and the response selectivity that defines functional maturity. Direction-selective responses are particularly vulnerable to the effects of early visual deprivation, but it remains unclear how stimulus-driven neural activity guides the emergence of cortical direction selectivity. Here we report observations from a motion training protocol that allowed us to monitor the impact of experience on the development of direction-selective responses in visually naive ferrets. Using intrinsic signal imaging techniques, we found that training with a single axis of motion induced the rapid emergence of direction columns that were confined to cortical regions preferentially activated by the training stimulus. Using two-photon calcium imaging techniques, we found that single neurons in visually naive animals exhibited weak directional biases and lacked the strong local coherence in the spatial organization of direction preference that was evident in mature animals. Training with a moving stimulus, but not with a flashed stimulus, strengthened the direction-selective responses of individual neurons and preferentially reversed the direction biases of neurons that deviated from their neighbours. Both effects contributed to an increase in local coherence. We conclude that early experience with moving visual stimuli drives the rapid emergence of direction-selective responses in the visual cortex.