Experience-dependent representation of visual categories in parietal cortex

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as 'chair', 'table' and 'vehicle', which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360 degrees of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP--an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making-robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.     

Neuronal switching of sensorimotor transformations for antisaccades

The influence of cognitive context on orienting behaviour can be explored using the mixed memory-prosaccade, memory-antisaccade task. A symbolic cue, such as the colour of a visual stimulus, instructs the subject to make a brief, rapid eye movement (a saccade) either towards the stimulus (prosaccade) or in the opposite direction (antisaccade). Thus, the appropriate sensorimotor transformation must be switched on to execute the instructed task. Despite advances in our understanding of the neuronal processing of antisaccades, it remains unclear how the brain selects and computes the sensorimotor transformation leading to an antisaccade. Here we show that area LIP of the posterior parietal cortex is involved in these processes. LIP's population activity turns from the visual direction to the motor direction during memory-antisaccade trials. About one-third of the visual neurons in LIP produce a brisk, transient discharge in certain memory-antisaccade trials. We call this discharge 'paradoxical' because its timing is visual-like but its direction is motor. The paradoxical discharge shows, first, that switching occurs already at the level of visual cells, as previously proposed by Schlag-Rey and colleagues; and second, that this switching is accomplished very rapidly, within 50 ms from the arrival of the visual signals in LIP.     

Movement selection in advance of action in the superior colliculus.

The primate superior colliculus contains a map of saccadic eye movements. Saccades are high-velocity eye movements to selected targets in the visual field, but little is known about the neural mechanisms responsible for target selection or the related problem of choosing a particular movement from the oculomotor repertoire. Two classes of neurons have been described in the superior colliculus which show bursts of activity before the saccade: discrete bursters display a vigorous pre-saccadic burst and prelude bursters show low-frequency activity as a prelude to burst onset. We have designed experiments to test whether prelude activity is related to saccade selection. Our tasks use a cue to specify which of two physically identical visual stimuli is the goal of an impending saccade. This cue is spatially and temporally isolated from the potential targets as well as from visual cues signalling movement initiation. Our results show that prelude activity occurs shortly after information is available for correct saccade selection and, more importantly, the activity is predictive of saccade choice. The results thus suggest that the superior colliculus participates in the process of saccade selection.     

The contribution of sensory experience to the maturation of orientation selectivity in ferret visual cortex.

Sensory experience begins when neural circuits in the cerebral cortex are still immature; however, the contribution of experience to cortical maturation remains unclear. In the visual cortex, the selectivity of neurons for oriented stimuli at the time of eye opening is poor and increases dramatically after the onset of visual experience. Here we investigate whether visual experience has a significant role in the maturation of orientation selectivity and underlying cortical circuits using two forms of deprivation: dark rearing, which completely eliminates experience, and binocular lid suture, which alters the pattern of sensory driven activity. Orientation maps were present in dark-reared ferrets, but fully mature levels of tuning were never attained. In contrast, only rudimentary levels of orientation selectivity were observed in lid-sutured ferrets. Despite these differences, horizontal connections in both groups were less extensive and less clustered than normal, suggesting that long-range cortical processing is not essential for the expression of orientation selectivity, but may be needed for the full maturation of tuning. Thus, experience is beneficial or highly detrimental to cortical maturation, depending on the pattern of sensory driven activity.     

Influence of the thalamus on spatial visual processing in frontal cortex

Each of our movements activates our own sensory receptors, and therefore keeping track of self-movement is a necessary part of analysing sensory input. One way in which the brain keeps track of self-movement is by monitoring an internal copy, or corollary discharge, of motor commands. This concept could explain why we perceive a stable visual world despite our frequent quick, or saccadic, eye movements: corollary discharge about each saccade would permit the visual system to ignore saccade-induced visual changes. The critical missing link has been the connection between corollary discharge and visual processing. Here we show that such a link is formed by a corollary discharge from the thalamus that targets the frontal cortex. In the thalamus, neurons in the mediodorsal nucleus relay a corollary discharge of saccades from the midbrain superior colliculus to the cortical frontal eye field. In the frontal eye field, neurons use corollary discharge to shift their visual receptive fields spatially before saccades. We tested the hypothesis that these two components-a pathway for corollary discharge and neurons with shifting receptive fields-form a circuit in which the corollary discharge drives the shift. First we showed that the known spatial and temporal properties of the corollary discharge predict the dynamic changes in spatial visual processing of cortical neurons when saccades are made. Then we moved from this correlation to causation by isolating single cortical neurons and showing that their spatial visual processing is impaired when corollary discharge from the thalamus is interrupted. Thus the visual processing of frontal neurons is spatiotemporally matched with, and functionally dependent on, corollary discharge input from the thalamus. These experiments establish the first link between corollary discharge and visual processing, delineate a brain circuit that is well suited for mediating visual stability, and provide a framework for studying corollary discharge in other sensory systems.     

Direct visuomotor transformations for reaching

The posterior parietal cortex (PPC) is thought to have a function in the sensorimotor transformations that underlie visually guided reaching, as damage to the PPC can result in difficulty reaching to visual targets in the absence of specific visual or motor deficits. This function is supported by findings that PPC neurons in monkeys are modulated by the direction of hand movement, as well as by visual, eye position and limb position signals. The PPC could transform visual target locations from retinal coordinates to hand-centred coordinates by combining sensory signals in a serial manner to yield a body-centred representation of target location, and then subtracting the body-centred location of the hand. We report here that in dorsal area 5 of the PPC, remembered target locations are coded with respect to both the eye and hand. This suggests that the PPC transforms target locations directly between these two reference frames. Data obtained in the adjacent parietal reach region (PRR) indicate that this transformation may be achieved by vectorially subtracting hand location from target location, with both locations represented in eye-centred coordinates.     

Optimal eye movement strategies in visual search

To perform visual search, humans, like many mammals, encode a large field of view with retinas having variable spatial resolution, and then use high-speed eye movements to direct the highest-resolution region, the fovea, towards potential target locations. Good search performance is essential for survival, and hence mammals may have evolved efficient strategies for selecting fixation locations. Here we address two questions: what are the optimal eye movement strategies for a foveated visual system faced with the problem of finding a target in a cluttered environment, and do humans employ optimal eye movement strategies during a search? We derive the ideal bayesian observer for search tasks in which a target is embedded at an unknown location within a random background that has the spectral characteristics of natural scenes. Our ideal searcher uses precise knowledge about the statistics of the scenes in which the target is embedded, and about its own visual system, to make eye movements that gain the most information about target location. We find that humans achieve nearly optimal search performance, even though humans integrate information poorly across fixations. Analysis of the ideal searcher reveals that there is little benefit from perfect integration across fixations--much more important is efficient processing of information on each fixation. Apparently, evolution has exploited this fact to achieve efficient eye movement strategies with minimal neural resources devoted to memory.     

Potentiation of cortical inhibition by visual deprivation

The fine-tuning of circuits in sensory cortex requires sensory experience during an early critical period. Visual deprivation during the critical period has catastrophic effects on visual function, including loss of visual responsiveness to the deprived eye, reduced visual acuity, and loss of tuning to many stimulus characteristics. These changes occur faster than the remodelling of thalamocortical axons, but the intracortical plasticity mechanisms that underlie them are incompletely understood. Long-term depression of excitatory intracortical synapses has been proposed as a general candidate mechanism for the loss of cortical responsiveness after visual deprivation. Alternatively (or in addition), the decreased ability of the deprived eye to activate cortical neurons could be due to enhanced intracortical inhibition. Here we show that visual deprivation leaves excitatory connections in layer 4 (the primary input layer to cortex) unaffected, but markedly potentiates inhibitory feedback between fast-spiking basket cells (FS cells) and star pyramidal neurons (star pyramids). Further, a previously undescribed form of long-term potentiation of inhibition (LTPi) could be induced at synapses from FS cells to star pyramids, and was occluded by previous visual deprivation. These data suggest that potentiation of inhibition is a major cellular mechanism underlying the deprivation-induced degradation of visual function, and that this form of LTPi is important in fine-tuning cortical circuitry in response to visual experience.     

Wavelength sensitivity in blindsight

Blindsight--the residual visual functions observed in visualfield defects resulting from destruction of part of the primary visual cortex (striate cortex) even though visual stimuli presented in the field defect are not consciously perceived--has generated new insights into the nature of consciousness and the role of the extrastriate pathways in visual processing. Some patients can detect and localize unseen stimuli when they are required to guess. Discrimination of movement, flicker and orientation may also be present, but residual colour discrimination is controversial. Negative results imply that only the pathways from eye to striate cortex can transmit information about colour in primates. By measuring sensitivity to light of different wavelengths in patients with blindsight we show that spectral sensitivity in the blind fields is surprisingly high, with a reduction of only 1 log unit or less across the visible spectrum. It is also essentially normal in form, whether the patients are adapted to light or dark. The shift in peak sensitivity from medium to shorter wavelengths in adaptation to the dark (the Purkinje shift) and the presence of discontinuities in the light-adapted curve together show that blindsight involves both rod and cone contributions, and that some colour opponency remains. As colour opponency requires input from primate beta retinal ganglion cells, two-thirds of which degenerate transneurally after a striate cortical lesion in juvenile monkeys, our results show that the surviving subpopulation of primate beta cells is functional.     

Functional specificity of local synaptic connections in neocortical networks

Neuronal connectivity is fundamental to information processing in the brain. Therefore, understanding the mechanisms of sensory processing requires uncovering how connection patterns between neurons relate to their function. On a coarse scale, long-range projections can preferentially link cortical regions with similar responses to sensory stimuli. But on the local scale, where dendrites and axons overlap substantially, the functional specificity of connections remains unknown. Here we determine synaptic connectivity between nearby layer 2/3 pyramidal neurons in vitro, the response properties of which were first characterized in mouse visual cortex in vivo. We found that connection probability was related to the similarity of visually driven neuronal activity. Neurons with the same preference for oriented stimuli connected at twice the rate of neurons with orthogonal orientation preferences. Neurons responding similarly to naturalistic stimuli formed connections at much higher rates than those with uncorrelated responses. Bidirectional synaptic connections were found more frequently between neuronal pairs with strongly correlated visual responses. Our results reveal the degree of functional specificity of local synaptic connections in the visual cortex, and point to the existence of fine-scale subnetworks dedicated to processing related sensory information.     

Miniature eye movements enhance fine spatial detail

Our eyes are constantly in motion. Even during visual fixation, small eye movements continually jitter the location of gaze. It is known that visual percepts tend to fade when retinal image motion is eliminated in the laboratory. However, it has long been debated whether, during natural viewing, fixational eye movements have functions in addition to preventing the visual scene from fading. In this study, we analysed the influence in humans of fixational eye movements on the discrimination of gratings masked by noise that has a power spectrum similar to that of natural images. Using a new method of retinal image stabilization, we selectively eliminated the motion of the retinal image that normally occurs during the intersaccadic intervals of visual fixation. Here we show that fixational eye movements improve discrimination of high spatial frequency stimuli, but not of low spatial frequency stimuli. This improvement originates from the temporal modulations introduced by fixational eye movements in the visual input to the retina, which emphasize the high spatial frequency harmonics of the stimulus. In a natural visual world dominated by low spatial frequencies, fixational eye movements appear to constitute an effective sampling strategy by which the visual system enhances the processing of spatial detail.     

Selective deficit of visual search in moving displays after extrastriate damage

A visual cue that is often associated with significant stimuli, such as those provided by prey and predators, is movement relative to the observer. An efficient visual system should be able to direct attention to those parts of the visual field that contain such stimuli. What is needed is a system that can filter by movement difference. This could direct attention to a moving item among stationary items, or an item moving in one direction against a background moving in a different direction. Visual search experiments have shown that people are indeed able to filter by movement; that is, they can attend to just the moving items in arrays of moving and stationary stimuli. Single-cell recordings from monkey visual cortex show that the medial temporal cortical area (MT) has some of the properties required to filter by movement. We have now linked these two observations by showing that a patient with bilateral lesions to the presumed human homologue of MT cannot restrict visual attention to the moving items in arrays of both moving and stationary items. This suggests that MT is the site of a movement filter used in normal visual processing.     

Performance monitoring by the supplementary eye field

Intelligent behaviour requires self-control based on the consequences of actions. The countermanding task is designed to study self-control; it requires subjects to withhold planned movements in response to an imperative stop signal, which they can do with varying success. In humans, the medial frontal cortex has been implicated in the supervisory control of action. In monkeys, the supplementary eye field in the dorsomedial frontal cortex is involved in producing eye movements, but its precise function has not been clarified. To investigate the role of the supplementary eye field in the control of eye movements, we recorded neural activity in macaque monkeys trained to perform an eye movement countermanding task. Distinct groups of neurons were active after errors, after successful withholding of a partially prepared movement, or in association with reinforcement. These three forms of activation could not be explained by sensory or motor factors. Our results lead us to put forward the hypothesis that the supplementary eye field contributes to monitoring the context and consequences of eye movements.     

Control of visual cortical signals by prefrontal dopamine

The prefrontal cortex is thought to modulate sensory signals in posterior cortices during top-down attention, but little is known about the underlying neural circuitry. Experimental and clinical evidence indicate that prefrontal dopamine has an important role in cognitive functions, acting predominantly through D1 receptors. Here we show that dopamine D1 receptors mediate prefrontal control of signals in the visual cortex of macaques (Macaca mulatta). We pharmacologically altered D1-receptor-mediated activity in the frontal eye field of the prefrontal cortex and measured the effect on the responses of neurons in area V4 of the visual cortex. This manipulation was sufficient to enhance the magnitude, the orientation selectivity and the reliability of V4 visual responses to an extent comparable with the known effects of top-down attention. The enhancement of V4 signals was restricted to neurons with response fields overlapping the part of visual space affected by the D1 receptor manipulation. Altering either D1- or D2-receptor-mediated frontal eye field activity increased saccadic target selection but the D2 receptor manipulation did not enhance V4 signals. Our results identify a role for D1 receptors in mediating the control of visual cortical signals by the prefrontal cortex and suggest how processing in sensory areas could be altered in mental disorders involving prefrontal dopamine.     

Top-down gain control of the auditory space map by gaze control circuitry in the barn owl

High-level circuits in the brain that control the direction of gaze are intimately linked with the control of visual spatial attention. Immediately before an animal directs its gaze towards a stimulus, both psychophysical sensitivity to that visual stimulus and the responsiveness of high-order neurons in the cerebral cortex that represent the stimulus increase dramatically. Equivalent effects on behavioural sensitivity and neuronal responsiveness to visual stimuli result from focal electrical microstimulation of gaze control centres in monkeys. Whether the gaze control system modulates neuronal responsiveness in sensory modalities other than vision is unknown. Here we show that electrical microstimulation applied to gaze control circuitry in the forebrain of barn owls regulates the gain of midbrain auditory responses in an attention-like manner. When the forebrain circuit was activated, midbrain responses to auditory stimuli at the location encoded by the forebrain site were enhanced and spatial selectivity was sharpened. The same stimulation suppressed responses to auditory stimuli represented at other locations in the midbrain map. Such space-specific, top-down regulation of auditory responses by gaze control circuitry in the barn owl suggests that the central nervous system uses a common strategy for dynamically regulating sensory gain that applies across modalities, brain areas and classes of vertebrate species. This approach provides a path for discovering mechanisms that underlie top-down gain control in the central nervous system.     

Experience with moving visual stimuli drives the early development of cortical direction selectivity

The onset of vision occurs when neural circuits in the visual cortex are immature, lacking both the full complement of connections and the response selectivity that defines functional maturity. Direction-selective responses are particularly vulnerable to the effects of early visual deprivation, but it remains unclear how stimulus-driven neural activity guides the emergence of cortical direction selectivity. Here we report observations from a motion training protocol that allowed us to monitor the impact of experience on the development of direction-selective responses in visually naive ferrets. Using intrinsic signal imaging techniques, we found that training with a single axis of motion induced the rapid emergence of direction columns that were confined to cortical regions preferentially activated by the training stimulus. Using two-photon calcium imaging techniques, we found that single neurons in visually naive animals exhibited weak directional biases and lacked the strong local coherence in the spatial organization of direction preference that was evident in mature animals. Training with a moving stimulus, but not with a flashed stimulus, strengthened the direction-selective responses of individual neurons and preferentially reversed the direction biases of neurons that deviated from their neighbours. Both effects contributed to an increase in local coherence. We conclude that early experience with moving visual stimuli drives the rapid emergence of direction-selective responses in the visual cortex.     

Cross-modal and cross-temporal association in neurons of frontal cortex

The prefrontal cortex is essential for the temporal integration of sensory information in behavioural and linguistic sequences. Such information is commonly encoded in more than one sense modality, notably sight and sound. Connections from sensory cortices to the prefrontal cortex support its integrative function. Here we present the first evidence that prefrontal cortex cells associate visual and auditory stimuli across time. We gave monkeys the task of remembering a tone of a certain pitch for 10 s and then choosing the colour associated with it. In this task, prefrontal cortex cells responded selectively to tones, and most of them also responded to colours according to the task rule. Thus, their reaction to a tone was correlated with their subsequent reaction to the associated colour. This correlation faltered in trials ending in behavioural error. We conclude that prefrontal cortex neurons are part of integrative networks that represent behaviourally meaningful cross-modal associations. The orderly and timely activation of neurons in such networks is crucial for the temporal transfer of information in the structuring of behaviour, reasoning and language.     

Transformation from temporal to rate coding in a somatosensory thalamocortical pathway

The anatomical connections from the whiskers to the rodent somatosensory (barrel) cortex form two parallel (lemniscal and paralemniscal) pathways. It is unclear whether the paralemniscal pathway is directly involved in tactile processing, because paralemniscal neuronal responses show poor spatial resolution, labile latencies and strong dependence on cortical feedback. Here we show that the paralemniscal system can transform temporally encoded vibrissal information into a rate code. We recorded the representations of the frequency of whisker movement along the two pathways in anaesthetized rats. In response to varying stimulus frequencies, the lemniscal neurons exhibited amplitude modulations and constant latencies. In contrast, paralemniscal neurons in both thalamus and cortex coded the input frequency as changes in latency. Because the onset latencies increased and the offset latencies remained constant, the latency increments were translated into a rate code: increasing onset latencies led to lower spike counts. A thalamocortical loop that includes cortical oscillations and thalamic gating can account for these results. Thus, variable latencies and effective cortical feedback in the paralemniscal system can serve the processing of temporal sensory cues, such as those that encode object location during whisking. In contrast, fixed time locking in the lemniscal system is crucial for reliable spatial processing.     

Small modulation of ongoing cortical dynamics by sensory input during natural vision

During vision, it is believed that neural activity in the primary visual cortex is predominantly driven by sensory input from the environment. However, visual cortical neurons respond to repeated presentations of the same stimulus with a high degree of variability. Although this variability has been considered to be noise owing to random spontaneous activity within the cortex, recent studies show that spontaneous activity has a highly coherent spatio-temporal structure. This raises the possibility that the pattern of this spontaneous activity may shape neural responses during natural viewing conditions to a larger extent than previously thought. Here, we examine the relationship between spontaneous activity and the response of primary visual cortical neurons to dynamic natural-scene and random-noise film images in awake, freely viewing ferrets from the time of eye opening to maturity. The correspondence between evoked neural activity and the structure of the input signal was weak in young animals, but systematically improved with age. This improvement was linked to a shift in the dynamics of spontaneous activity. At all ages including the mature animal, correlations in spontaneous neural firing were only slightly modified by visual stimulation, irrespective of the sensory input. These results suggest that in both the developing and mature visual cortex, sensory evoked neural activity represents the modulation and triggering of ongoing circuit dynamics by input signals, rather than directly reflecting the structure of the input signal itself.     

Adaptive coding of visual information in neural populations

Our perception of the environment relies on the capacity of neural networks to adapt rapidly to changes in incoming stimuli. It is increasingly being realized that the neural code is adaptive, that is, sensory neurons change their responses and selectivity in a dynamic manner to match the changes in input stimuli. Understanding how rapid exposure, or adaptation, to a stimulus of fixed structure changes information processing by cortical networks is essential for understanding the relationship between sensory coding and behaviour. Physiological investigations of adaptation have contributed greatly to our understanding of how individual sensory neurons change their responses to influence stimulus coding, yet whether and how adaptation affects information coding in neural populations is unknown. Here we examine how brief adaptation (on the timescale of visual fixation) influences the structure of interneuronal correlations and the accuracy of population coding in the macaque (Macaca mulatta) primary visual cortex (V1). We find that brief adaptation to a stimulus of fixed structure reorganizes the distribution of correlations across the entire network by selectively reducing their mean and variability. The post-adaptation changes in neuronal correlations are associated with specific, stimulus-dependent changes in the efficiency of the population code, and are consistent with changes in perceptual performance after adaptation. Our results have implications beyond the predictions of current theories of sensory coding, suggesting that brief adaptation improves the accuracy of population coding to optimize neuronal performance during natural viewing.