Experience-dependent representation of visual categories in parietal cortex

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as 'chair', 'table' and 'vehicle', which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360 degrees of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP--an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making-robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.     

Global and fine information coded by single neurons in the temporal visual cortex.

When we see a person's face, we can easily recognize their species, individual identity and emotional state. How does the brain represent such complex information? A substantial number of neurons in the macaque temporal cortex respond to faces. However, the neuronal mechanisms underlying the processing of complex information are not yet clear. Here we recorded the activity of single neurons in the temporal cortex of macaque monkeys while presenting visual stimuli consisting of geometric shapes, and monkey and human faces with various expressions. Information theory was used to investigate how well the neuronal responses could categorize the stimuli. We found that single neurons conveyed two different scales of facial information in their firing patterns, starting at different latencies. Global information, categorizing stimuli as monkey faces, human faces or shapes, was conveyed in the earliest part of the responses. Fine information about identity or expression was conveyed later, beginning on average 51 ms after global information. We speculate that global information could be used as a 'header' to prepare destination areas for receiving more detailed information.     

Microstimulation of inferotemporal cortex influences face categorization

The inferior temporal cortex (IT) of primates is thought to be the final visual area in the ventral stream of cortical areas responsible for object recognition. Consistent with this hypothesis, single IT neurons respond selectively to highly complex visual stimuli such as faces. However, a direct causal link between the activity of face-selective neurons and face perception has not been demonstrated. In the present study of macaque monkeys, we artificially activated small clusters of IT neurons by means of electrical microstimulation while the monkeys performed a categorization task, judging whether noisy visual images belonged to 'face' or 'non-face' categories. Here we show that microstimulation of face-selective sites, but not other sites, strongly biased the monkeys' decisions towards the face category. The magnitude of the effect depended upon the degree of face selectivity of the stimulation site, the size of the stimulated cluster of face-selective neurons, and the exact timing of microstimulation. Our results establish a causal relationship between the activity of face-selective neurons and face perception.     

Feature-based attention influences motion processing gain in macaque visual cortex.

Changes in neural responses based on spatial attention have been demonstrated in many areas of visual cortex, indicating that the neural correlate of attention is an enhanced response to stimuli at an attended location and reduced responses to stimuli elsewhere. Here we demonstrate non-spatial, feature-based attentional modulation of visual motion processing, and show that attention increases the gain of direction-selective neurons in visual cortical area MT without narrowing the direction-tuning curves. These findings place important constraints on the neural mechanisms of attention and we propose to unify the effects of spatial location, direction of motion and other features of the attended stimuli in a 'feature similarity gain model' of attention.     

Top-down gain control of the auditory space map by gaze control circuitry in the barn owl

High-level circuits in the brain that control the direction of gaze are intimately linked with the control of visual spatial attention. Immediately before an animal directs its gaze towards a stimulus, both psychophysical sensitivity to that visual stimulus and the responsiveness of high-order neurons in the cerebral cortex that represent the stimulus increase dramatically. Equivalent effects on behavioural sensitivity and neuronal responsiveness to visual stimuli result from focal electrical microstimulation of gaze control centres in monkeys. Whether the gaze control system modulates neuronal responsiveness in sensory modalities other than vision is unknown. Here we show that electrical microstimulation applied to gaze control circuitry in the forebrain of barn owls regulates the gain of midbrain auditory responses in an attention-like manner. When the forebrain circuit was activated, midbrain responses to auditory stimuli at the location encoded by the forebrain site were enhanced and spatial selectivity was sharpened. The same stimulation suppressed responses to auditory stimuli represented at other locations in the midbrain map. Such space-specific, top-down regulation of auditory responses by gaze control circuitry in the barn owl suggests that the central nervous system uses a common strategy for dynamically regulating sensory gain that applies across modalities, brain areas and classes of vertebrate species. This approach provides a path for discovering mechanisms that underlie top-down gain control in the central nervous system.     

Identifying natural images from human brain activity

A challenging goal in neuroscience is to be able to read out, or decode, mental content from brain activity. Recent functional magnetic resonance imaging (fMRI) studies have decoded orientation, position and object category from activity in visual cortex. However, these studies typically used relatively simple stimuli (for example, gratings) or images drawn from fixed categories (for example, faces, houses), and decoding was based on previous measurements of brain activity evoked by those same stimuli or categories. To overcome these limitations, here we develop a decoding method based on quantitative receptive-field models that characterize the relationship between visual stimuli and fMRI activity in early visual areas. These models describe the tuning of individual voxels for space, orientation and spatial frequency, and are estimated directly from responses evoked by natural images. We show that these receptive-field models make it possible to identify, from a large set of completely novel natural images, which specific image was seen by an observer. Identification is not a mere consequence of the retinotopic organization of visual areas; simpler receptive-field models that describe only spatial tuning yield much poorer identification performance. Our results suggest that it may soon be possible to reconstruct a picture of a person's visual experience from measurements of brain activity alone.     

Experience with moving visual stimuli drives the early development of cortical direction selectivity

The onset of vision occurs when neural circuits in the visual cortex are immature, lacking both the full complement of connections and the response selectivity that defines functional maturity. Direction-selective responses are particularly vulnerable to the effects of early visual deprivation, but it remains unclear how stimulus-driven neural activity guides the emergence of cortical direction selectivity. Here we report observations from a motion training protocol that allowed us to monitor the impact of experience on the development of direction-selective responses in visually naive ferrets. Using intrinsic signal imaging techniques, we found that training with a single axis of motion induced the rapid emergence of direction columns that were confined to cortical regions preferentially activated by the training stimulus. Using two-photon calcium imaging techniques, we found that single neurons in visually naive animals exhibited weak directional biases and lacked the strong local coherence in the spatial organization of direction preference that was evident in mature animals. Training with a moving stimulus, but not with a flashed stimulus, strengthened the direction-selective responses of individual neurons and preferentially reversed the direction biases of neurons that deviated from their neighbours. Both effects contributed to an increase in local coherence. We conclude that early experience with moving visual stimuli drives the rapid emergence of direction-selective responses in the visual cortex.     

Spatio-temporal correlations and visual signalling in a complete neuronal population

Statistical dependencies in the responses of sensory neurons govern both the amount of stimulus information conveyed and the means by which downstream neurons can extract it. Although a variety of measurements indicate the existence of such dependencies, their origin and importance for neural coding are poorly understood. Here we analyse the functional significance of correlated firing in a complete population of macaque parasol retinal ganglion cells using a model of multi-neuron spike responses. The model, with parameters fit directly to physiological data, simultaneously captures both the stimulus dependence and detailed spatio-temporal correlations in population responses, and provides two insights into the structure of the neural code. First, neural encoding at the population level is less noisy than one would expect from the variability of individual neurons: spike times are more precise, and can be predicted more accurately when the spiking of neighbouring neurons is taken into account. Second, correlations provide additional sensory information: optimal, model-based decoding that exploits the response correlation structure extracts 20% more information about the visual scene than decoding under the assumption of independence, and preserves 40% more visual information than optimal linear decoding. This model-based approach reveals the role of correlated activity in the retinal coding of visual stimuli, and provides a general framework for understanding the importance of correlated activity in populations of neurons.     

Functional specificity of local synaptic connections in neocortical networks

Neuronal connectivity is fundamental to information processing in the brain. Therefore, understanding the mechanisms of sensory processing requires uncovering how connection patterns between neurons relate to their function. On a coarse scale, long-range projections can preferentially link cortical regions with similar responses to sensory stimuli. But on the local scale, where dendrites and axons overlap substantially, the functional specificity of connections remains unknown. Here we determine synaptic connectivity between nearby layer 2/3 pyramidal neurons in vitro, the response properties of which were first characterized in mouse visual cortex in vivo. We found that connection probability was related to the similarity of visually driven neuronal activity. Neurons with the same preference for oriented stimuli connected at twice the rate of neurons with orthogonal orientation preferences. Neurons responding similarly to naturalistic stimuli formed connections at much higher rates than those with uncorrelated responses. Bidirectional synaptic connections were found more frequently between neuronal pairs with strongly correlated visual responses. Our results reveal the degree of functional specificity of local synaptic connections in the visual cortex, and point to the existence of fine-scale subnetworks dedicated to processing related sensory information.     

Lateral habenula as a source of negative reward signals in dopamine neurons

Midbrain dopamine neurons are key components of the brain's reward system, which is thought to guide reward-seeking behaviours. Although recent studies have shown how dopamine neurons respond to rewards and sensory stimuli predicting reward, it is unclear which parts of the brain provide dopamine neurons with signals necessary for these actions. Here we show that the primate lateral habenula, part of the structure called the epithalamus, is a major candidate for a source of negative reward-related signals in dopamine neurons. We recorded the activity of habenula neurons and dopamine neurons while rhesus monkeys were performing a visually guided saccade task with positionally biased reward outcomes. Many habenula neurons were excited by a no-reward-predicting target and inhibited by a reward-predicting target. In contrast, dopamine neurons were excited and inhibited by reward-predicting and no-reward-predicting targets, respectively. Each time the rewarded and unrewarded positions were reversed, both habenula and dopamine neurons reversed their responses as the bias in saccade latency reversed. In unrewarded trials, the excitation of habenula neurons started earlier than the inhibition of dopamine neurons. Furthermore, weak electrical stimulation of the lateral habenula elicited strong inhibitions in dopamine neurons. These results suggest that the inhibitory input from the lateral habenula plays an important role in determining the reward-related activity of dopamine neurons.     

Cortical remodelling induced by activity of ventral tegmental dopamine neurons.

Representations of sensory stimuli in the cerebral cortex can undergo progressive remodelling according to the behavioural importance of the stimuli. The cortex receives widespread projections from dopamine neurons in the ventral tegmental area (VTA), which are activated by new stimuli or unpredicted rewards, and are believed to provide a reinforcement signal for such learning-related cortical reorganization. In the primary auditory cortex (AI) dopamine release has been observed during auditory learning that remodels the sound-frequency representations. Furthermore, dopamine modulates long-term potentiation, a putative cellular mechanism underlying plasticity. Here we show that stimulating the VTA together with an auditory stimulus of a particular tone increases the cortical area and selectivity of the neural responses to that sound stimulus in AI. Conversely, the AI representations of nearby sound frequencies are selectively decreased. Strong, sharply tuned responses to the paired tones also emerge in a second cortical area, whereas the same stimuli evoke only poor or non-selective responses in this second cortical field in naive animals. In addition, we found that strong long-range coherence of neuronal discharge emerges between AI and this secondary auditory cortical area.     

Small modulation of ongoing cortical dynamics by sensory input during natural vision

During vision, it is believed that neural activity in the primary visual cortex is predominantly driven by sensory input from the environment. However, visual cortical neurons respond to repeated presentations of the same stimulus with a high degree of variability. Although this variability has been considered to be noise owing to random spontaneous activity within the cortex, recent studies show that spontaneous activity has a highly coherent spatio-temporal structure. This raises the possibility that the pattern of this spontaneous activity may shape neural responses during natural viewing conditions to a larger extent than previously thought. Here, we examine the relationship between spontaneous activity and the response of primary visual cortical neurons to dynamic natural-scene and random-noise film images in awake, freely viewing ferrets from the time of eye opening to maturity. The correspondence between evoked neural activity and the structure of the input signal was weak in young animals, but systematically improved with age. This improvement was linked to a shift in the dynamics of spontaneous activity. At all ages including the mature animal, correlations in spontaneous neural firing were only slightly modified by visual stimulation, irrespective of the sensory input. These results suggest that in both the developing and mature visual cortex, sensory evoked neural activity represents the modulation and triggering of ongoing circuit dynamics by input signals, rather than directly reflecting the structure of the input signal itself.     

Analgesia and hyperalgesia from GABA-mediated modulation of the cerebral cortex

It is known that pain perception can be altered by mood, attention and cognition, or by direct stimulation of the cerebral cortex, but we know little of the neural mechanisms underlying the cortical modulation of pain. One of the few cortical areas consistently activated by painful stimuli is the rostral agranular insular cortex (RAIC) where, as in other parts of the cortex, the neurotransmitter gamma-aminobutyric acid (GABA) robustly inhibits neuronal activity. Here we show that changes in GABA neurotransmission in the RAIC can raise or lower the pain threshold--producing analgesia or hyperalgesia, respectively--in freely moving rats. Locally increasing GABA, by using an enzyme inhibitor or gene transfer mediated by a viral vector, produces lasting analgesia by enhancing the descending inhibition of spinal nociceptive neurons. Selectively activating GABA(B)-receptor-bearing RAIC neurons produces hyperalgesia through projections to the amygdala, an area involved in pain and fear. Whereas most studies focus on the role of the cerebral cortex as the end point of nociceptive processing, we suggest that cerebral cortex activity can change the set-point of pain threshold in a top-down manner.     

A subset of dopamine neurons signals reward for odour memory in Drosophila

Animals approach stimuli that predict a pleasant outcome. After the paired presentation of an odour and a reward, Drosophila melanogaster can develop a conditioned approach towards that odour. Despite recent advances in understanding the neural circuits for associative memory and appetitive motivation, the cellular mechanisms for reward processing in the fly brain are unknown. Here we show that a group of dopamine neurons in the protocerebral anterior medial (PAM) cluster signals sugar reward by transient activation and inactivation of target neurons in intact behaving flies. These dopamine neurons are selectively required for the reinforcing property of, but not a reflexive response to, the sugar stimulus. In vivo calcium imaging revealed that these neurons are activated by sugar ingestion and the activation is increased on starvation. The output sites of the PAM neurons are mainly localized to the medial lobes of the mushroom bodies (MBs), where appetitive olfactory associative memory is formed. We therefore propose that the PAM cluster neurons endow a positive predictive value to the odour in the MBs. Dopamine in insects is known to mediate aversive reinforcement signals. Our results highlight the cellular specificity underlying the various roles of dopamine and the importance of spatially segregated local circuits within the MBs.     

Neurophysiological investigation of the basis of the fMRI signal.

Functional magnetic resonance imaging (fMRI) is widely used to study the operational organization of the human brain, but the exact relationship between the measured fMRI signal and the underlying neural activity is unclear. Here we present simultaneous intracortical recordings of neural signals and fMRI responses. We compared local field potentials (LFPs), single- and multi-unit spiking activity with highly spatio-temporally resolved blood-oxygen-level-dependent (BOLD) fMRI responses from the visual cortex of monkeys. The largest magnitude changes were observed in LFPs, which at recording sites characterized by transient responses were the only signal that significantly correlated with the haemodynamic response. Linear systems analysis on a trial-by-trial basis showed that the impulse response of the neurovascular system is both animal- and site-specific, and that LFPs yield a better estimate of BOLD responses than the multi-unit responses. These findings suggest that the BOLD contrast mechanism reflects the input and intracortical processing of a given area rather than its spiking output.     

Gaze direction controls response gain in primary visual-cortex neurons

To localize objects in space, the brain needs to combine information about the position of the stimulus on the retinae with information about the location of the eyes in their orbits. Interaction between these two types of information occurs in several cortical areas, but the role of the primary visual cortex (area V1) in this process has remained unclear. Here we show that, for half the cells recorded in area V1 of behaving monkeys, the classically described visual responses are strongly modulated by gaze direction. Specifically, we find that selectivity for horizontal retinal disparity-the difference in the position of a stimulus on each retina which relates to relative object distance-and for stimulus orientation may be present at a given gaze direction, but be absent or poorly expressed at another direction. Shifts in preferred disparity also occurred in several neurons. These neural changes were most often present at the beginning of the visual response, suggesting a feedforward gain control by eye position signals. Cortical neural processes for encoding information about the three-dimensional position of a stimulus in space therefore start as early as area V1.     

Motor learning in a recurrent network model based on the vestibulo-ocular reflex.

Most models of neural networks have assumed that neurons process information on a timescale of milliseconds and that the long-term modification of synaptic strengths underlies learning and memory. But neurons also have cellular mechanisms that operate on a timescale of tens or hundreds of milliseconds, such as a gradual rise in firing rate in response to injection of constant current or a rapid rise followed by a slower adaptation. These dynamic properties of neuronal responses are mediated by ion channels that are subject to modulation. We demonstrate here how a neural network with recurrent feedback connections can convert long-term modulation of neural responses that occur over these intermediate timescales into changes in the amplitude of the steady output from the system. This general principle may be relevant to many feedback systems in the brain. Here it is applied to the vestibulo-ocular reflex, whose amplitude is subject to long-term adaptive modification by visual inputs. The model reconciles apparently contradictory data on the neural locus of the cellular mechanisms that mediate this simple form of learning and memory.     

Guarding the gateway to cortex with attention in visual thalamus

The massive visual input from the eye to the brain requires selective processing of some visual information at the expense of other information, a process referred to as visual attention. Increases in the responses of visual neurons with attention have been extensively studied along the visual processing streams in monkey cerebral cortex, from primary visual areas to parietal and frontal cortex. Here we show, by recording neurons in attending macaque monkeys (Macaca mulatta), that attention modulates visual signals before they even reach cortex by increasing responses of both magnocellular and parvocellular neurons in the first relay between retina and cortex, the lateral geniculate nucleus (LGN). At the same time, attention decreases neuronal responses in the adjacent thalamic reticular nucleus (TRN). Crick argued for such modulation of the LGN by observing that it is inhibited by the TRN, and suggested that "if the thalamus is the gateway to the cortex, the reticular complex might be described as the guardian of the gateway", a reciprocal relationship we now show to be more than just hypothesis. The reciprocal modulation in LGN and TRN appears only during the initial visual response, but the modulation of LGN reappears later in the response, suggesting separate early and late sources of attentional modulation in LGN.     

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.     

Neuronal correlates of parametric working memory in the prefrontal cortex.

Humans and monkeys have similar abilities to discriminate the difference in frequency between two mechanical vibrations applied sequentially to the fingertips. A key component of this sensory task is that the second stimulus is compared with the trace left by the first (base) stimulus, which must involve working memory. Where and how is this trace held in the brain? This question was investigated by recording from single neurons in the prefrontal cortex of monkeys while they performed the somatosensory discrimination task. Here we describe neurons in the inferior convexity of the prefrontal cortex whose discharge rates varied, during the delay period between the two stimuli, as a monotonic function of the base stimulus frequency. We describe this as 'monotonic stimulus encoding', and we suggest that the result may generalize: monotonic stimulus encoding may be the basic representation of one-dimensional sensory stimulus quantities in working memory. Thus we predict that other behavioural tasks that require ordinal comparisons between scalar analogue stimuli would give rise to monotonic responses similar to those reported here.