Neuronal correlate of visual associative long-term memory in the primate temporal cortex

In human long-term memory, ideas and concepts become associated in the learning process. No neuronal correlate for this cognitive function has so far been described, except that memory traces are thought to be localized in the cerebral cortex; the temporal lobe has been assigned as the site for visual experience because electric stimulation of this area results in imagery recall and lesions produce deficits in visual recognition of objects. We previously reported that in the anterior ventral temporal cortex of monkeys, individual neurons have a sustained activity that is highly selective for a few of the 100 coloured fractal patterns used in a visual working-memory task. Here I report the development of this selectivity through repeated trials involving the working memory. The few patterns for which a neuron was conjointly selective were frequently related to each other through stimulus-stimulus association imposed during training. The results indicate that the selectivity acquired by these cells represents a neuronal correlate of the associative long-term memory of pictures.     

Neural organization for the long-term memory of paired associates

Most of our long-term memories of episodes or objects are organized so that we can retrieve them by association. Clinical neuropsychologists assess human memory by the paired-associate learning test, in which a series of paired words or figures is presented and the subject is then asked to retrieve the other pair member associated with each cue. Patients with lesions of the temporal lobe show marked impairment in this test. In our study, we trained monkeys in a pair-association task using a set of computer-generated paired patterns. We found two types of task-related neurons in the anterior temporal cortex. One type selectively responded to both pictures of the paired associates. The other type, which had the strongest response to one picture during the cue presentation, exhibited increasing activity during the delay period when the associate of that picture was used as a cue. These results provide new evidence that single neurons acquire selectivity for visual patterns through associative learning. They also indicate neural mechanisms for storage and retrieval in the long-term memory of paired associates.     

Visual categorization shapes feature selectivity in the primate temporal cortex.

The way that we perceive and interact with objects depends on our previous experience with them. For example, a bird expert is more likely to recognize a bird as a sparrow, a sandpiper or a cockatiel than a non-expert. Neurons in the inferior temporal cortex have been shown to be important in the representation of visual objects; however, it is unknown which object features are represented and how these representations are affected by categorization training. Here we show that feature selectivity in the macaque inferior temporal cortex is shaped by categorization of objects on the basis of their visual features. Specifically, we recorded from single neurons while monkeys performed a categorization task with two sets of parametric stimuli. Each stimulus set consisted of four varying features, but only two of the four were important for the categorization task (diagnostic features). We found enhanced neuronal representation of the diagnostic features relative to the non-diagnostic ones. These findings demonstrate that stimulus features important for categorization are instantiated in the activity of single units (neurons) in the primate inferior temporal cortex.     

The neuronal representation of pitch in primate auditory cortex

Pitch perception is critical for identifying and segregating auditory objects, especially in the context of music and speech. The perception of pitch is not unique to humans and has been experimentally demonstrated in several animal species. Pitch is the subjective attribute of a sound's fundamental frequency (f(0)) that is determined by both the temporal regularity and average repetition rate of its acoustic waveform. Spectrally dissimilar sounds can have the same pitch if they share a common f(0). Even when the acoustic energy at f(0) is removed ('missing fundamental') the same pitch is still perceived. Despite its importance for hearing, how pitch is represented in the cerebral cortex is unknown. Here we show the existence of neurons in the auditory cortex of marmoset monkeys that respond to both pure tones and missing fundamental harmonic complex sounds with the same f(0), providing a neural correlate for pitch constancy. These pitch-selective neurons are located in a restricted low-frequency cortical region near the anterolateral border of the primary auditory cortex, and is consistent with the location of a pitch-selective area identified in recent imaging studies in humans.     

Microstimulation of inferotemporal cortex influences face categorization

The inferior temporal cortex (IT) of primates is thought to be the final visual area in the ventral stream of cortical areas responsible for object recognition. Consistent with this hypothesis, single IT neurons respond selectively to highly complex visual stimuli such as faces. However, a direct causal link between the activity of face-selective neurons and face perception has not been demonstrated. In the present study of macaque monkeys, we artificially activated small clusters of IT neurons by means of electrical microstimulation while the monkeys performed a categorization task, judging whether noisy visual images belonged to 'face' or 'non-face' categories. Here we show that microstimulation of face-selective sites, but not other sites, strongly biased the monkeys' decisions towards the face category. The magnitude of the effect depended upon the degree of face selectivity of the stimulation site, the size of the stimulated cluster of face-selective neurons, and the exact timing of microstimulation. Our results establish a causal relationship between the activity of face-selective neurons and face perception.     

Top-down signal from prefrontal cortex in executive control of memory retrieval.

Knowledge or experience is voluntarily recalled from memory by reactivation of the neural representations in the cerebral association cortex. In inferior temporal cortex, which serves as the storehouse of visual long-term memory, activation of mnemonic engrams through electric stimulation results in imagery recall in humans, and neurons can be dynamically activated by the necessity for memory recall in monkeys. Neuropsychological studies and previous split-brain experiments predicted that prefrontal cortex exerts executive control upon inferior temporal cortex in memory retrieval; however, no neuronal correlate of this process has ever been detected. Here we show evidence of the top-down signal from prefrontal cortex. In the absence of bottom-up visual inputs, single inferior temporal neurons were activated by the top-down signal, which conveyed information on semantic categorization imposed by visual stimulus-stimulus association. Behavioural performance was severely impaired with loss of the top-down signal. Control experiments confirmed that the signal was transmitted not through a subcortical but through a fronto-temporal cortical pathway. Thus, feedback projections from prefrontal cortex to the posterior association cortex appear to serve the executive control of voluntary recall.     

Entrained rhythmic activities of neuronal ensembles as perceptual memory of time interval

The ability to process temporal information is fundamental to sensory perception, cognitive processing and motor behaviour of all living organisms, from amoebae to humans. Neural circuit mechanisms based on neuronal and synaptic properties have been shown to process temporal information over the range of tens of microseconds to hundreds of milliseconds. How neural circuits process temporal information in the range of seconds to minutes is much less understood. Studies of working memory in monkeys and rats have shown that neurons in the prefrontal cortex, the parietal cortex and the thalamus exhibit ramping activities that linearly correlate with the lapse of time until the end of a specific time interval of several seconds that the animal is trained to memorize. Many organisms can also memorize the time interval of rhythmic sensory stimuli in the timescale of seconds and can coordinate motor behaviour accordingly, for example, by keeping the rhythm after exposure to the beat of music. Here we report a form of rhythmic activity among specific neuronal ensembles in the zebrafish optic tectum, which retains the memory of the time interval (in the order of seconds) of repetitive sensory stimuli for a duration of up to approximately 20 s. After repetitive visual conditioning stimulation (CS) of zebrafish larvae, we observed rhythmic post-CS activities among specific tectal neuronal ensembles, with a regular interval that closely matched the CS. Visuomotor behaviour of the zebrafish larvae also showed regular post-CS repetitions at the entrained time interval that correlated with rhythmic neuronal ensemble activities in the tectum. Thus, rhythmic activities among specific neuronal ensembles may act as an adjustable 'metronome' for time intervals in the order of seconds, and serve as a mechanism for the short-term perceptual memory of rhythmic sensory experience.     

Invariant visual representation by single neurons in the human brain

It takes a fraction of a second to recognize a person or an object even when seen under strikingly different conditions. How such a robust, high-level representation is achieved by neurons in the human brain is still unclear. In monkeys, neurons in the upper stages of the ventral visual pathway respond to complex images such as faces and objects and show some degree of invariance to metric properties such as the stimulus size, position and viewing angle. We have previously shown that neurons in the human medial temporal lobe (MTL) fire selectively to images of faces, animals, objects or scenes. Here we report on a remarkable subset of MTL neurons that are selectively activated by strikingly different pictures of given individuals, landmarks or objects and in some cases even by letter strings with their names. These results suggest an invariant, sparse and explicit code, which might be important in the transformation of complex visual percepts into long-term and more abstract memories.     

Cortical microstimulation influences perceptual judgements of motion direction

Neurons in the visual cortex respond selectively to perceptually salient features of the visual scene, such as the direction and speed of moving objects, the orientation of local contours, or the colour or relative depth of a visual pattern. It is commonly assumed that the brain constructs its percept of the visual scene from information encoded in the selective responses of such neurons. We have now tested this hypothesis directly by measuring the effect on psychophysical performance of modifying the firing rates of physiologically characterized neurons. We required rhesus monkeys to report the direction of motion in a visual display while we electrically stimulated clusters of directionally selective neurons in the middle temporal visual area (MT, or V5), an extrastriate area that plays a prominent role in the analysis of visual motion information. Microstimulation biased the animals' judgements towards the direction of motion encoded by the stimulated neurons. This result indicates that physiological properties measured at the neuronal level can be causally related to a specific aspect of perceptual performance.     

Cortical activity reductions during repetition priming can result from rapid response learning

Recent observation of objects speeds up their subsequent identification and classification. This common form of learning, known as repetition priming, can operate in the absence of explicit memory for earlier experiences, and functional neuroimaging has shown that object classification improved in this way is accompanied by 'neural priming' (reduced neural activity) in prefrontal, fusiform and other cortical regions. These observations have led to suggestions that cortical representations of items undergo 'tuning', whereby neurons encoding irrelevant information respond less as a given object is observed repeatedly, thereby facilitating future availability of pertinent object knowledge. Here we provide experimental support for an alternative hypothesis, in which reduced cortical activity occurs because subjects rapidly learn their previous responses. After a primed object classification (such as 'bigger than a shoebox'), cue reversal ('smaller than a shoebox') greatly slowed performance and completely eliminated neural priming in fusiform cortex, which suggests that these cortical item representations were no more available for primed objects than they were for new objects. In contrast, prefrontal cortex activity tracked behavioural priming and predicted the degree to which cue reversal would slow down object classification--highlighting the role of the prefrontal cortex in executive control.     

Global and fine information coded by single neurons in the temporal visual cortex.

When we see a person's face, we can easily recognize their species, individual identity and emotional state. How does the brain represent such complex information? A substantial number of neurons in the macaque temporal cortex respond to faces. However, the neuronal mechanisms underlying the processing of complex information are not yet clear. Here we recorded the activity of single neurons in the temporal cortex of macaque monkeys while presenting visual stimuli consisting of geometric shapes, and monkey and human faces with various expressions. Information theory was used to investigate how well the neuronal responses could categorize the stimuli. We found that single neurons conveyed two different scales of facial information in their firing patterns, starting at different latencies. Global information, categorizing stimuli as monkey faces, human faces or shapes, was conveyed in the earliest part of the responses. Fine information about identity or expression was conveyed later, beginning on average 51 ms after global information. We speculate that global information could be used as a 'header' to prepare destination areas for receiving more detailed information.     

Sustained firing in auditory cortex evoked by preferred stimuli

It has been well documented that neurons in the auditory cortex of anaesthetized animals generally display transient responses to acoustic stimulation, and typically respond to a brief stimulus with one or fewer action potentials. The number of action potentials evoked by each stimulus usually does not increase with increasing stimulus duration. Such observations have long puzzled researchers across disciplines and raised serious questions regarding the role of the auditory cortex in encoding ongoing acoustic signals. Contrary to these long-held views, here we show that single neurons in both primary (area A1) and lateral belt areas of the auditory cortex of awake marmoset monkeys (Callithrix jacchus) are capable of firing in a sustained manner over a prolonged period of time, especially when they are driven by their preferred stimuli. In contrast, responses become more transient or phasic when auditory cortex neurons respond to non-preferred stimuli. These findings suggest that when the auditory cortex is stimulated by a sound, a particular population of neurons fire maximally throughout the duration of the sound. Responses of other, less optimally driven neurons fade away quickly after stimulus onset. This results in a selective representation of the sound across both neuronal population and time.     

A general mechanism for perceptual decision-making in the human brain

Findings from single-cell recording studies suggest that a comparison of the outputs of different pools of selectively tuned lower-level sensory neurons may be a general mechanism by which higher-level brain regions compute perceptual decisions. For example, when monkeys must decide whether a noisy field of dots is moving upward or downward, a decision can be formed by computing the difference in responses between lower-level neurons sensitive to upward motion and those sensitive to downward motion. Here we use functional magnetic resonance imaging and a categorization task in which subjects decide whether an image presented is a face or a house to test whether a similar mechanism is also at work for more complex decisions in the human brain and, if so, where in the brain this computation might be performed. Activity within the left dorsolateral prefrontal cortex is greater during easy decisions than during difficult decisions, covaries with the difference signal between face- and house-selective regions in the ventral temporal cortex, and predicts behavioural performance in the categorization task. These findings show that even for complex object categories, the comparison of the outputs of different pools of selectively tuned neurons could be a general mechanism by which the human brain computes perceptual decisions.     

An area for vergence eye movement in primate frontal cortex

To view objects at different distances, humans rely on vergence eye movements to appropriately converge or diverge the eyes and on ocular accommodation to focus the object. Despite the importance of these coordinated eye movements (the 'near response') very little is known about the role of the cerebral cortex in their control. As near-response neurons exist within the nucleus reticularis tegmenti pontis, which receives input from the frontal eye field region of frontal cortex, and this cortical region is known to be involved in saccadic and smooth-pursuit eye movements, we propose that a nearby region might play a role in vergence and ocular accommodation. Here we provide evidence from rhesus monkeys that a region of frontal cortex located immediately anterior to the saccade-related frontal eye field region is involved in vergence and ocular accommodation, and in the sensorimotor transformations required for these eye movements. We conclude that the macaque frontal cortex is involved in the control of all voluntary eye movements, and suggest that the definition of the frontal eye fields should be expanded to include this region.     

Performance monitoring by the supplementary eye field

Intelligent behaviour requires self-control based on the consequences of actions. The countermanding task is designed to study self-control; it requires subjects to withhold planned movements in response to an imperative stop signal, which they can do with varying success. In humans, the medial frontal cortex has been implicated in the supervisory control of action. In monkeys, the supplementary eye field in the dorsomedial frontal cortex is involved in producing eye movements, but its precise function has not been clarified. To investigate the role of the supplementary eye field in the control of eye movements, we recorded neural activity in macaque monkeys trained to perform an eye movement countermanding task. Distinct groups of neurons were active after errors, after successful withholding of a partially prepared movement, or in association with reinforcement. These three forms of activation could not be explained by sensory or motor factors. Our results lead us to put forward the hypothesis that the supplementary eye field contributes to monitoring the context and consequences of eye movements.     

Neuronal correlates of parametric working memory in the prefrontal cortex.

Humans and monkeys have similar abilities to discriminate the difference in frequency between two mechanical vibrations applied sequentially to the fingertips. A key component of this sensory task is that the second stimulus is compared with the trace left by the first (base) stimulus, which must involve working memory. Where and how is this trace held in the brain? This question was investigated by recording from single neurons in the prefrontal cortex of monkeys while they performed the somatosensory discrimination task. Here we describe neurons in the inferior convexity of the prefrontal cortex whose discharge rates varied, during the delay period between the two stimuli, as a monotonic function of the base stimulus frequency. We describe this as 'monotonic stimulus encoding', and we suggest that the result may generalize: monotonic stimulus encoding may be the basic representation of one-dimensional sensory stimulus quantities in working memory. Thus we predict that other behavioural tasks that require ordinal comparisons between scalar analogue stimuli would give rise to monotonic responses similar to those reported here.     

A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons

Neurons in area 7a of the posterior parietal cortex of monkeys respond to both the retinal location of a visual stimulus and the position of the eyes and by combining these signals represent the spatial location of external objects. A neural network model, programmed using back-propagation learning, can decode this spatial information from area 7a neurons and accounts for their observed response properties.     

Spatial awareness is a function of the temporal not the posterior parietal lobe.

Our current understanding of spatial behaviour and parietal lobe function is largely based on the belief that spatial neglect in humans (a lack of awareness of space on the side of the body contralateral to a brain injury) is typically associated with lesions of the posterior parietal lobe. However, in monkeys, this disorder is observed after lesions of the superior temporal cortex, a puzzling discrepancy between the species. Here we show that, contrary to the widely accepted view, the superior temporal cortex is the neural substrate of spatial neglect in humans, as it is in monkeys. Unlike the monkey brain, spatial awareness in humans is a function largely confined to the right superior temporal cortex, a location topographically reminiscent of that for language on the left. Hence, the decisive phylogenetic transition from monkey to human brain seems to be a restriction of a formerly bilateral function to the right side, rather than a shift from the temporal to the parietal lobe. One may speculate that this lateralization of spatial awareness parallels the emergence of an elaborate representation for language on the left side.     

Turning on and off recurrent balanced cortical activity

The vast majority of synaptic connections onto neurons in the cerebral cortex arise from other cortical neurons, both excitatory and inhibitory, forming local and distant 'recurrent' networks. Although this is a basic theme of cortical organization, its study has been limited largely to theoretical investigations, which predict that local recurrent networks show a proportionality or balance between recurrent excitation and inhibition, allowing the generation of stable periods of activity. This recurrent activity might underlie such diverse operations as short-term memory, the modulation of neuronal excitability with attention, and the generation of spontaneous activity during sleep. Here we show that local cortical circuits do indeed operate through a proportional balance of excitation and inhibition generated through local recurrent connections, and that the operation of such circuits can generate self-sustaining activity that can be turned on and off by synaptic inputs. These results confirm the long-hypothesized role of recurrent activity as a basic operation of the cerebral cortex.     

Robust habit learning in the absence of awareness and independent of the medial temporal lobe

Habit memory is thought to involve slowly acquired associations between stimuli and responses and to depend on the basal ganglia. Habit memory has been well studied in experimental animals but is poorly understood in humans because of their strong tendency to acquire information as conscious (declarative) knowledge. Here we show that humans have a robust capacity for gradual trial-and-error learning that operates outside awareness for what is learned and independently of the medial temporal lobe. We tested two patients with large medial temporal lobe lesions and no capacity for declarative memory. Both patients gradually acquired a standard eight-pair object discrimination task over many weeks but at the start of each session could not describe the task, the instructions or the objects. The acquired knowledge was rigidly organized, and performance collapsed when the task format was altered.