On-line, voluntary control of human temporal lobe neurons

Daily life continually confronts us with an exuberance of external, sensory stimuli competing with a rich stream of internal deliberations, plans and ruminations. The brain must select one or more of these for further processing. How this competition is resolved across multiple sensory and cognitive regions is not known; nor is it clear how internal thoughts and attention regulate this competition. Recording from single neurons in patients implanted with intracranial electrodes for clinical reasons, here we demonstrate that humans can regulate the activity of their neurons in the medial temporal lobe (MTL) to alter the outcome of the contest between external images and their internal representation. Subjects looked at a hybrid superposition of two images representing familiar individuals, landmarks, objects or animals and had to enhance one image at the expense of the other, competing one. Simultaneously, the spiking activity of their MTL neurons in different subregions and hemispheres was decoded in real time to control the content of the hybrid. Subjects reliably regulated, often on the first trial, the firing rate of their neurons, increasing the rate of some while simultaneously decreasing the rate of others. They did so by focusing onto one image, which gradually became clearer on the computer screen in front of their eyes, and thereby overriding sensory input. On the basis of the firing of these MTL neurons, the dynamics of the competition between visual images in the subject's mind was visualized on an external display.     

Neuronal correlate of pictorial short-term memory in the primate temporal cortex

It has been proposed that visual-memory traces are located in the temporal lobes of the cerebral cortex, as electric stimulation of this area in humans results in recall of imagery. Lesions in this area also affect recognition of an object after a delay in both humans and monkeys, indicating a role in short-term memory of images. Single-unit recordings from the temporal cortex have shown that some neurons continue to fire when one of two or four colours are to be remembered temporarily. But neuronal responses selective to specific complex objects, including hands and faces, cease soon after the offset of stimulus presentation. These results led to the question of whether any of these neurons could serve the memory of complex objects. We report here a group of shape-selective neurons in an anterior ventral part of the temporal cortex of monkeys that exhibited sustained activity during the delay period of a visual short-term memory task. The activity was highly selective for the pictorial information to be memorized and was independent of the physical attributes such as size, orientation, colour or position of the object. These observations show that the delay activity represents the short-term memory of the categorized percept of a picture.     

Microstimulation of inferotemporal cortex influences face categorization

The inferior temporal cortex (IT) of primates is thought to be the final visual area in the ventral stream of cortical areas responsible for object recognition. Consistent with this hypothesis, single IT neurons respond selectively to highly complex visual stimuli such as faces. However, a direct causal link between the activity of face-selective neurons and face perception has not been demonstrated. In the present study of macaque monkeys, we artificially activated small clusters of IT neurons by means of electrical microstimulation while the monkeys performed a categorization task, judging whether noisy visual images belonged to 'face' or 'non-face' categories. Here we show that microstimulation of face-selective sites, but not other sites, strongly biased the monkeys' decisions towards the face category. The magnitude of the effect depended upon the degree of face selectivity of the stimulation site, the size of the stimulated cluster of face-selective neurons, and the exact timing of microstimulation. Our results establish a causal relationship between the activity of face-selective neurons and face perception.     

Neural encoding of individual words and faces by the human hippocampus and amygdala

Patients with lesions in the medial temporal lobe (MTL) of the brain, which includes the hippocampus, amygdala and parahippocampal gyrus, are severely impaired in their ability to remember and recognize words or faces which they saw only a short time ago. These lesions also prevent the effect of word repetition on cortical event-related potentials that are associated with these tasks. We have been able to study the response of individual neurons in the human medial temporal lobe to such delayed recognition tasks in epileptic patients undergoing neurosurgery. We found that some MTL neurons preferentially fired on sight of one particular word from a set of ten words used in a memory task, and others fired in response to one particular face. This stimulus-specific firing was maximal during the time that the neocortical event potentials are most sensitive to stimulus repetition, suggesting that the MTL contributes specific information to the cortex during the retrieval of recent memories.     

Global and fine information coded by single neurons in the temporal visual cortex.

When we see a person's face, we can easily recognize their species, individual identity and emotional state. How does the brain represent such complex information? A substantial number of neurons in the macaque temporal cortex respond to faces. However, the neuronal mechanisms underlying the processing of complex information are not yet clear. Here we recorded the activity of single neurons in the temporal cortex of macaque monkeys while presenting visual stimuli consisting of geometric shapes, and monkey and human faces with various expressions. Information theory was used to investigate how well the neuronal responses could categorize the stimuli. We found that single neurons conveyed two different scales of facial information in their firing patterns, starting at different latencies. Global information, categorizing stimuli as monkey faces, human faces or shapes, was conveyed in the earliest part of the responses. Fine information about identity or expression was conveyed later, beginning on average 51 ms after global information. We speculate that global information could be used as a 'header' to prepare destination areas for receiving more detailed information.     

Norm-based face encoding by single neurons in the monkey inferotemporal cortex

The rich and immediate perception of a familiar face, including its identity, expression and even intent, is one of the most impressive shared faculties of human and non-human primate brains. Many visually responsive neurons in the inferotemporal cortex of macaque monkeys respond selectively to faces, sometimes to only one or a few individuals, while showing little sensitivity to scale and other details of the retinal image. Here we show that face-responsive neurons in the macaque monkey anterior inferotemporal cortex are tuned to a fundamental dimension of face perception. Using a norm-based caricaturization framework previously developed for human psychophysics, we varied the identity information present in photo-realistic human faces, and found that neurons of the anterior inferotemporal cortex were most often tuned around the average, identity-ambiguous face. These observations are consistent with face-selective responses in this area being shaped by a figural comparison, reflecting structural differences between an incoming face and an internal reference or norm. As such, these findings link the tuning of neurons in the inferotemporal cortex to psychological models of face identity perception.     

Cortical microstimulation influences perceptual judgements of motion direction

Neurons in the visual cortex respond selectively to perceptually salient features of the visual scene, such as the direction and speed of moving objects, the orientation of local contours, or the colour or relative depth of a visual pattern. It is commonly assumed that the brain constructs its percept of the visual scene from information encoded in the selective responses of such neurons. We have now tested this hypothesis directly by measuring the effect on psychophysical performance of modifying the firing rates of physiologically characterized neurons. We required rhesus monkeys to report the direction of motion in a visual display while we electrically stimulated clusters of directionally selective neurons in the middle temporal visual area (MT, or V5), an extrastriate area that plays a prominent role in the analysis of visual motion information. Microstimulation biased the animals' judgements towards the direction of motion encoded by the stimulated neurons. This result indicates that physiological properties measured at the neuronal level can be causally related to a specific aspect of perceptual performance.     

fMRI evidence for objects as the units of attentional selection.

Contrasting theories of visual attention emphasize selection by spatial location, visual features (such as motion or colour) or whole objects. Here we used functional magnetic resonance imaging (fMRI) to test key predictions of the object-based theory, which proposes that pre-attentive mechanisms segment the visual array into discrete objects, groups, or surfaces, which serve as targets for visual attention. Subjects viewed stimuli consisting of a face transparently superimposed on a house, with one moving and the other stationary. In different conditions, subjects attended to the face, the house or the motion. The magnetic resonance signal from each subject's fusiform face area, parahippocampal place area and area MT/MST provided a measure of the processing of faces, houses and visual motion, respectively. Although all three attributes occupied the same location, attending to one attribute of an object (such as the motion of a moving face) enhanced the neural representation not only of that attribute but also of the other attribute of the same object (for example, the face), compared with attributes of the other object (for example, the house). These results cannot be explained by models in which attention selects locations or features, and provide physiological evidence that whole objects are selected even when only one visual attribute is relevant.     

A general mechanism for perceptual decision-making in the human brain

Findings from single-cell recording studies suggest that a comparison of the outputs of different pools of selectively tuned lower-level sensory neurons may be a general mechanism by which higher-level brain regions compute perceptual decisions. For example, when monkeys must decide whether a noisy field of dots is moving upward or downward, a decision can be formed by computing the difference in responses between lower-level neurons sensitive to upward motion and those sensitive to downward motion. Here we use functional magnetic resonance imaging and a categorization task in which subjects decide whether an image presented is a face or a house to test whether a similar mechanism is also at work for more complex decisions in the human brain and, if so, where in the brain this computation might be performed. Activity within the left dorsolateral prefrontal cortex is greater during easy decisions than during difficult decisions, covaries with the difference signal between face- and house-selective regions in the ventral temporal cortex, and predicts behavioural performance in the categorization task. These findings show that even for complex object categories, the comparison of the outputs of different pools of selectively tuned neurons could be a general mechanism by which the human brain computes perceptual decisions.     

Dissociation between mental imagery and object recognition in a brain-damaged patient.

Visual imagery is the creation of mental representations that share many features with veridical visual percepts. Studies of normal and brain-damaged people reinforce the view that visual imagery and visual perception are mediated by a common neural substrate and activate the same representations. Thus, brain-damaged patients with intact vision who have an impairment in perception should have impaired visual imagery. Here we present evidence to the contrary from a patient with severely impaired object recognition (visual object agnosia) but with normal mental imagery. He draws objects in considerable detail from memory and uses information derived from mental images in a variety of tasks. In contrast, he cannot identify visually presented objects, even those he has drawn himself. He has normal visual acuity and intact perception of equally complex material in other domains. We conclude that rich internal representations can be activated to support visual imagery even when they cannot support visually mediated perception of objects.     

Neuronal synchrony does not correlate with motion coherence in cortical area MT

Natural visual scenes are cluttered with multiple objects whose individual features must somehow be selectively linked (or 'bound') if perception is to coincide with reality. Recent neurophysiological evidence supports a 'binding-by-synchrony' hypothesis: neurons excited by features of the same object fire synchronously, while neurons excited by features of different objects do not. Moving plaid patterns offer a straightforward means to test this idea. By appropriate manipulations of apparent transparency, the component gratings of a plaid pattern can be seen as parts of a single coherently moving surface or as two non-coherently moving surfaces. We examined directional tuning and synchrony of area-MT neurons in awake, fixating primates in response to perceptually coherent and non-coherent plaid patterns. Here we show that directional tuning correlated highly with perceptual coherence, which is consistent with an earlier study. Although we found stimulus-dependent synchrony, coherent plaids elicited significantly less synchrony than did non-coherent plaids. Our data therefore do not support the binding-by-synchrony hypothesis as applied to this class of motion stimuli in area MT.     

视觉神经表征中底-顶和顶-底的双向激活——来自人与猴的实验证据

长期以来,建立在坚实生理学证据之上的底-顶加工说和特征检测理论在视觉研究中占主导地位。对于顶-底加工人们只能靠一般常识,即知识或经验通过激活记忆中的神经表征影响视觉过程。但是近年来,来自人和猴的研究为顶-底的加工提供了实验证据。 首先位于猴腹侧加工系统内的物体和面孔视觉记忆表征,提供了神经编码是怎样创立、组织和再激活的最佳实验证据。联想性编码是通过学习由一些具有特殊功能的神经元建立的,这些神经元具有将时间性关联刺激的表征联系起来的能力。其次,不仅来自视网膜的底-顶信号,而且来自前额叶的顶-底信号都能触发联想性编码的提取,既可以作为有意识回忆的神经基础,又是顶-底加工影响视觉过程的基础。脑损伤病人研究、具有高时间分辨率的人类功能性核磁共振成像(functional magnetic resonance imaging, fMRI)和猴fMRI研究以及猴细胞电生理分析相结合,将进一步加强人们对视觉脑机制的全面理解。     

A neurological dissociation between perceiving objects and grasping them

Studies of the visual capacity of neurological patients have provided evidence for a dissociation between the perceptual report of a visual stimulus and the ability to direct spatially accurate movements toward that stimulus. Some patients with damage to the parietal lobe, for example, are unable to reach accurately towards visual targets that they unequivocally report seeing. Conversely, some patients with extensive damage to primary visual cortex can make accurate pointing movements or saccades toward a stimulus presented in their 'blind' scotoma. But in investigations of visuomotor control in patients with visual disorders, little consideration has been given to complex acts such as manual prehension. Grasping a three-dimensional object requires knowledge not only of the object's spatial location, but also of its form, orientation and size. We have examined a patient with a profound disorder in the perception of such object qualities. Our quantitative analyses demonstrate strikingly accurate guidance of hand and finger movements directed at the very objects whose qualities she fails to perceive. These data suggest that the neural substrates for the visual perception of object qualities such as shape, orientation and size are distinct from those underlying the use of those qualities in the control of manual skills.     

基于缝雕刻的图像重定向算法

图像重定向算法利用人对图像内容的感知差异进行非均匀的图像缩放,以适应新的显示需求.本文提出了一种基于缝雕刻的图像重定向算法.它利用每条seam上像素的累加能量和最大能量构造混合能量模型,使算法既能够利用人眼对梯度信息的敏感性,又可以减少复杂纹理背景对seam提取的干扰.实验结果显示,相比其他几种算法,本文算法可以更好地保护图像中视觉关注对象的边缘轮廓结构,处理后的图像不仅视觉效果更好,与原图的相似性距离也更小.     

Guarding the gateway to cortex with attention in visual thalamus

The massive visual input from the eye to the brain requires selective processing of some visual information at the expense of other information, a process referred to as visual attention. Increases in the responses of visual neurons with attention have been extensively studied along the visual processing streams in monkey cerebral cortex, from primary visual areas to parietal and frontal cortex. Here we show, by recording neurons in attending macaque monkeys (Macaca mulatta), that attention modulates visual signals before they even reach cortex by increasing responses of both magnocellular and parvocellular neurons in the first relay between retina and cortex, the lateral geniculate nucleus (LGN). At the same time, attention decreases neuronal responses in the adjacent thalamic reticular nucleus (TRN). Crick argued for such modulation of the LGN by observing that it is inhibited by the TRN, and suggested that "if the thalamus is the gateway to the cortex, the reticular complex might be described as the guardian of the gateway", a reciprocal relationship we now show to be more than just hypothesis. The reciprocal modulation in LGN and TRN appears only during the initial visual response, but the modulation of LGN reappears later in the response, suggesting separate early and late sources of attentional modulation in LGN.     

Sheep don't forget a face.

The human brain has evolved specialized neural mechanisms for visual recognition of faces, which afford us a remarkable ability to discriminate between, remember and think about many hundreds of different individuals. Sheep also recognize and are attracted to individual sheep and humans by their faces, as they possess similar specialized neural systems in the temporal and frontal lobes for assisting in this important social task, including a greater involvement of the right brain hemisphere. Here we show that individual sheep can remember 50 other different sheep faces for over 2 years, and that the specialized neural circuits involved maintain selective encoding of individual sheep and human faces even after long periods of separation.     

Neural organization for the long-term memory of paired associates

Most of our long-term memories of episodes or objects are organized so that we can retrieve them by association. Clinical neuropsychologists assess human memory by the paired-associate learning test, in which a series of paired words or figures is presented and the subject is then asked to retrieve the other pair member associated with each cue. Patients with lesions of the temporal lobe show marked impairment in this test. In our study, we trained monkeys in a pair-association task using a set of computer-generated paired patterns. We found two types of task-related neurons in the anterior temporal cortex. One type selectively responded to both pictures of the paired associates. The other type, which had the strongest response to one picture during the cue presentation, exhibited increasing activity during the delay period when the associate of that picture was used as a cue. These results provide new evidence that single neurons acquire selectivity for visual patterns through associative learning. They also indicate neural mechanisms for storage and retrieval in the long-term memory of paired associates.     

Evidence for modality-specific meaning systems in the brain

Patients with cerebral lesions offer a unique opportunity to investigate the organization of meaning systems in the brain. Clinical neurologists have long been aware that knowledge of particular classes or categories of information may be selectively impaired in some cases and selectively spared in others. For example knowledge of letters, colours, objects, or people may be lost as a consequence of damage to the left hemisphere of the brain. Recently there has been quantitative evidence for even more specific impairment and preservation of particular classes of knowledge. More recently the evidence for knowledge of living things as compared with inanimate objects is particularly striking. Such observations have suggested that our semantic knowledge base is categorical in its organization. In this preliminary report, we describe a patient whose semantic knowledge deficit was not only category specific, but also modality specific. Although his knowledge of the visual world was almost entirely normal, his knowledge of living things (but not objects!) was gravely impaired when assessed in the verbal domain. These findings call into question the widely accepted view that the brain has a single all-purpose meaning store.     

A neural representation of depth from motion parallax in macaque visual cortex

Perception of depth is a fundamental challenge for the visual system, particularly for observers moving through their environment. The brain makes use of multiple visual cues to reconstruct the three-dimensional structure of a scene. One potent cue, motion parallax, frequently arises during translation of the observer because the images of objects at different distances move across the retina with different velocities. Human psychophysical studies have demonstrated that motion parallax can be a powerful depth cue, and motion parallax seems to be heavily exploited by animal species that lack highly developed binocular vision. However, little is known about the neural mechanisms that underlie this capacity. Here we show, by using a virtual-reality system to translate macaque monkeys (Macaca mulatta) while they viewed motion parallax displays that simulated objects at different depths, that many neurons in the middle temporal area (area MT) signal the sign of depth (near versus far) from motion parallax in the absence of other depth cues. To achieve this, neurons must combine visual motion with extra-retinal (non-visual) signals related to the animal's movement. Our findings suggest a new neural substrate for depth perception and demonstrate a robust interaction of visual and non-visual cues in area MT. Combined with previous studies that implicate area MT in depth perception based on binocular disparities, our results suggest that area MT contains a more general representation of three-dimensional space that makes use of multiple cues.