Covert skill learning in a cortical-basal ganglia circuit

We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning; influential 'actor-critic' models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit, contributes to skill learning even when it does not contribute to such 'exploratory' variation in behavioural performance during training. Blocking the output of the AFP while training Bengalese finches to modify their songs prevented the gradual improvement that normally occurs in this complex skill during training. However, unblocking the output of the AFP after training caused an immediate transition from naive performance to excellent performance, indicating that the AFP covertly gained the ability to implement learned skill performance without contributing to skill practice. In contrast, inactivating the output nucleus of the AFP during training completely prevented learning, indicating that learning requires activity within the AFP during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance.     

Deconstruction of a neural circuit for hunger

Hunger is a complex behavioural state that elicits intense food seeking and consumption. These behaviours are rapidly recapitulated by activation of starvation-sensitive AGRP neurons, which present an entry point for reverse-engineering neural circuits for hunger. Here we mapped synaptic interactions of AGRP neurons with multiple cell populations in mice and probed the contribution of these distinct circuits to feeding behaviour using optogenetic and pharmacogenetic techniques. An inhibitory circuit with paraventricular hypothalamus (PVH) neurons substantially accounted for acute AGRP neuron-evoked eating, whereas two other prominent circuits were insufficient. Within the PVH, we found that AGRP neurons target and inhibit oxytocin neurons, a small population that is selectively lost in Prader-Willi syndrome, a condition involving insatiable hunger. By developing strategies for evaluating molecularly defined circuits, we show that AGRP neuron suppression of oxytocin neurons is critical for evoked feeding. These experiments reveal a new neural circuit that regulates hunger state and pathways associated with overeating disorders.     

基于FPGA并行加速的脉冲神经网络在线学习硬件结构的设计与实现

当前，基于数字电路的脉冲神经网络硬件设计，在学习功能方面的突触并行性不高，导致硬件整体延时较大，在一定程度上限制了脉冲神经网络模型在线学习的速度。针对上述问题，文中提出了一种基于FPGA并行加速的高效脉冲神经网络在线学习硬件结构，通过神经元和突触的双并行设计对模型的训练与推理过程进行加速。首先，设计具有并行脉冲传递功能和并行脉冲时间依赖可塑性学习功能的突触结构；然后，搭建输入编码层和赢家通吃结构的学习层，并优化赢家通吃网络的侧向抑制的实现，形成规模为784～400的脉冲神经网络模型。实验结果表明：在MNIST数据集上，使用该硬件结构的脉冲神经网络模型训练一幅图像需要的时间为1.61 ms、能耗约为3.18 mJ，推理一幅图像需要的时间为1.19 ms、能耗约为2.37 mJ，识别MNIST测试集样本的准确率可达87.51%；在文中设计的硬件框架下，突触并行结构能使训练速度提升38%以上，硬件能耗降低约24.1%，有助于促进边缘智能计算设备及技术的发展。     

Brain-wide neuronal dynamics during motor adaptation in zebrafish

A fundamental question in neuroscience is how entire neural circuits generate behaviour and adapt it to changes in sensory feedback. Here we use two-photon calcium imaging to record the activity of large populations of neurons at the cellular level, throughout the brain of larval zebrafish expressing a genetically encoded calcium sensor, while the paralysed animals interact fictively with a virtual environment and rapidly adapt their motor output to changes in visual feedback. We decompose the network dynamics involved in adaptive locomotion into four types of neuronal response properties, and provide anatomical maps of the corresponding sites. A subset of these signals occurred during behavioural adjustments and are candidates for the functional elements that drive motor learning. Lesions to the inferior olive indicate a specific functional role for olivocerebellar circuitry in adaptive locomotion. This study enables the analysis of brain-wide dynamics at single-cell resolution during behaviour.     

Neural correlates, computation and behavioural impact of decision confidence

Humans and other animals must often make decisions on the basis of imperfect evidence. Statisticians use measures such as P values to assign degrees of confidence to propositions, but little is known about how the brain computes confidence estimates about decisions. We explored this issue using behavioural analysis and neural recordings in rats in combination with computational modelling. Subjects were trained to perform an odour categorization task that allowed decision confidence to be manipulated by varying the distance of the test stimulus to the category boundary. To understand how confidence could be computed along with the choice itself, using standard models of decision-making, we defined a simple measure that quantified the quality of the evidence contributing to a particular decision. Here we show that the firing rates of many single neurons in the orbitofrontal cortex match closely to the predictions of confidence models and cannot be readily explained by alternative mechanisms, such as learning stimulus-outcome associations. Moreover, when tested using a delayed reward version of the task, we found that rats' willingness to wait for rewards increased with confidence, as predicted by the theoretical model. These results indicate that confidence estimates, previously suggested to require 'metacognition' and conscious awareness are available even in the rodent brain, can be computed with relatively simple operations, and can drive adaptive behaviour. We suggest that confidence estimation may be a fundamental and ubiquitous component of decision-making.     

Odour-plume dynamics influence the brain's olfactory code

The neural computations used to represent olfactory information in the brain have long been investigated. Recent studies in the insect antennal lobe suggest that precise temporal and/or spatial patterns of activity underlie the recognition and discrimination of different odours, and that these patterns may be strengthened by associative learning. It remains unknown, however, whether these activity patterns persist when odour intensity varies rapidly and unpredictably, as often occurs in nature. Here we show that with naturally intermittent odour stimulation, spike patterns recorded from moth antennal-lobe output neurons varied predictably with the fine-scale temporal dynamics and intensity of the odour. These data support the hypothesis that olfactory circuits compensate for contextual variations in the stimulus pattern with high temporal precision. The timing of output neuron activity is constantly modulated to reflect ongoing changes in stimulus intensity and dynamics that occur on a millisecond timescale.     

Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills

The ability to learn new skills and perfect them with practice applies not only to physical skills but also to abstract skills, like motor planning or neuroprosthetic actions. Although plasticity in corticostriatal circuits has been implicated in learning physical skills, it remains unclear if similar circuits or processes are required for abstract skill learning. Here we use a novel behavioural task in rodents to investigate the role of corticostriatal plasticity in abstract skill learning. Rodents learned to control the pitch of an auditory cursor to reach one of two targets by modulating activity in primary motor cortex irrespective of physical movement. Degradation of the relation between action and outcome, as well as sensory-specific devaluation and omission tests, demonstrate that these learned neuroprosthetic actions are intentional and goal-directed, rather than habitual. Striatal neurons change their activity with learning, with more neurons modulating their activity in relation to target-reaching as learning progresses. Concomitantly, strong relations between the activity of neurons in motor cortex and the striatum emerge. Specific deletion of striatal NMDA receptors impairs the development of this corticostriatal plasticity, and disrupts the ability to learn neuroprosthetic skills. These results suggest that corticostriatal plasticity is necessary for abstract skill learning, and that neuroprosthetic movements capitalize on the neural circuitry involved in natural motor learning.     

Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task

When an animal learns to make movements in response to different stimuli, changes in activity in the motor cortex seem to accompany and underlie this learning. The precise nature of modifications in cortical motor areas during the initial stages of motor learning, however, is largely unknown. Here we address this issue by chronically recording from neuronal ensembles located in the rat motor cortex, throughout the period required for rats to learn a reaction-time task. Motor learning was demonstrated by a decrease in the variance of the rats' reaction times and an increase in the time the animals were able to wait for a trigger stimulus. These behavioural changes were correlated with a significant increase in our ability to predict the correct or incorrect outcome of single trials based on three measures of neuronal ensemble activity: average firing rate, temporal patterns of firing, and correlated firing. This increase in prediction indicates that an association between sensory cues and movement emerged in the motor cortex as the task was learned. Such modifications in cortical ensemble activity may be critical for the initial learning of motor tasks.     

Switching on and off fear by distinct neuronal circuits

Switching between exploratory and defensive behaviour is fundamental to survival of many animals, but how this transition is achieved by specific neuronal circuits is not known. Here, using the converse behavioural states of fear extinction and its context-dependent renewal as a model in mice, we show that bi-directional transitions between states of high and low fear are triggered by a rapid switch in the balance of activity between two distinct populations of basal amygdala neurons. These two populations are integrated into discrete neuronal circuits differentially connected with the hippocampus and the medial prefrontal cortex. Targeted and reversible neuronal inactivation of the basal amygdala prevents behavioural changes without affecting memory or expression of behaviour. Our findings indicate that switching between distinct behavioural states can be triggered by selective activation of specific neuronal circuits integrating sensory and contextual information. These observations provide a new framework for understanding context-dependent changes of fear behaviour.     

Correlation between neural spike trains increases with firing rate

Populations of neurons in the retina, olfactory system, visual and somatosensory thalamus, and several cortical regions show temporal correlation between the discharge times of their action potentials (spike trains). Correlated firing has been linked to stimulus encoding, attention, stimulus discrimination, and motor behaviour. Nevertheless, the mechanisms underlying correlated spiking are poorly understood, and its coding implications are still debated. It is not clear, for instance, whether correlations between the discharges of two neurons are determined solely by the correlation between their afferent currents, or whether they also depend on the mean and variance of the input. We addressed this question by computing the spike train correlation coefficient of unconnected pairs of in vitro cortical neurons receiving correlated inputs. Notably, even when the input correlation remained fixed, the spike train output correlation increased with the firing rate, but was largely independent of spike train variability. With a combination of analytical techniques and numerical simulations using 'integrate-and-fire' neuron models we show that this relationship between output correlation and firing rate is robust to input heterogeneities. Finally, this overlooked relationship is replicated by a standard threshold-linear model, demonstrating the universality of the result. This connection between the rate and correlation of spiking activity links two fundamental features of the neural code.     

Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning

What neural changes underlie individual differences in goal-directed learning? The lateral amygdala (LA) is important for assigning emotional and motivational significance to discrete environmental cues, including those that signal rewarding events. Recognizing that a cue predicts a reward enhances an animal's ability to acquire that reward; however, the cellular and synaptic mechanisms that underlie cue-reward learning are unclear. Here we show that marked changes in both cue-induced neuronal firing and input-specific synaptic strength occur with the successful acquisition of a cue-reward association within a single training session. We performed both in vivo and ex vivo electrophysiological recordings in the LA of rats trained to self-administer sucrose. We observed that reward-learning success increased in proportion to the number of amygdala neurons that responded phasically to a reward-predictive cue. Furthermore, cue-reward learning induced an AMPA (alpha-amino-3-hydroxy-5-methyl-isoxazole propionic acid)-receptor-mediated increase in the strength of thalamic, but not cortical, synapses in the LA that was apparent immediately after the first training session. The level of learning attained by individual subjects was highly correlated with the degree of synaptic strength enhancement. Importantly, intra-LA NMDA (N-methyl-d-aspartate)-receptor blockade impaired reward-learning performance and attenuated the associated increase in synaptic strength. These findings provide evidence of a connection between LA synaptic plasticity and cue-reward learning, potentially representing a key mechanism underlying goal-directed behaviour.     

Anhedonia requires MC4R-mediated synaptic adaptations in nucleus accumbens

Chronic stress is a strong diathesis for depression in humans and is used to generate animal models of depression. It commonly leads to several major symptoms of depression, including dysregulated feeding behaviour, anhedonia and behavioural despair. Although hypotheses defining the neural pathophysiology of depression have been proposed, the critical synaptic adaptations in key brain circuits that mediate stress-induced depressive symptoms remain poorly understood. Here we show that chronic stress in mice decreases the strength of excitatory synapses on D1 dopamine receptor-expressing nucleus accumbens medium spiny neurons owing to activation of the melanocortin 4 receptor. Stress-elicited increases in behavioural measurements of anhedonia, but not increases in measurements of behavioural despair, are prevented by blocking these melanocortin 4 receptor-mediated synaptic changes in vivo. These results establish that stress-elicited anhedonia requires a neuropeptide-triggered, cell-type-specific synaptic adaptation in the nucleus accumbens and that distinct circuit adaptations mediate other major symptoms of stress-elicited depression.     

Choice-specific sequences in parietal cortex during a virtual-navigation decision task

The posterior parietal cortex (PPC) has an important role in many cognitive behaviours; however, the neural circuit dynamics underlying PPC function are not well understood. Here we optically imaged the spatial and temporal activity patterns of neuronal populations in mice performing a PPC-dependent task that combined a perceptual decision and memory-guided navigation in a virtual environment. Individual neurons had transient activation staggered relative to one another in time, forming a sequence of neuronal activation spanning the entire length of a task trial. Distinct sequences of neurons were triggered on trials with opposite behavioural choices and defined divergent, choice-specific trajectories through a state space of neuronal population activity. Cells participating in the different sequences and at distinct time points in the task were anatomically intermixed over microcircuit length scales (<100 micrometres). During working memory decision tasks, the PPC may therefore perform computations through sequence-based circuit dynamics, rather than long-lived stable states, implemented using anatomically intermingled microcircuits.     

神经放电节律的不同复杂分岔序列的实验研究

神经放电节律在参数空间的分岔结构不仅是重要的动力学问题,对于揭示神经信息编码也具有重要的意义．在大鼠损伤坐骨神经模型,研究了不同K^＋通道阻断剂的浓度下神经自发放电随Ca^2＋浓度变化的分岔序列规律．结果发现,包含节律模式种类较少的简单分岔序列可以与包含多类周期和混沌的复杂分岔序列相互转换,复杂分岔序列之间也可以转换．分岔点附近的混沌节律的不同是识别分岔序列不同的主要标志．研究结果不仅丰富了神经放电的非线性动力学现象,也会为建立神经放电在参数空间的分岔序列结构进而揭示神经编码机制提供帮助．     

Signal but not noise changes with perceptual learning

Perceptual discrimination improves with practice. This 'perceptual learning' is often specific to the stimuli presented during training, indicating that practice may alter the response characteristics of cortical sensory neurons. Although much is known about how learning modifies cortical circuits, it remains unclear how these changes relate to behaviour. Different theories assume that practice improves discrimination by enhancing the signal, diminishing internal noise or both. Here, to distinguish among these alternatives, we fashioned sets of faces and textures whose signal strength could be varied, and we trained observers to identify these patterns embedded in noise. Performance increased by up to 400% across several sessions over several days. Comparisons of human performance to that of an ideal discriminator showed that learning increased the efficiency with which observers encoded task-relevant information. Observer response consistency, measured by a double-pass technique in which identical stimuli are shown twice in each experimental session, did not change during training, showing that learning had no effect on internal noise. These results indicate that perceptual learning may enhance signal strength, and provide important constraints for theories of learning.     

Temporal dissociation of parallel processing in the human subcortical outputs.

Many tasks require rapid and fine-tuned adjustment of motor performance based on incoming sensory information. This process of sensorimotor adaptation engages two parallel subcorticocortical neural circuits, involving the cerebellum and basal ganglia, respectively. How these distributed circuits are functionally coordinated has not been shown in humans. The cerebellum and basal ganglia show very similar convergence of input-output organization, which presents an ideal neuroimaging model for the study of parallel processing at a systems level. Here we used functional magnetic resonance imaging to measure the temporal coherence of brain activity during a tactile discrimination task. We found that, whereas the prefrontal cortex maintained a high level of activation, output activities in the cerebellum and basal ganglia showed different phasic patterns. Moreover, cerebellar activity significantly correlated with the activity of the supplementary motor area but not with that of the primary motor cortex; in contrast, basal ganglia activity was more strongly associated with the activity of the primary motor cortex than with that of the supplementary motor area. These results demonstrate temporally partitioned activity in the cerebellum and basal ganglia, implicating functional independence in the parallel subcortical outputs. This further supports the idea of task-related dynamic reconfiguration of large-scale neural networks.     

Attention modulates synchronized neuronal firing in primate somatosensory cortex

A potentially powerful information processing strategy in the brain is to take advantage of the temporal structure of neuronal spike trains. An increase in synchrony within the neural representation of an object or location increases the efficacy of that neural representation at the next synaptic stage in the brain; thus, increasing synchrony is a candidate for the neural correlate of attentional selection. We investigated the synchronous firing of pairs of neurons in the secondary somatosensory cortex (SII) of three monkeys trained to switch attention between a visual task and a tactile discrimination task. We found that most neuron pairs in SII cortex fired synchronously and, furthermore, that the degree of synchrony was affected by the monkey's attentional state. In the monkey performing the most difficult task, 35% of neuron pairs that fired synchronously changed their degree of synchrony when the monkey switched attention between the tactile and visual tasks. Synchrony increased in 80% and decreased in 20% of neuron pairs affected by attention.     

Practising orientation identification improves orientation coding in V1 neurons

The adult brain shows remarkable plasticity, as demonstrated by the improvement in fine sensorial discriminations after intensive practice. The behavioural aspects of such perceptual learning are well documented, especially in the visual system. Specificity for stimulus attributes clearly implicates an early cortical site, where receptive fields retain fine selectivity for these attributes; however, the neuronal correlates of a simple visual discrimination task remained unidentified. Here we report electrophysiological correlates in the primary visual cortex (V1) of monkeys for learning orientation identification. We link the behavioural improvement in this type of learning to an improved neuronal performance of trained compared to naive neurons. Improved long-term neuronal performance resulted from changes in the characteristics of orientation tuning of individual neurons. More particularly, the slope of the orientation tuning curve that was measured at the trained orientation increased only for the subgroup of trained neurons most likely to code the orientation identified by the monkey. No modifications of the tuning curve were observed for orientations for which the monkey had not been trained. Thus training induces a specific and efficient increase in neuronal sensitivity in V1.     

A neural correlate of response bias in monkey caudate nucleus

Primates are equipped with neural circuits in the prefrontal cortex, the parietal cortex and the basal ganglia that predict the availability of reward during the performance of behavioural tasks. It is not known, however, how reward value is incorporated in the control of action. Here we identify neurons in the monkey caudate nucleus that create a spatially selective response bias depending on the expected gain. In behavioural tasks, the monkey had to make a visually guided eye movement in every trial, but was rewarded for a correct response in only half of the trials. Reward availability was predictable on the basis of the spatial position of the visual target. We found that caudate neurons change their discharge rate systematically, even before the appearance of the visual target, and usually fire more when the contralateral position is associated with reward. Strong anticipatory activity of neurons with a contralateral preference is associated with decreased latency for eye movements in the contralateral direction. We conclude that this neuronal mechanism creates an advance bias that favours a spatial response when it is associated with a high reward value.     

Moving visual stimuli rapidly induce direction sensitivity of developing tectal neurons

During development of the visual system, the pattern of visual inputs may have an instructive role in refining developing neural circuits. How visual inputs of specific spatiotemporal patterns shape the circuit development remains largely unknown. We report here that, in the developing Xenopus retinotectal system, the receptive field of tectal neurons can be 'trained' to become direction-sensitive within minutes after repetitive exposure of the retina to moving bars in a particular direction. The induction of direction-sensitivity depends on the speed of the moving bar, can not be induced by random visual stimuli, and is accompanied by an asymmetric modification of the tectal neuron's receptive field. Furthermore, such training-induced changes require spiking of the tectal neuron and activation of a NMDA (N-methyl-D-aspartate) subtype of glutamate receptors during training, and are attributable to an activity-induced enhancement of glutamate-mediated inputs. Thus, developing neural circuits can be modified rapidly and specifically by visual inputs of defined spatiotemporal patterns, in a manner consistent with predictions based on spike-time-dependent synaptic modification.