Covert skill learning in a cortical-basal ganglia circuit

We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning; influential 'actor-critic' models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit, contributes to skill learning even when it does not contribute to such 'exploratory' variation in behavioural performance during training. Blocking the output of the AFP while training Bengalese finches to modify their songs prevented the gradual improvement that normally occurs in this complex skill during training. However, unblocking the output of the AFP after training caused an immediate transition from naive performance to excellent performance, indicating that the AFP covertly gained the ability to implement learned skill performance without contributing to skill practice. In contrast, inactivating the output nucleus of the AFP during training completely prevented learning, indicating that learning requires activity within the AFP during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance.     

Interruption of a basal ganglia-forebrain circuit prevents plasticity of learned vocalizations

Birdsong, like speech, is a learned vocal behaviour that relies greatly on hearing; in both songbirds and humans the removal of auditory feedback by deafening leads to a gradual deterioration of adult vocal production. Here we investigate the neural mechanisms that contribute to the processing of auditory feedback during the maintenance of song in adult zebra finches. We show that the deleterious effects on song production that normally follow deafening can be prevented by a second insult to the nervous system--the lesion of a basal ganglia-forebrain circuit. The results suggest that the removal of auditory feedback leads to the generation of an instructive signal that actively drives non-adaptive changes in song; they also suggest that this instructive signal is generated within (or conveyed through) the basal ganglia-forebrain pathway. Our findings provide evidence that cortical-basal ganglia circuits may participate in the evaluation of sensory feedback during calibration of motor performance, and demonstrate that damage to such circuits can have little effect on previously learned behaviour while conspicuously disrupting the capacity to adaptively modify that behaviour.     

Performance variability enables adaptive plasticity of 'crystallized' adult birdsong

Significant trial-by-trial variation persists even in the most practiced skills. One prevalent view is that such variation is simply 'noise' that the nervous system is unable to control or that remains below threshold for behavioural relevance. An alternative hypothesis is that such variation enables trial-and-error learning, in which the motor system generates variation and differentially retains behaviours that give rise to better outcomes. Here we test the latter possibility for adult bengalese finch song. Adult birdsong is a complex, learned motor skill that is produced in a highly stereotyped fashion from one rendition to the next. Nevertheless, there is subtle trial-by-trial variation even in stable, 'crystallized' adult song. We used a computerized system to monitor small natural variations in the pitch of targeted song elements and deliver real-time auditory disruption to a subset of those variations. Birds rapidly shifted the pitch of their vocalizations in an adaptive fashion to avoid disruption. These vocal changes were precisely restricted to the targeted features of song. Hence, birds were able to learn effectively by associating small variations in their vocal behaviour with differential outcomes. Such a process could help to maintain stable, learned song despite changes to the vocal control system arising from ageing or injury. More generally, our results suggest that residual variability in well learned skills is not entirely noise but rather reflects meaningful motor exploration that can support continuous learning and optimization of performance.     

Projection neurons within a vocal motor pathway are born during song learning in zebra finches

Many birds learn song during a restricted 'sensitive' period. Juveniles memorize a song model, and then learn the pattern of muscle contractions necessary to reproduce the song. Of the neural changes accompanying avian song learning, perhaps the most remarkable is the production of new neurons which are inserted into the hyperstriatum ventralis pars caudalis (HVc), a region critical for song production. We report here that in young male zebra finches many of the new neurons incorporated into the HVc innervate the robust nucleus of the archistriatum (RA) which projects to motor neurons controlling the vocal musculature. Furthermore, far fewer of these new neurons are incorporated into the HVc of either adult males that are beyond the sensitive learning period, or young females (who do not develop song). Thus, a major portion of the vocal motor pathway is actually created during song learning. This may enable early sensory experience and vocal practice to not only modify existing neuronal circuits, but also shape the insertion and initial synaptic contacts of neurons controlling adult song.     

Different time courses of learning-related activity in the prefrontal cortex and striatum

To navigate our complex world, our brains have evolved a sophisticated ability to quickly learn arbitrary rules such as 'stop at red'. Studies in monkeys using a laboratory test of this capacity--conditional association learning--have revealed that frontal lobe structures (including the prefrontal cortex) as well as subcortical nuclei of the basal ganglia are involved in such learning. Neural correlates of associative learning have been observed in both brain regions, but whether or not these regions have unique functions is unclear, as they have typically been studied separately using different tasks. Here we show that during associative learning in monkeys, neural activity in these areas changes at different rates: the striatum (an input structure of the basal ganglia) showed rapid, almost bistable, changes compared with a slower trend in the prefrontal cortex that was more in accordance with slow improvements in behavioural performance. Also, pre-saccadic activity began progressively earlier in the striatum but not in the prefrontal cortex as learning took place. These results support the hypothesis that rewarded associations are first identified by the basal ganglia, the output of which 'trains' slower learning mechanisms in the frontal cortex.     

Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task

When an animal learns to make movements in response to different stimuli, changes in activity in the motor cortex seem to accompany and underlie this learning. The precise nature of modifications in cortical motor areas during the initial stages of motor learning, however, is largely unknown. Here we address this issue by chronically recording from neuronal ensembles located in the rat motor cortex, throughout the period required for rats to learn a reaction-time task. Motor learning was demonstrated by a decrease in the variance of the rats' reaction times and an increase in the time the animals were able to wait for a trigger stimulus. These behavioural changes were correlated with a significant increase in our ability to predict the correct or incorrect outcome of single trials based on three measures of neuronal ensemble activity: average firing rate, temporal patterns of firing, and correlated firing. This increase in prediction indicates that an association between sensory cues and movement emerged in the motor cortex as the task was learned. Such modifications in cortical ensemble activity may be critical for the initial learning of motor tasks.     

Temporal dissociation of parallel processing in the human subcortical outputs.

Many tasks require rapid and fine-tuned adjustment of motor performance based on incoming sensory information. This process of sensorimotor adaptation engages two parallel subcorticocortical neural circuits, involving the cerebellum and basal ganglia, respectively. How these distributed circuits are functionally coordinated has not been shown in humans. The cerebellum and basal ganglia show very similar convergence of input-output organization, which presents an ideal neuroimaging model for the study of parallel processing at a systems level. Here we used functional magnetic resonance imaging to measure the temporal coherence of brain activity during a tactile discrimination task. We found that, whereas the prefrontal cortex maintained a high level of activation, output activities in the cerebellum and basal ganglia showed different phasic patterns. Moreover, cerebellar activity significantly correlated with the activity of the supplementary motor area but not with that of the primary motor cortex; in contrast, basal ganglia activity was more strongly associated with the activity of the primary motor cortex than with that of the supplementary motor area. These results demonstrate temporally partitioned activity in the cerebellum and basal ganglia, implicating functional independence in the parallel subcortical outputs. This further supports the idea of task-related dynamic reconfiguration of large-scale neural networks.     

去势对成年雄性斑胸草雀HVC神经元电生理特性的影响

鸣唱控制系统的高级发声中枢HVC（high vocal center）是发声运动通路和前端脑通路的始端，是发声行为的起始控制脑区，亦可接受听觉信号的输入及反馈，是鸣禽鸣唱调控最为重要的脑区.以往研究表明,雄激素及其代谢产物对鸣禽鸣唱控制有重要作用.去势显著改变鸣禽体内激素含量，进而影响鸣禽鸣曲稳定性，但其具体机制尚未阐明.我们运用全细胞膜片钳记录法，在离体细胞水平研究了去势引起的雄激素水平降低对HVC不同神经元电生理特性的影响.研究结果显示,去势组与对照组相比，投射神经元HVCRA，HVCX膜输入电阻减小，膜时间常数降低，动作电位后超极化幅值升高及达到峰值时间延长，表明雄激素可以提高两类投射神经元的兴奋性. 综上所述，雄激素可以一定程度上提高HVC神经元的兴奋性，雄激素可增强HVC对发声运动通路（vocal motor pathway,VMP)的控制，抑制前端脑通路(anterior forebrain pathway,AFP)来实现维持鸣曲的稳定.     

Recurrent network activity drives striatal synaptogenesis

Neural activity during development critically shapes postnatal wiring of the mammalian brain. This is best illustrated by the sensory systems, in which the patterned feed-forward excitation provided by sensory organs and experience drives the formation of mature topographic circuits capable of extracting specific features of sensory stimuli. In contrast, little is known about the role of early activity in the development of the basal ganglia, a phylogenetically ancient group of nuclei fundamentally important for complex motor action and reward-based learning. These nuclei lack direct sensory input and are only loosely topographically organized, forming interlocking feed-forward and feed-back inhibitory circuits without laminar structure. Here we use transgenic mice and viral gene transfer methods to modulate neurotransmitter release and neuronal activity in vivo in the developing striatum. We find that the balance of activity between the two inhibitory and antagonist pathways in the striatum regulates excitatory innervation of the basal ganglia during development. These effects indicate that the propagation of activity through a multi-stage network regulates the wiring of the basal ganglia, revealing an important role of positive feedback in driving network maturation.     

钙激活的大电导钾离子通道在成年斑胸草雀脑中的分布

BK 通道，即钙离子激活的大电导钾离子通道，它通过产生快速的后超极化（fA HP）来控制动作电位的持续时间、发放频率。为研究BK通道在鸣禽鸣唱学习中的作用提供形态学依据，用免疫组化法观察了BK通道在成年雄性斑胸草雀脑中的分布。证实了其在端脑、基底节纹状体、中脑、小脑等脑区都有广泛的表达，其中 RA、HVC、LM AN、X区、DM 等与鸣唱系统相关的核团都有显著的表达。这暗示了BK通道可能与鸣禽鸣唱信息整合、听觉反馈、鸣曲可塑性和稳定性以及呼吸调节都有密不可分的联系。     

Endocannabinoid-mediated rescue of striatal LTD and motor deficits in Parkinson's disease models

The striatum is a major forebrain nucleus that integrates cortical and thalamic afferents and forms the input nucleus of the basal ganglia. Striatal projection neurons target the substantia nigra pars reticulata (direct pathway) or the lateral globus pallidus (indirect pathway). Imbalances between neural activity in these two pathways have been proposed to underlie the profound motor deficits observed in Parkinson's disease and Huntington's disease. However, little is known about differences in cellular and synaptic properties in these circuits. Indeed, current hypotheses suggest that these cells express similar forms of synaptic plasticity. Here we show that excitatory synapses onto indirect-pathway medium spiny neurons (MSNs) exhibit higher release probability and larger N-methyl-d-aspartate receptor currents than direct-pathway synapses. Moreover, indirect-pathway MSNs selectively express endocannabinoid-mediated long-term depression (eCB-LTD), which requires dopamine D2 receptor activation. In models of Parkinson's disease, indirect-pathway eCB-LTD is absent but is rescued by a D2 receptor agonist or inhibitors of endocannabinoid degradation. Administration of these drugs together in vivo reduces parkinsonian motor deficits, suggesting that endocannabinoid-mediated depression of indirect-pathway synapses has a critical role in the control of movement. These findings have implications for understanding the normal functions of the basal ganglia, and also suggest approaches for the development of therapeutic drugs for the treatment of striatal-based brain disorders.     

Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories

Learning to perform a behavioural procedure as a well-ingrained habit requires extensive repetition of the behavioural sequence, and learning not to perform such behaviours is notoriously difficult. Yet regaining a habit can occur quickly, with even one or a few exposures to cues previously triggering the behaviour. To identify neural mechanisms that might underlie such learning dynamics, we made long-term recordings from multiple neurons in the sensorimotor striatum, a basal ganglia structure implicated in habit formation, in rats successively trained on a reward-based procedural task, given extinction training and then given reacquisition training. The spike activity of striatal output neurons, nodal points in cortico-basal ganglia circuits, changed markedly across multiple dimensions during each of these phases of learning. First, new patterns of task-related ensemble firing successively formed, reversed and then re-emerged. Second, task-irrelevant firing was suppressed, then rebounded, and then was suppressed again. These changing spike activity patterns were highly correlated with changes in behavioural performance. We propose that these changes in task representation in cortico-basal ganglia circuits represent neural equivalents of the explore-exploit behaviour characteristic of habit learning.     

How sleep affects the developmental learning of bird song

Sleep affects learning and development in humans and other animals, but the role of sleep in developmental learning has never been examined. Here we show the effects of night-sleep on song development in the zebra finch by recording and analysing the entire song ontogeny. During periods of rapid learning we observed a pronounced deterioration in song structure after night-sleep. The song regained structure after intense morning singing. Daily improvement in similarity to the tutored song occurred during the late phase of this morning recovery; little further improvement occurred thereafter. Furthermore, birds that showed stronger post-sleep deterioration during development achieved a better final imitation. The effect diminished with age. Our experiments showed that these oscillations were not a result of sleep inertia or lack of practice, indicating the possible involvement of an active process, perhaps neural song-replay during sleep. We suggest that these oscillations correspond to competing demands of plasticity and consolidation during learning, creating repeated opportunities to reshape previously learned motor skills.     

An ultra-sparse code underlies the generation of neural sequences in a songbird

Sequences of motor activity are encoded in many vertebrate brains by complex spatio-temporal patterns of neural activity; however, the neural circuit mechanisms underlying the generation of these pre-motor patterns are poorly understood. In songbirds, one prominent site of pre-motor activity is the forebrain robust nucleus of the archistriatum (RA), which generates stereotyped sequences of spike bursts during song and recapitulates these sequences during sleep. We show that the stereotyped sequences in RA are driven from nucleus HVC (high vocal centre), the principal pre-motor input to RA. Recordings of identified HVC neurons in sleeping and singing birds show that individual HVC neurons projecting onto RA neurons produce bursts sparsely, at a single, precise time during the RA sequence. These HVC neurons burst sequentially with respect to one another. We suggest that at each time in the RA sequence, the ensemble of active RA neurons is driven by a subpopulation of RA-projecting HVC neurons that is active only at that time. As a population, these HVC neurons may form an explicit representation of time in the sequence. Such a sparse representation, a temporal analogue of the 'grandmother cell' concept for object recognition, eliminates the problem of temporal interference during sequence generation and learning attributed to more distributed representations.     

Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills

The ability to learn new skills and perfect them with practice applies not only to physical skills but also to abstract skills, like motor planning or neuroprosthetic actions. Although plasticity in corticostriatal circuits has been implicated in learning physical skills, it remains unclear if similar circuits or processes are required for abstract skill learning. Here we use a novel behavioural task in rodents to investigate the role of corticostriatal plasticity in abstract skill learning. Rodents learned to control the pitch of an auditory cursor to reach one of two targets by modulating activity in primary motor cortex irrespective of physical movement. Degradation of the relation between action and outcome, as well as sensory-specific devaluation and omission tests, demonstrate that these learned neuroprosthetic actions are intentional and goal-directed, rather than habitual. Striatal neurons change their activity with learning, with more neurons modulating their activity in relation to target-reaching as learning progresses. Concomitantly, strong relations between the activity of neurons in motor cortex and the striatum emerge. Specific deletion of striatal NMDA receptors impairs the development of this corticostriatal plasticity, and disrupts the ability to learn neuroprosthetic skills. These results suggest that corticostriatal plasticity is necessary for abstract skill learning, and that neuroprosthetic movements capitalize on the neural circuitry involved in natural motor learning.     

Primate spinal interneurons show pre-movement instructed delay activity.

Preparatory changes in neural activity before the execution of a movement have been documented in tasks that involve an instructed delay period (an interval between a transient instruction cue and a subsequently triggered movement). Such preparatory activity occurs in many motor centres in the brain, including the primary motor cortex, premotor cortex, supplementary motor area and basal ganglia. Activity during the instructed delay period reflects movement planning, as it correlates with parameters of the cue and the subsequent movement (such as direction and extent), although it occurs well before muscle activity. How such delay-period activity shapes the ensuing motor action remains unknown. Here we show that spinal interneurons also exhibit early pre-movement delay activity that often differs from their responses during the subsequent muscle activity. This delay activity resembles the set-related activity found in various supraspinal areas, indicating that movement preparation may occur simultaneously over widely distributed regions, including spinal levels. Our results also suggest that two processes occur in the spinal circuitry during this delay period: the motor network is primed with rate changes in the same direction as subsequent movement-related activity; and a superimposed global inhibition suppresses the expression of this activity in muscles.     

The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems

The striatum (caudate-putamen) of the basal ganglia in the mammalian forebrain is a mosaic of two interdigitating, neurochemically distinct compartments. One type, the 'patch' compartment, is identified by patches of dense opiate receptor binding, and is enriched in enkephalin- and substance P-like immunoreactivity. The other compartment, the 'matrix', has a high acetylcholinesterase activity, and is shown here to have a dense plexus of fibres displaying somatostatin-like immunoreactivity. The present study demonstrates the two compartments have distinct connections, using a method that concurrently reveals striatal input, output and neurochemical systems in the rat. Patches receive inputs from the prelimbic cortex (a medial frontal cortical area with direct 'limbic' inputs from the amygdala and hippocampus); they also project to the substantia nigra pars compacta (the source of the nigrostriatal dopaminergic system). Conversely, the matrix receives inputs from sensory and motor cortical areas; here it is shown to project to the substantia nigra pars reticulata (the source of the non-dopaminergic nigrothalamic and nigrotectal system). Also, an intrinsic striatal somatostatin-immunoreactive system is described that may provide a link between the two compartments. The striatal patch and matrix compartments thus appear to be functionally distinct and interactive parallel input-output processing channels.     

去势对成年雄性斑胸草雀HVC-RA突触可塑性的影响

应用在体电生理方法研究了去势前后成年雄性斑胸草雀发声运动通路中HVC-RA 突触的可塑性变化,进一步探讨雄激素在调节鸣唱行为中的作用和机制.结果表明:低频刺激可引起 HVC-RA突触群体峰电位幅度的短时程抑制(Short-term depression, STD),高频刺激可引起群体峰电位幅度的长时程抑制(Long-term depression, LTD).而去势后30 d,鸣曲稳定时再给予同样的条件刺激,发现无论低频或高频刺激,HVC-RA 突触的短时程抑制和长时程抑制现象同时消失.研究结果显示:鸣曲稳定性可能依赖于HVC-RA通路的突触可塑性,雄激素在维持鸣曲稳定过程中发挥重要作用.     

The role of sex steroids in the acquisition and production of birdsong

Male birdsong is generally regarded as a secondary sexual characteristic under the control of gonadal steroids. Song typically waxes and wanes with the seasonal cycle of testicular growth and regression and decreases after adult castration. Testosterone therapy reinstates song, induces it in females, augments it in intact males, and spring testosterone profiles correlate with seasonal song production. Thus, testosterone has been viewed as a major factor in song acquisition and production acting either directly, or after aromatization within the brain. We show here, however, that song learning and early phases of the development of singing both take place in castrated male birds with no significant levels of testosterone in their blood plasma. Testosterone seems to be required for song crystallization, however. Oestradiol was unexpectedly still present after castration, evidently from a non-testicular source, throughout the period of male song acquisition.