Temporal difference models describe higher-order learning in humans

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.     

Marine ecology: different measures of biodiversity

In their analysis of global trends of diversity in marine plankton, Irigoien et al. find that taxonomic diversity of zooplankton (consumer) is a unimodal function of community biomass that is unrelated to phytoplankton (producer) diversity. Their results are unexpected because in terrestrial organisms primary-producer diversity is a good predictor of consumer diversity. I contend that this apparent uncoupling of producer and consumer diversity in marine plankton is likely to be an artefact due to the authors' use of different measures of diversity for producers and for consumers.     

Cortical substrates for exploratory decisions in humans

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.     

A RhoGDP dissociation inhibitor spatially regulates growth in root hair cells

Root hairs are cellular protuberances extending from the root surface into the soil; there they provide access to immobile inorganic ions such as phosphate, which are essential for growth. Their cylindrical shape results from a polarized mechanism of cell expansion called tip growth in which elongation is restricted to a small area at the surface of the hair-forming cell (trichoblast) tip. Here we identify proteins that spatially control the sites at which cell growth occurs by isolating Arabidopsis mutants (scn1) that develop ectopic sites of growth on trichoblasts. We cloned SCN1 and showed that SCN1 is a RhoGTPase GDP dissociation inhibitor (RhoGDI) that spatially restricts the sites of growth to a single point on the trichoblast. We showed previously that localized production of reactive oxygen species by RHD2/AtrbohC NADPH oxidase is required for hair growth; here we show that SCN1/AtrhoGDI1 is a component of the mechanism that focuses RHD2/AtrbohC-catalysed production of reactive oxygen species to hair tips during wild-type development. We propose that the spatial organization of growth in plant cells requires the local RhoGDI-regulated activation of the RHD2/AtrbohC NADPH oxidase.     

Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.     

Reactive oxygen species produced by NADPH oxidase regulate plant cell growth

Cell expansion is a central process in plant morphogenesis, and the elongation of roots and root hairs is essential for uptake of minerals and water from the soil. Ca2+ influx from the extracellular store is required for (and sets the rates of) cell elongation in roots. Arabidopsis thaliana rhd2 mutants are defective in Ca2+ uptake and consequently cell expansion is compromised--rhd2 mutants have short root hairs and stunted roots. To determine the regulation of Ca2+ acquisition in growing root cells we show here that RHD2 is an NADPH oxidase, a protein that transfers electrons from NADPH to an electron acceptor leading to the formation of reactive oxygen species (ROS). We show that ROS accumulate in growing wild-type (WT) root hairs but their levels are markedly decreased in rhd2 mutants. Blocking the activity of the NADPH oxidase with diphenylene iodonium (DPI) inhibits ROS formation and phenocopies Rhd2-. Treatment of rhd2 roots with ROS partly suppresses the mutant phenotype and stimulates the activity of plasma membrane hyperpolarization-activated Ca2+ channels, the predominant root Ca2+ acquisition system. This indicates that NADPH oxidases control development by making ROS that regulate plant cell expansion through the activation of Ca2+ channels.     

Reward value of attractiveness and gaze

Faces are visual objects in our environment that provide strong social cues, with the eyes assuming particular importance. Here we show that the perceived attractiveness of an unfamiliar face increases brain activity in the ventral striatum of the viewer when meeting the person's eye, and decreases activity when eye gaze is directed away. Depending on the direction of gaze, attractiveness can thus activate dopaminergic regions that are strongly linked to reward prediction, indicating that central reward systems may be engaged during the initiation of social interactions.