How does dopamine regulate both learning and motivation?

A new study from the Netherlands Institute for Neuroscience brings together two schools of thought on the function of the neurotransmitter dopamine: one saying that dopamine provides a learning signal, the other saying that dopamine drives motivation. 'But it is probably both', says Ingo Willuhn.

It is well-known that the dopamine system is implicated in signaling reward-related information as well as in actions that generate rewarding outcomes. This can be investigated using either Pavlovian and operant conditioning experiments. Pavlovian conditioning describes how your brain makes an association between two situations or stimuli that previously seemed unrelated. A famous example is Pavlov's experiment, where a dog heard a sound before receiving food. After several such pairings of the sound with food delivery, the sound alone began to cause the dog to salivate. Operant conditioning, or instrumental learning, differs from this in that the behavior of an individual is important to earn a food reward. Meaning that the individual after hearing a sound, has to perform a so-called operant action to receive the reward. In animal experiments, such a operant response is often the pressing of a lever.

Dopamine measurements in nucleus accumbens

In the final PhD paper of Jessica Goedhoop in collaboration with Tara Arbab and Ingo Willuhn from the Netherlands Institute for Neuroscience, they take a closer look at the role of dopamine signaling in learning and motivation. The team directly compared the two conditioning paradigms: male rats underwent either Pavlovian or operant conditioning while dopamine release was measured in the nucleus accumbens, a brain region central for processing this information. During the experiments a cue light was illuminated for a duration of 5 seconds. For the Pavlovian group, a food pellet was delivered into the reward magazine directly after the cue light turned off. For the operant conditioning group, turning off the cue light was followed by extension of the lever below the cue light into the operant box. The lever was retracted after one lever press, which immediately resulted in the delivery of one food pellet reward into the food magazine. If there was no lever press within 5 seconds after lever extension, the lever was retracted and no reward was delivered.

Sustained dopamine release in operant conditioning

Rats in both groups released the same quantity of dopamine at the onset of the reward-predictive cue. However, only the operant-conditioning group showed a subsequent, sustained plateau in dopamine concentration throughout the entire 5-second cue presentation (throughout cue presentation and before lever press). This dopamine sustainment was observed reliably and consistently throughout systematic manipulation of experimental parameters and behavioral training. Therefore, the researchers believe that sustained dopamine levels may be an intermediate between learning and action, conceptually related to the motivation to generate a reward-achieving action.

Ingo Willuhn: 'There have been a lot of studies on dopamine. We have a decent idea of when dopamine is released in the brain, but there is still lots of discussion on what the precise variables are that determine such dopamine signaling. Essentially discussion on what dopamine "means." To investigate this, scientists usually perform either Pavlovian or operant conditioning experiments. But they test slightly different things. Both have to do with learning an association between a neutral stimulus and a reward. But operant conditioning requires the motivation to perform an action in addition to that (to earn the reward). Therefore, we compared the two types of conditioning in the same experiment.'

Adding a piece to the puzzle

'Our results bring together the two camps of scientists that often battle with each other: one says that dopamine is a so-called reward-prediction error signal, meaning that dopamine is released when something better than expected happens, and is suppressed when something worse than expected happens. It is a learning (or teaching) signal. The other camp says that this is not true. They say that dopamine has something to do with motivation. Increased dopamine release will invigorate the subjects and they work harder to get the reward. There have been a few attempts in the past to bring these two camps together, but there is still need for more knowledge on the subject.'

'What we saw in our study is that only in the operant-learning task dopamine levels stayed high. It seems that the motivation is encoded in this plateau. Reward prediction is the initial dopamine peak, but how much the signal stays up, reflects motivation. Thus, our paper suggests that there is a possibility that dopamine is involved in both, learning and motivation. The next steps will be to get more details out of this. We need to replicate the experiments and make them more sophisticated. The more sophisticated you make it, the more precise our predictions have to be. We are going to build on it and see whether it still holds up.'

Implications

'Dopamine is not only involved in everyday life but also in disorders such as addiction, Parkinson's disease, and schizophrenia. Because of the two camps existing, there is disagreement about what happens exactly. For example, some researchers say that when addicts take drugs dopamine release increases and as a consequence all the environmental cues become more meaningful. Addicts learn that these cues are associated with the drug and they take more and more drug, because they are constantly reminded of the drug everywhere. In this view, addiction is misguided learning. Other researchers would say that motivation to take the drug intensifies with more frequent drug intake, because the drug elevates dopamine release. This study indicates that it may be both. Depending on the precise timing, both systems could be the driver, and both could be involved.'

Read more at Science Daily

Sugar is processed differently in the brains of obesity-prone vs. obesity-resistant rats

On a diet? Perhaps you're avoiding sweets or carbs altogether or curbing late-night munchies. These are examples of behavior modifications and when it comes to food, avoiding those diet triggers can be pretty hard to do.

To understand what drives people to overeat, scientists are looking more closely at a brain structure involved in motivation, called the nucleus accumbens. This small region drives reward-seeking behaviors underlying the pursuit of sex, recreational drugs like nicotine and alcohol, and food.

"These brain motivation centers evolved to help us survive; finding food and having sex are essential to the survival of an individual and of a species," said Carrie Ferrario, Ph.D., associate professor in the Department of Pharmacology at U-M Medical School.

"What was advantageous when food was hard to find has become a disadvantage and unhealthy in the current food dense environment. This is compounded by the over-abundance of over-processed, low nutrition foods that may satisfy our taste but leave our bodies unnourished. People don't tend to find it difficult to turn down an extra serving of broccoli, but just one more french-fry or making room for a bit of chocolate dessert...that's a different story. The real challenge is overcoming these urges and changing our behavior when it comes to food," Ferrario added.

Given the immense toll obesity takes on virtually all body systems, Ferrario, Peter Vollbrecht, Ph.D., of Western Michigan University, and their colleagues are using rat models to understand potential brain differences between animals who are prone to over-eating and obesity and those who are not.

Previous research from Ferrario's lab pinpointed differences in the nucleus accumbens in obesity-prone and obesity-resistant rats. Their latest study, published in the Journal of Neurochemistry, tracked what was happening in real time in the brain when these animals were presented with glucose, a type of sugar, labeled with a tracer. The tracer allowed the researchers to measure this new sugar in the brain.

Sugar is the brain's main fuel source and once there, the molecule is broken down and used to create new molecules such as glutamine, glutamate, and GABA, each with an important role in influencing the activation of neurons in the brain and nervous system.

"Glucose that is consumed gets broken down and then its carbons get incorporated into neurotransmitters. We see those labelled carbons showing up in those molecules -- glutamate, glutamine, and GABA -- over time," explained Vollbrecht.

They found that glucose was taking longer to get into the nucleus accumbens of obesity-prone animals.

Furthermore, when measuring the concentration of the glutamate, glutamine, and GABA, they discovered excess levels of glutamate, an excitatory neurotransmitter. This, said the team, implied a defect in a neurotransmitter recycling process, typically maintained in the nervous system by star-shaped cells called astrocytes.

Normally, astrocytes will pull glutamate out of the space between neurons, called the synapse, convert it into glutamine, and then shuttle it back to cells that produce GABA or glutamate. This sequence is crucial for turning neurons off and on. "The findings suggest that we're getting too much glutamate and it's not being taken out of the synapse," said Vollbrecht.

Ferrario added, "The balance between glutamate and GABA (the main inhibitory transmitter) is really important for brain function and will influence activity of the neurons in the nucleus accumbens."

This balance, and therefore brain activity, is different in obesity-prone vs. obesity-resistant rats.

The fact that these rats are either prone to obesity or not is important for disentangling cause and effect, says Vollbrecht. "It allows us to remove diet as one of the variables."

Read more at Science Daily

From couch to ultra-marathon – mental imagery technique can aid running challenge completion

A motivational intervention known as functional imagery training (FIT) can help self-professed non-runners to complete an ultra-marathon (50km plus), according to new research.

The study, led by the University of Plymouth, started by examining the motivation of 31 non-runners who wanted to get fitter, by giving them a recognised behaviour change technique, often used by counsellors, known as Motivational Interviewing (MI). Participants were then left for five months to do whatever they presumed would benefit their fitness and health. After this period all were contacted and asked if they would consider completing an ultra-marathon.

Fifteen participants went on to express an interest in attempting an ultra-marathon as they continued to improve their fitness. Seven were randomly assigned FIT, while eight continued with just MI.

MI is a technique that sees a counsellor support someone to develop, highlight and verbalise their need or motivation for change, and their reasons for wanting to change. Functional Imagery Training builds on MI, as it teaches clients how to elicit and practice motivational imagery themselves, with participants encouraged to utilise all their senses to visualise how it would feel to achieve their goal.

Of the eight participants in the MI only group, four started the race, and two finished. Meanwhile, all seven of the FIT group started, and six finished -- showing that those assigned to the technique were five times more likely to complete the challenge.

While researchers acknowledge the small population size, the study, published in the Journal of Imagery Research in Sport and Physical Activity, adds to the growing body of evidence that FIT can significantly reinforce a person's motivation to complete a challenging goal.

FIT has also previously been shown to boost weight loss, with another Plymouth-led study showing that overweight people who used it lost an average of five times more weight than those using motivational interviewing alone.

Developed at the University of Plymouth, FIT is a unique approach to behaviour change that uses mental imagery to motivate change.

It teaches people new ways of thinking about their immediate future to help them stay motivated as they achieve each small step towards their goal -- with users describing it as a 'mindset shift', where they exercised because they wanted to, rather than feeling they had to.

For example, at difficult points in the race, a participant in the new study -- a teacher -- pictured the conversation they would have at work on Monday morning: visualising the staff room, holding a coffee in hand, imagining the smell and the taste, talking about the challenges with a peer, then using this self-developed image to imagine the feeling at the finish line, as they go from little exercise to ultra-athlete.

Lead author Dr Jon Rhodes, who has worked with professional athletes to improve their resilience, said: "An ultra-marathon requires a huge amount of mental, as well as physical, strength -- even from people who run regularly. Trialling Functional Imagery Training on historically self-professed non-runners was a real test of its efficacy, and to see that it made a difference was a promising finding.

"It shows that multi-sensory imagery is the key difference between those who reach the starting line and then go on to finish, and those who do not -- showing it is critical to maintaining changes and pushing the boundaries of physical and mental performance."

Melissa King, aged 38 from Newquay in Cornwall, was allocated the FIT intervention as she prepared for the Exmoor Ultra Marathon earlier this year. Completing the event successfully, she said: "My honest reason for starting running in the first place was just to see if I could do it.

"FIT has been a huge mindset shift, and I now use imagery as a way to reset; to ground me and help me to focus on being present, appreciating why I am here while also prompting me to cue my immediate, medium- and long-term goals. Having never run before, my ultimate aim is now a running trip in South America which will take a couple of months. It's in three years' time, so I'm starting training early and really embracing the challenge."

Read more at Science Daily

To do or not to do: Cracking the code of motivation

Our motivation to put effort for achieving a goal is controlled by a reward system wired in the brain. However, many neuropathological conditions impair the reward system, diminishing the will to work. Recently, scientists in Japan experimentally manipulated the reward system network of monkeys and studied their behavior. They deciphered a few critical missing pieces of the reward system puzzle that might help in increasing motivation.

Why do we do things? What persuades us to put an effort to achieve goals, however mundane? What, for instance, drives us to search for food? Neurologically, the answer is hidden in the reward system of the brain -- an evolutionary mechanism that controls our willingness to work or to take a risk as the cost of achieving our goals and enjoying the perceived rewards. In people suffering from depression, schizophrenia, or Parkinson's disease, often the reward system of the brain is impaired, leading them to a state of diminished motivation for work or chronic fatigue.

To find a way to overcome the debilitating behavioral blocks, neuroscientists are investigating the "anatomy" of the reward system and determining how it evaluates the cost-benefit trade-off while deciding on whether to pursue a task. Recently, Dr. Yukiko Hori of National Institutes for Quantum and Radiological Science and Technology, Japan, along with her colleagues have conducted a study that has answered some of the most critical questions on benefit- and cost-based motivation of reward systems. The findings of their study have been published in PLoS Biology.

Discussing what prompted them to undertake the study, Dr. Hori explains: "Mental responses such as 'feeling more costly and being too lazy to act,' are often a problem in patients with mental disorders such as depression, and the solution lies in the better understanding of what causes such responses. We wanted to look deeper into the mechanism of motivational disturbances in the brain."

To do so, Dr. Hori and her team focused on dopamine (DA), the "neurotransmitter" or the signaling molecule that plays the central role in inducing motivation and regulation of behavior based on cost-benefit analysis. The effect of DA in the brain transmits via DA receptors, or molecular anchors that bind the DA molecules and propagate the signals through the neuronal network of the brain. However, as these receptors have distinct roles in DA signal transduction, it was imperative to assess their relative impacts on DA signaling. Therefore, using macaque monkeys as models, the researchers aimed to decipher the roles of two classes of DA receptors -- the D1-like receptor (D1R) and the D2-like receptor (D2R) -- in developing benefit- and cost-based motivation.

In their study, the researchers first trained the animals to perform "reward size" tasks and "work/delay tasks." These tasks allowed them to measure how perceived reward size and required effort influenced the task-performing behavior. Dr. Takafumi Minamimoto, the corresponding author of the study explains, "We systematically manipulated the D1R and D2R of these monkeys by injecting them with specific receptor-binding molecules that dampened their biological responses to DA signaling. By positron emission tomography-based imaging of the brains of the animals, the extent of bindings or blockades of the receptors was measured." Then, under experimental conditions, they offered the monkeys the chance to perform tasks to achieve rewards and noted whether the monkeys accepted or refused to perform the tasks and how quickly they responded to the cues related to the tasks.

Analysis of these data unearthed some intriguing insights into the neurobiological mechanism of the decision-making process. The researchers observed that decision-making based on perceived benefit and cost required the involvement of both D1R and D2R, in both incentivizing the motivation (the process in which the size of the rewards inspired the monkeys to perform the tasks) and in increasing delay discounting (the tendency to prefer immediate, smaller rewards over larger, but delayed rewards). It also became clear that DA transmission via D1R and D2R regulates the cost-based motivational process by distinct neurobiological processes for benefits or "reward availability" and costs or "energy expenditure associated with the task." However, workload discounting -- the process of discounting the value of the rewards based on the proportion of the effort needed -- was exclusively related to D2R manipulation.

Read more at Science Daily

Just enough information will motivate young children to learn, drive curiosity

Preschool children are sensitive to the gap between how much they know and how much there is to learn, according to a Rutgers University-New Brunswick study.

The research, published in the journal Psychological Science, found preschool children are more likely to choose to gather more information about something if they know just enough about it to find it interesting, but not too much that it becomes boring.

Researchers say this "optimal" amount of existing knowledge creates the perfect mix of uncertainty and curiosity in children and motivates them to learn more.

"There is an infinite amount of information in the real world," said lead author Jenny Wang, an assistant professor of cognitive psychology at Rutgers. "Yet despite having to learn so much in such a short amount of time, young children seem to learn happily and effectively. We wanted to understand what drives their curiosity."

The study focused on how children's knowledge level influences what information they find interesting. The findings suggest that children are not simply attracted to information by its novelty.

According to Wang, children are naturally curious but the difficult question is how to harness this natural curiosity.

"Ultimately, findings like this will help parents and educators better support children when they actively explore and learn about the world," Wang said.

In a series of experiments, Wang and her coauthors designed in-person and online storybooks to measure how much 3- to 5-year-old preschool children know about different "knowledge domains." The experiment also assessed their ability to understand and comprehend a specific topic, such as contagion, and asked how children's current knowledge level predicts their interest in learning more about it, including whether someone will get sick after playing with a sneezing friend.

"Intuitively, curiosity seems to belong to those who know the most, like scientists, and those who know the least, like babies," said Wang, who directs the Rutgers Cognition and Learning Center (CALC). "But what we found here is quite surprising: it was children in the middle who showed the most interest in learning more about contagion, compared to children who knew too little or too much."

Read more at Science Daily

Study helps explain why motivation to learn declines with age

 As people age, they often lose their motivation to learn new things or engage in everyday activities. In a study of mice, MIT neuroscientists have now identified a brain circuit that is critical for maintaining this kind of motivation.

This circuit is particularly important for learning to make decisions that require evaluating the cost and reward that come with a particular action. The researchers showed that they could boost older mice's motivation to engage in this type of learning by reactivating this circuit, and they could also decrease motivation by suppressing the circuit.

"As we age, it's harder to have a get-up-and-go attitude toward things," says Ann Graybiel, an Institute Professor at MIT and member of the McGovern Institute for Brain Research. "This get-up-and-go, or engagement, is important for our social well-being and for learning -- it's tough to learn if you aren't attending and engaged."

Graybiel is the senior author of the study, which appears today in Cell. The paper's lead authors are Alexander Friedman, a former MIT research scientist who is now an assistant professor at the University of Texas at El Paso, and Emily Hueske, an MIT research scientist.

Evaluating cost and benefit

The striatum is part of the basal ganglia -- a collection of brain centers linked to habit formation, control of voluntary movement, emotion, and addiction. For several decades, Graybiel's lab has been studying clusters of cells called striosomes, which are distributed throughout the striatum. Graybiel discovered striosomes many years ago, but their function had remained mysterious, in part because they are so small and deep within the brain that it is difficult to image them with functional magnetic resonance imaging (fMRI).

In recent years, Friedman, Graybiel, and colleagues including MIT research fellow Ken-ichi Amemori have discovered that striosomes play an important role in a type of decision-making known as approach-avoidance conflict. These decisions involve choosing whether to take the good with the bad -- or to avoid both -- when given options that have both positive and negative elements. An example of this kind of decision is having to choose whether to take a job that pays more but forces a move away from family and friends. Such decisions often provoke great anxiety.

In a related study, Graybiel's lab found that striosomes connect to cells of the substantia nigra, one of the brain's major dopamine-producing centers. These studies led the researchers to hypothesize that striosomes may be acting as a gatekeeper that absorbs sensory and emotional information coming from the cortex and integrates it to produce a decision on how to act. These actions can then be invigorated by the dopamine-producing cells.

The researchers later discovered that chronic stress has a major impact on this circuit and on this kind of emotional decision-making. In a 2017 study performed in rats and mice, they showed that stressed animals were far more likely to choose high-risk, high-payoff options, but that they could block this effect by manipulating the circuit.

In the new Cell study, the researchers set out to investigate what happens in striosomes as mice learn how to make these kinds of decisions. To do that, they measured and analyzed the activity of striosomes as mice learned to choose between positive and negative outcomes.

During the experiments, the mice heard two different tones, one of which was accompanied by a reward (sugar water), and another that was paired with a mildly aversive stimulus (bright light). The mice gradually learned that if they licked a spout more when they heard the first tone, they would get more of the sugar water, and if they licked less during the second, the light would not be as bright.

Learning to perform this kind of task requires assigning value to each cost and each reward. The researchers found that as the mice learned the task, striosomes showed higher activity than other parts of the striatum, and that this activity correlated with the mice's behavioral responses to both of the tones. This suggests that striosomes could be critical for assigning subjective value to a particular outcome.

"In order to survive, in order to do whatever you are doing, you constantly need to be able to learn. You need to learn what is good for you, and what is bad for you," Friedman says.

"A person, or this case a mouse, may value a reward so highly that the risk of experiencing a possible cost is overwhelmed, while another may wish to avoid the cost to the exclusion of all rewards. And these may result in reward-driven learning in some and cost-driven learning in others," Hueske says.

The researchers found that inhibitory neurons that relay signals from the prefrontal cortex help striosomes to enhance their signal-to-noise ratio, which helps to generate the strong signals that are seen when the mice evaluate a high-cost or high-reward option.

Loss of motivation

Next, the researchers found that in older mice (between 13 and 21 months, roughly equivalent to people in their 60s and older), the mice's engagement in learning this type of cost-benefit analysis went down. At the same time, their striosomal activity declined compared to that of younger mice. The researchers found a similar loss of motivation in a mouse model of Huntington's disease, a neurodegenerative disorder that affects the striatum and its striosomes.

When the researchers used genetically targeted drugs to boost activity in the striosomes, they found that the mice became more engaged in performance of the task. Conversely, suppressing striosomal activity led to disengagement.

In addition to normal age-related decline, many mental health disorders can skew the ability to evaluate the costs and rewards of an action, from anxiety and depression to conditions such as PTSD. For example, a depressed person may undervalue potentially rewarding experiences, while someone suffering from addiction may overvalue drugs but undervalue things like their job or their family.

The researchers are now working on possible drug treatments that could stimulate this circuit, and they suggest that training patients to enhance activity in this circuit through biofeedback could offer another potential way to improve their cost-benefit evaluations.

"If you could pinpoint a mechanism which is underlying the subjective evaluation of reward and cost, and use a modern technique that could manipulate it, either psychiatrically or with biofeedback, patients may be able to activate their circuits correctly," Friedman says.

Read more at Science Daily