Learning with all your senses: Multimodal enrichment as the optimal learning strategy of the future

Many educational approaches assume that integrating complementary sensory and motor information into the learning experience can enhance learning, for example gestures help in learning new vocabulary in foreign language classes. In her recent publication, neuroscientist Katharina von Kriegstein from Technische Universität Dresden and Brian Mathias of the University of Aberdeen summarize these methods under the term "multimodal enrichment." This means enrichment with multiple senses and movement. Numerous current scientific studies prove that multimodal enrichment can enhance learning outcomes. Experiments in classrooms show similar results.

In the review article, the two researchers compare these findings with cognitive, neuroscience, and computational theories of multimodal enrichment. Recent neuroscience research has found that the positive effects of enriched learning are associated with response in brain regions that serve perception and motor function. For example, hearing a recently learned foreign language word, may elicit activity in motor brain regions if the word was associated with the performance of a congruent gesture during learning. These brain responses are causal to the benefits of multimodal enrichment for learning outcome. Computer algorithms confirm this hypothesis.

"The brain is optimized for learning with all the senses and with movement. Brain structures for perception and motor skills work together to promote this type of learning. We hope that our deeper understanding of the brain's learning mechanisms, will facilitate the development of optimal learning strategies in the future," explains Brian Mathias.

Katharina von Kriegstein adds, "The results of the literature we reviewed contribute to our understanding of why several long-used learning strategies, such as parts of the Montessori method, are effective. They also provide clear clues as to why some approaches are not as effective. Recently uncovered neuroscientific mechanisms may inspire the updating of cognitive and computational theories of learning, providing new hypotheses about learning. We anticipate that such an interdisciplinary and evidence-based approach will lead to the optimization of learning and teaching strategies in the future, for both humans and artificial systems."

Read more at Science Daily

Improving reading skills through action video games

Decoding letters into sound is a key point in learning to read but is not enough to master it. "Reading calls upon several other essential mechanisms that we don't necessarily think about, such as knowing how to move our eyes on the page or how to use our working memory to link words together in a coherent sentence," points out Daphné Bavelier, a professor in the Psychology Section of the Faculty of Psychology and Educational Sciences (FPSE) at the UNIGE. "These other skills, such as vision, the deployment of attention, working memory, and cognitive flexibility, are known to be improved by action video games," explains Angela Pasqualotto, first author of this study, which is based on her PhD thesis at the Department of Psychology and Cognitive Science of the University of Trento under the direction of Professors Venuti and De Angeli.

A child-friendly action video game to support learning

With this in mind, a video game was designed that combines action video games with mini games that train different executive functions, such as working memory, inhibition and cognitive flexibility, functions that are called upon during reading. "The universe of this game is an alternative world in which the child, accompanied by his Raku, a flying creature, must carry out different missions to save planets and progress in the game," Angela Pasqualotto adds. The idea is to reproduce the components of an action game, without incorporating violence, so that it is suitable for young children. "For example, the Raku flies through a meteor shower, moving around to avoid those or aiming at them to weaken their impact, while collecting useful resources for the rest of the game, a bit like what you find in action video games."

The scientists then worked with 150 Italian schoolchildren aged 8 to 12, divided into two groups: the first one played the video game developed by the team, and the second one played Scratch, a game that teaches children how to code. Both games require attentional control and executive functions, but in different manners. The action video game requires children to perform tasks within a time limit such as remembering a sequence of symbols or responding only when the Raku makes a specific sound while increasing the difficulty of these tasks according to the child's performance. Scratch, the control game, requires planning, reasoning and problem solving. Children must manipulate objects and logical structures to establish the desired programming sequence.

"First, we tested the children's ability to read words, non-words and paragraphs, and also we conducted an attention test that measures the child's attentional control, a capacity we know is trained by action video games," explains Daphne Bavelier. The children then followed the training with either the action video game or the control game, for six weeks, two hours a week under supervision at school. Children were tested at school by clinicians of the Laboratory of Observation Diagnosis and Education (UNITN).

Long-term improvement in reading skills

Shortly after the end of the training, the scientists repeated the tests on both groups of children. "We found a 7-fold improvement in attentional control in the children who played the action video game compared to the control group," says Angela Pasqualotto. Even more remarkably, the research team observed a clear enhancement in reading, not only in terms of reading speed, but also in accuracy, whereas no improvement was noted for the control group. This improvement in literacy occurs even though the action video game does not require any reading activity.

"What is particularly interesting about this study is that we carried out three further assessment tests at 6 months, 12 months and 18 months after training. On each occasion, the trained children performed better than the control group, which proves that these improvements were sustained," Angela Pasqualotto says. Moreover, the grades in Italian of the trained children became significantly better over time, showing a virtuous improvement in learning ability. "The effects are thus long-term, in line with the action video game strengthening the ability to learn how to learn," says Daphne Bavelier.

Read more at Science Daily

Insights into how a stroke affects reading could help with rehabilitation

Georgetown University researchers, looking at the ability of people to sound out words after a stroke, found that knowing which region of the brain was impacted by the stroke could have important implications for helping target rehabilitation efforts.

The finding appeared August 30, 2021, in Brain Communications.

"One in five stroke survivors in the United States live with persistent language impairment. Most of these people also struggle with reading," says the study's first author, J. Vivian Dickens, PhD, a Georgetown University MD/PhD student conducting research in the university's Cognitive Recovery Lab and Center for Aphasia Research and Rehabilitation at Georgetown's Medical Center. "Our study clarifies the neuroanatomical and cognitive bases of post-stroke reading and language deficits, which could help facilitate predictions of deficits in stroke survivors and suggest targeted treatments."

The research focus was on phonological processing, which is understanding and being able to use the sounds that comprise language. There are three principal aspects to this processing: auditory, or the ability to recognize the sounds of words, such as judging if words rhyme; motor, which is the ability to produce accurate and clear speech; and auditory-motor translation, which is the translation of sounds heard into speech.

"The goal of this study was to understand how post-stroke difficulties with the three different aspects of phonology relate to difficulties with reading," says Dickens. "There are two broad ways that people read words: one involves sounding out words, which is particularly important for reading new words; the other involves whole-word recognition. People with post-stroke language impairment frequently have specific trouble sounding out words."

The investigators tested reading and phonological abilities in 67 people, 30 of whom had had a stroke and 37 that had not. Advanced MRI techniques allowed the researchers to trace out white matter connections, which are akin to wiring diagrams for the brain, as well as map out stroke locations in the brains of affected study participants.

"We found two different patterns of reading problems. Strokes involving the left frontal lobe caused problems with motor phonology and one of the two ways of reading, specifically sounding out words. In contrast, strokes involving the left temporal and parietal lobes caused problems with auditory-motor translation and both ways of reading," says Dickens. "These results may help clinicians develop therapies focused on specific reading problems that individual stroke survivors often struggle with."

"This study focused on reading aloud single words, a classic measure of reading ability," says Peter E. Turkeltaub, MD, PhD, director of the Cognitive Recovery Lab in the Center for Brain Plasticity and Recovery, medical director in the Center for Aphasia Research and Rehabilitation and senior author of the article. "Our results are an important step forward in revealing the mechanisms of translating print to sound, which is crucial for developing rehabilitative therapies for patients who have had strokes."

The investigators are planning studies to help confirm the extent to which these findings can be generalized to silent reading, which relies on the same core psychological processes as oral reading and is more important for reading in daily life. The researchers are also hoping to turn their research tasks into useful clinical tests to diagnose phonological processing.

Read more at Science Daily

Think about this: Keeping your brain active may delay Alzheimer's dementia 5 years

Keeping your brain active in old age has always been a smart idea, but a new study suggests that reading, writing letters and playing card games or puzzles in later life may delay the onset of Alzheimer's dementia by up to five years. The research is published in the July 14, 2021, online issue of Neurology, the medical journal of the American Academy of Neurology.

"The good news is that it's never too late to start doing the kinds of inexpensive, accessible activities we looked at in our study," said study author Robert S. Wilson, PhD, of Rush University Medical Center in Chicago. "Our findings suggest it may be beneficial to start doing these things, even in your 80s, to delay the onset of Alzheimer's dementia."

The study looked at 1,978 people with an average age of 80 who did not have dementia at the start of the study. The people were followed for an average of seven years. To determine if they had developed dementia, participants were given annual examinations, which included a number of cognitive tests.

When the study began, people rated their participation in seven activities on a five-point scale. The questions included: "During the past year, how often did you read books?" and "During the past year, how often did you play games like checkers, board games, cards or puzzles?"

Participants also answered questions about cognitive activity in childhood, adulthood and middle age.

Researchers then averaged each person's responses, with a score of one meaning once a year or less and score of five meaning every day or almost every day. People in the group with high cognitive activity scored an average of 4.0 which meant activities several times per week, compared to an average score of 2.1 for those with low cognitive activity, which meant activities several times per year.

During the study follow-up period, 457 people with an average age of 89 were diagnosed with Alzheimer's dementia. People with the highest levels of activity, on average, developed dementia at age 94. The people with the lowest cognitive activity, on average, developed dementia at age 89, a difference of five years. The results were similar when researchers adjusted for other factors that could affect dementia risk, such as education level and sex.

To test the idea that low cognitive activity may be an early sign of dementia, not the other way around, researchers also looked at the brains of 695 people who died during the study. Brain tissue was examined for markers of Alzheimer's like amyloid and tau protein deposits, but researchers found no association between how active they were cognitively and markers of Alzheimer's disease and related disorders in their brains.

"Our study shows that people who engage in more cognitively stimulating activities may be delaying the age at which they develop dementia," Wilson said. "It is important to note, after we accounted for late life level of cognitive activity, neither education nor early life cognitive activity were associated with the age at which a person developed Alzheimer's dementia. Our research suggests that the link between cognitive activity and the age at which a person developed dementia is mainly driven by the activities you do later in life."

Read more at Science Daily

Key brain region was 'recycled' as humans developed the ability to read

Humans began to develop systems of reading and writing only within the past few thousand years. Our reading abilities set us apart from other animal species, but a few thousand years is much too short a timeframe for our brains to have evolved new areas specifically devoted to reading.

To account for the development of this skill, some scientists have hypothesized that parts of the brain that originally evolved for other purposes have been "recycled" for reading. As one example, they suggest that a part of the visual system that is specialized to perform object recognition has been repurposed for a key component of reading called orthographic processing -- the ability to recognize written letters and words.

A new study from MIT neuroscientists offers evidence for this hypothesis. The findings suggest that even in nonhuman primates, who do not know how to read, a part of the brain called the inferotemporal (IT) cortex is capable of performing tasks such as distinguishing words from nonsense words, or picking out specific letters from a word.

"This work has opened up a potential linkage between our rapidly developing understanding of the neural mechanisms of visual processing and an important primate behavior -- human reading," says James DiCarlo, the head of MIT's Department of Brain and Cognitive Sciences, an investigator in the McGovern Institute for Brain Research and the Center for Brains, Minds, and Machines, and the senior author of the study.

Rishi Rajalingham, an MIT postdoc,, is the lead author of the study, which appears today in Nature Communications. Other MIT authors are postdoc Kohitij Kar and technical associate Sachi Sanghavi. The research team also includes Stanislas Dehaene, a professor of experimental cognitive psychology at the Collège de France.

Word recognition

Reading is a complex process that requires recognizing words, assigning meaning to those words, and associating words with their corresponding sound. These functions are believed to be spread out over different parts of the human brain.

Functional magnetic resonance imaging (fMRI) studies have identified a region called the visual word form area (VWFA) that lights up when the brain processes a written word. This region is involved in the orthographic stage: It discriminates words from jumbled strings of letters or words from unknown alphabets. The VWFA is located in the IT cortex, a part of the visual cortex that is also responsible for identifying objects.

DiCarlo and Dehaene became interested in studying the neural mechanisms behind word recognition after cognitive psychologists in France reported that baboons could learn to discriminate words from nonwords, in a study that appeared in Science in 2012.

Using fMRI, Dehaene's lab has previously found that parts of the IT cortex that respond to objects and faces become highly specialized for recognizing written words once people learn to read.

"However, given the limitations of human imaging methods, it has been challenging to characterize these representations at the resolution of individual neurons, and to quantitatively test if and how these representations might be reused to support orthographic processing," Dehaene says. "These findings inspired us to ask if nonhuman primates could provide a unique opportunity to investigate the neuronal mechanisms underlying orthographic processing."

The researchers hypothesized that if parts of the primate brain are predisposed to process text, they might be able to find patterns reflecting that in the neural activity of nonhuman primates as they simply look at words.

To test that idea, the researchers recorded neural activity from about 500 neural sites across the IT cortex of macaques as they looked at about 2,000 strings of letters, some of which were English words and some of which were nonsensical strings of letters.

"The efficiency of this methodology is that you don't need to train animals to do anything," Rajalingham says. "What you do is just record these patterns of neural activity as you flash an image in front of the animal."

The researchers then fed that neural data into a simple computer model called a linear classifier. This model learns to combine the inputs from each of the 500 neural sites to predict whether the string of letters that provoked that activity pattern was a word or not. While the animal itself is not performing this task, the model acts as a "stand-in" that uses the neural data to generate a behavior, Rajalingham says.

Using that neural data, the model was able to generate accurate predictions for many orthographic tasks, including distinguishing words from nonwords and determining if a particular letter is present in a string of words. The model was about 70 percent accurate at distinguishing words from nonwords, which is very similar to the rate reported in the 2012 Science study with baboons. Furthermore, the patterns of errors made by model were similar to those made by the animals.

Neuronal recycling

The researchers also recorded neural activity from a different brain area that also feeds into IT cortex: V4, which is part of the visual cortex. When they fed V4 activity patterns into the linear classifier model, the model poorly predicted (compared to IT) the human or baboon performance on the orthographic processing tasks.

The findings suggest that the IT cortex is particularly well-suited to be repurposed for skills that are needed for reading, and they support the hypothesis that some of the mechanisms of reading are built upon highly evolved mechanisms for object recognition, the researchers say.

Read more at Science Daily