When we feel things that are not there

Virtual reality (VR) is not only a technology for games and entertainment, but also has potential in science and medicine. Researchers at Ruhr University Bochum, Germany, have now gained new insights into human perception with the help of VR. They used virtual reality scenarios in which subjects touched their own bodies with a virtual object. To the researchers' surprise, this led to a tingling sensation at the spot where the avatarized body was touched. This effect occurred even though there was no real physical contact between the virtual object and the body. The scientists led by Dr. Artur Pilacinski and Professor Christian Klaes from the Department of Neurotechnology describe this phenomenon as a phantom touch illusion. They published their results in the journal Scientific Reports of the Nature Publishing Group in September 2023.

"People in virtual reality sometimes have the feeling that they are touching things, although they are actually only encountering virtual objects," says first author Artur Pilacinski from the Knappschaftskrankenhaus Bochum Langendreer, University Clinic of Ruhr University Bochum, explaining the origin of the research question. "We show that the phantom touch illusion is described by most subjects as a tingling or prickling, electrifying sensation or as if the wind was passing through their hand."

Body sensation arises from complex combination of different sensory perceptions

The neuroscientists wanted to understand what is behind this phenomenon and find out which processes in the brain and body play a role in it. They observed that the phantom touch illusion also occurred when the subjects touched parts of their bodies that were not visible in virtual reality. Second author Marita Metzler adds: "This suggests that human perception and body sensation are not only based on vision, but on a complex combination of many sensory perceptions and the internal representation of our body."

This study involved 36 volunteers wearing VR glasses. First, they got used to the VR environment by moving around and touching virtual objects. Then they were given the task of touching their hand in the virtual environment with a virtual stick.

Comparison between virtual and suggested touch sensations

Participants were asked if they felt anything. If not, they were allowed to continue touching and the question was asked again later. If they felt sensations, they were asked to describe them and rate their intensity on different hand locations. This process was repeated for both hands. There was a consistent reporting of the sensation as "tingling" by a majority of participants.

In a control experiment, it was investigated whether similar sensations could also be perceived without visual contact with virtual objects purely due to experimental situation demands. Here, a small laser pointer was used instead of virtual objects to touch the hand. This control experiment did not result in phantom touch suggesting that phantom touch illusion was unique to virtual touch.

The discovery of the phantom touch illusion opens up new possibilities for further research into human perception and could also be applied in the fields of virtual reality and medicine. Christian Klaes, member of the Research Department of Neuroscience at Ruhr University, says: "It could even help to deepen the understanding of neurological diseases and disorders that affect the perception of one's own body."

Further collaboration with the University of Sussex

The Bochum team plans to continue their research on the phantom touch illusion and the underlying processes. For this reason, a collaboration with the University of Sussex has been started. "It is important to first distinguish between the actual sensations of phantom touch and other cognitive processes that may be involved in reporting such embodied sensations, such as suggestion, or experimental situation demands," says Artur Pilacinski. "We also want to further explore and understand the neural basis of the phantom touch illusion in collaboration with other partners."

Read more at Science Daily

Machine learning gives users 'superhuman' ability to open and control tools in virtual reality

Researchers have developed a virtual reality application where a range of 3D modelling tools can be opened and controlled using just the movement of a user's hand.

The researchers, from the University of Cambridge, used machine learning to develop 'HotGestures' -- analogous to the hot keys used in many desktop applications.

HotGestures give users the ability to build figures and shapes in virtual reality without ever having to interact with a menu, helping them stay focused on a task without breaking their train of thought.

The idea of being able to open and control tools in virtual reality has been a movie trope for decades, but the researchers say that this is the first time such a 'superhuman' ability has been made possible. The results are reported in the journal IEEE Transactions on Visualization and Computer Graphics.

Virtual reality (VR) and related applications have been touted as game-changers for years, but outside of gaming, their promise has not fully materialised. "Users gain some qualities when using VR, but very few people want to use it for an extended period of time," said Professor Per Ola Kristensson from Cambridge's Department of Engineering, who led the research. "Beyond the visual fatigue and ergonomic issues, VR isn't really offering anything you can't get in the real world."

Most users of desktop software will be familiar with the concept of hot keys -- command shortcuts such as ctrl-c to copy and ctrl-v to paste. While these shortcuts omit the need to open a menu to find the right tool or command, they rely on the user having the correct command memorised.

"We wanted to take the concept of hot keys and turn it into something more meaningful for virtual reality -- something that wouldn't rely on the user having a shortcut in their head already," said Kristensson, who is also co-Director of the Centre for Human-Inspired Artificial Intelligence.

Instead of hot keys, Kristensson and his colleagues developed 'HotGestures', where users perform a gesture with their hand to open and control the tool they need in 3D virtual reality environments.

For example, performing a cutting motion opens the scissor tool, and the spray motion opens the spray can tool. There is no need for the user to open a menu to find the tool they need, or to remember a specific shortcut. Users can seamlessly switch between different tools by performing different gestures during a task, without having to pause their work to browse a menu or to press a button on a controller or keyboard.

"We all communicate using our hands in the real world, so it made sense to extend this form of communication to the virtual world," said Kristensson.

For the study, the researchers built a neural network gesture recognition system that can recognise gestures by performing predictions on an incoming hand joint data stream. The system was built to recognise ten different gestures associated with building 3D models: pen, cube, cylinder, sphere, palette, spray, cut, scale, duplicate and delete.

The team carried out two small studies where participants used HotGestures, menu commands or a combination. The gesture-based technique provided fast and effective shortcuts for tool selection and usage. Participants found HotGestures to be distinctive, fast, and easy to use while also complementing conventional menu-based interaction. The researchers designed the system so that there were no false activations -- the gesture-based system was able to correctly recognise what was a command and what was normal hand movement. Overall, the gesture-based system was faster than a menu-based system.

"There is no VR system currently available that can do this," said Kristensson. "If using VR is just like using a keyboard and a mouse, then what's the point of using it? It needs to give you almost superhuman powers that you can't get elsewhere."

The researchers have made the source code and dataset publicly available so that designers of VR applications can incorporate it into their products.

"We want this to be a standard way of interacting with VR," said Kristensson. "We've had the tired old metaphor of the filing cabinet for decades. We need new ways of interacting with technology, and we think this is a step in that direction. When done right, VR can be like magic."

Read more at Science Daily

Turning old maps into 3D digital models of lost neighborhoods

Imagine strapping on a virtual reality headset and "walking" through a long-gone neighborhood in your city -- seeing the streets and buildings as they appeared decades ago.

That's a very real possibility now that researchers have developed a method to create 3D digital models of historic neighborhoods using machine learning and historic Sanborn Fire Insurance maps.

But the digital models will be more than just a novelty -- they will give researchers a resource to conduct studies that would have been nearly impossible before, such as estimating the economic loss caused by the demolition of historic neighborhoods.

"The story here is we now have the ability to unlock the wealth of data that is embedded in these Sanborn fire atlases," said Harvey Miller, co-author of the study and professor of geography at The Ohio State University.

"It enables a whole new approach to urban historical research that we could never have imagined before machine learning. It is a game changer."

The study was published today (June 28, 2023) in the journal PLOS ONE.

This research begins with the Sanborn maps, which were created to allow fire insurance companies to assess their liability in about 12,000 cities and towns in the United States during the 19th and 20th centuries. In larger cities, they were often updated regularly, said Miller, who is director of Ohio State's Center for Urban and Regional Analysis (CURA).

The problem for researchers was that trying to manually collect usable data from these maps was tedious and time-consuming -- at least until the maps were digitized. Digital versions are now available from the Library of Congress.

Study co-author Yue Lin, a doctoral student in geography at Ohio State, developed machine learning tools that can extract details about individual buildings from the maps, including their locations and footprints, the number of floors, their construction materials and their primary use, such as dwelling or business.

"We are able to get a very good idea of what the buildings look like from data we get from the Sanborn maps," Lin said.

The researchers tested their machine learning technique on two adjacent neighborhoods on the near east side of Columbus, Ohio, that were largely destroyed in the 1960s to make way for the construction of I-70.

One of the neighborhoods, Hanford Village, was developed in 1946 to house returning Black veterans of World War II.

"The GI bill gave returning veterans funds to purchase homes, but they could only be used on new builds," said study co-author Gerika Logan, outreach coordinator of CURA. "So most of the homes were lost to the highway not long after they were built."

The other neighborhood in the study was Driving Park, which also housed a thriving Black community until I-70 split it in two.

The researchers used 13 Sanborn maps for the two neighborhoods produced in 1961, just before I-70 was built. Machine learning techniques were able to extract the data from the maps and create digital models.

Comparing data from the Sanford maps to today showed that a total of 380 buildings were demolished in the two neighborhoods for the highway, including 286 houses, 86 garages, five apartments and three stores.

Analysis of the results showed that the machine learning model was very accurate in recreating the information contained in the maps -- about 90% accurate for building footprints and construction materials.

"The accuracy was impressive. We can actually get a visual sense of what these neighborhoods looked like that wouldn't be possible in any other way," Miller said.

"We want to get to the point in this project where we can give people virtual reality headsets and let them walk down the street as it was in 1960 or 1940 or perhaps even 1881."

Using the machine learning techniques developed for this study, researchers could develop similar 3D models for nearly any of the 12,000 cities and towns that have Sanborn maps, Miller said.

This will allow researchers to re-create neighborhoods lost to natural disasters like floods, as well as urban renewal, depopulation and other types of change.

Because the Sanborn maps include information on businesses that occupied specific buildings, researchers could re-create digital neighborhoods to determine the economic impact of losing them to urban renewal or other factors. Another possibility would be to study how replacing homes with highways that absorbed the sun's heat affected the urban heat island effect.

"There's a lot of different types of research that can be done. This will be a tremendous resource for urban historians and a variety of other researchers," Miller said.

Read more at Science Daily

Smelling in VR environment possible with new gaming technology

An odor machine, so-called olfactometer, makes it possible to smell in VR environments. First up is a "wine tasting game" where the user smells wine in a virtual wine cellar and gets points if the guess on aromas in each wine is correct. The new technology that can be printed on 3D printers has been developed in collaboration between Stockholm University and Malmö University. The research, funded by the Marianne and Marcus Wallenberg Foundation, was recently published in the International Journal of Human -- Computer Studies.

"We hope that the new technical possibilities will lead to scents having a more important role in game development, says Jonas Olofsson, professor of psychology and leader of the research project at Stockholm University.

In the past, computer games have focused mostly on what we can see -- moving images on screens. Other senses have not been present. But an interdisciplinary research group at Stockholm University and Malmö University has now constructed a scent machine that can be controlled by a gaming computer. In the game, the participant moves in a virtual wine cellar, picking up virtual wine glasses containing different types of wine, guessing the aromas. The small scent machine is attached to the VR system's controller, and when the player lifts the glass, it releases a scent.

"The possibility to move on from a passive to a more active sense of smell in the game world paves the way for the development of completely new smell-based game mechanics based on the players' movements and judgments," says Simon Niedenthal, interaction and game researcher at Malmö University.

The olfactometer consists of four different valves each connected to a channel. In the middle there is a fan sucking the air into a tube. With the help of the computer, the player can control the four channels so that they open to different degrees and provide different mixtures of scent. Scent blends that can mimic the complexity of a real wine glass. The game has different levels of difficulty with increasing levels of complexity.

"In the same way that a normal computer game becomes more difficult the better the player becomes; the scent game can also challenge players who already have a sensitive nose. This means that the scent machine can even be used to train wine tasters or perfumers," says Jonas Olofsson.

All code, blueprints and instructions for the machine are openly available online, as is code for the virtual wine tasting game. The research group, Sensory Cognitive Interaction Laboratory, which is located at the Department of Psychology, Stockholm University, now hopes that scented computer games can become useful for other purposes.

"For those who, for example, lost their sense of smell after COVID-19 or for other reasons, the new technology can mean an opportunity to regain their sense of smell with the help of game-based training," says Jonas Olofsson, research team leader.

Smell training is a method recommended by doctors for those who lose their sense of smell after colds and other viruses, but according to Jonas Olofsson, many people stop training because it becomes too boring.

"I hope that the fact that drawings and code are openly available as "open source" will lead to an opportunity for game companies to start creating new, commercial products for scent training using the new technology," says Jonas Olofsson.

According to Simon Niedenthal, "open source" leads to promoting accessibility, reproducibility and comparison of results in research. It also contributes to creating a cohesive research and design community within the game development field.

"But it also means that the costs of the equipment are greatly reduced, which makes it available to more people. To us that is important," says Simon Niedenthal.

"We believe in open science, that research results should be made available to the public and that other researchers should be able to repeat our results. With the help of our research, others can build scent machines and explore new ways of using scents in games," says Jonas Olofsson.

Read more at Science Daily

New chip-based beam steering device lays groundwork for smaller, cheaper lidar

Researchers have developed a new chip-based beam steering technology that provides a promising route to small, cost-effective and high-performance lidar (or light detection and ranging) systems. Lidar, which uses laser pulses to acquire 3D information about a scene or object, is used in a wide range of applications such as autonomous driving, free-space optical communications, 3D holography, biomedical sensing and virtual reality.

"Optical beam steering is a key technology for lidar systems, but conventional mechanical-based beam steering systems are bulky, expensive, sensitive to vibration and limited in speed," said research team leader Hao Hu from the Technical University of Denmark. "Although devices known as chip-based optical phased arrays (OPAs) can quickly and precisely steer light in a non-mechanical way, so far, these devices have had poor beam quality and a field of view typically below 100 degrees."

In Optica, Optica Publishing Group's journal for high-impact research, Hu and co-author Yong Liu describe their new chip-based OPA that solves many of the problems that have plagued OPAs. They show that the device can eliminate a key optical artifact known as aliasing, achieving beam steering over a large field of view while maintaining high beam quality, a combination that could greatly improve lidar systems.

"We believe our results are groundbreaking in the field of optical beam steering," said Hu. "This development lays the groundwork for OPA-based lidar that is low cost and compact, which would allow lidar to be widely used for a variety of applications such as high-level advanced driver-assistance systems that can assist in driving and parking and increase safety."

A new OPA design

OPAs perform beam steering by electronically controlling light's phase profile to form specific light patterns. Most OPAs use an array of waveguides to emit many beams of light and then interference is applied in far field (away from the emitter) to form the pattern. However, the fact that these waveguide emitters are typically spaced far apart from each other and generate multiple beams in the far field creates an optical artifact known as aliasing. To avoid the aliasing error and achieve a 180° field of view, the emitters need to be close together, but this causes strong crosstalk between adjacent emitters and degrades the beam quality. Thus, until now, there has been a trade-off between OPA field of view and beam quality.

To overcome this trade-off, the researchers designed a new type of OPA that replaces the multiple emitters of traditional OPAs with a slab grating to create a single emitter. This setup eliminates the aliasing error because the adjacent channels in the slab grating can be very close to each other. The coupling between the adjacent channels is not detrimental in the slab grating because it enables the interference and beam formation in the near field (close to the single emitter). The light can then be emitted to the far field with the desired angle. The researchers also applied additional optical techniques to lower the background noise and reduce other optical artifacts such as side lobes.

High quality and wide field of view

To test their new device, the researchers built a special imaging system to measure the average far-field optical power along the horizontal direction over a 180° field of view. They demonstrated aliasing-free beam steering in this direction, including steering beyond ±70°, although some beam degradation was seen.

They then characterized beam steering in the vertical direction by tuning the wavelength from 1480 nm to 1580 nm, achieving a 13.5° tuning range. Finally, they showed the versatility of the OPA by using it to form 2D images of the letters "D," "T" and "U" centered at the angles of -60°, 0° and 60° by tuning both the wavelength and the phase shifters. The experiments were performed with a beam width of 2.1°, which the researchers are now working to decrease to achieve beam steering with a higher resolution and a longer range.

Read more at Science Daily

Virtual reality app trial shown to reduce common phobias

Results from a University of Otago, Christchurch trial suggest fresh hope for the estimated one-in-twelve people worldwide suffering from a fear of flying, needles, heights, spiders and dogs.

The trial, led by Associate Professor Cameron Lacey, from the Department of Psychological Medicine, studied phobia patients using a headset and a smartphone app treatment programme -- a combination of Virtual Reality (VR) 360-degree video exposure therapy and cognitive behavioural therapy (CBT).

Participants downloaded a fully self-guided smartphone app called "oVRcome," developed by Christchurch tech entrepreneur Adam Hutchinson, aimed at treating patients with phobia and anxiety.

The app was paired with a headset to immerse participants in virtual environments to help treat their phobia.

The results from the trial, just published in the Australian and New Zealand Journal of Psychiatry, showed a 75 per cent reduction in phobia symptoms after six weeks of the treatment programme.

"The improvements they reported suggests there's great potential for the use of VR and mobile phone apps as a means of self-guided treatment for people struggling with often-crippling phobias," Associate Professor Lacey says.

"Participants demonstrated a strong acceptability of the app, highlighting its potential for delivering easily accessible, cost-effective treatment at scale, of particular use for those unable to access in-person exposure therapy to treat their phobias."

A total of 129 people took part in the six-week randomised, controlled trial, between May 2021 and December 2021, with a 12-week follow-up. Participants needed to be aged between 18-64 years, have a fear of either flying, heights, needles, spiders and dogs. They were emailed weekly questionnaires to record their progress. Those experiencing adverse events could request contact from a clinical psychologist at any stage.

"Participants experiencing all five types of phobia showed comparable improvements in the Severity Measures for Specific Phobia scale over the course of the trial. The average severity score decreased from 28/40 (moderate to severe symptoms) to 7/40 (minimal symptoms) after six weeks. There were no participant withdrawals due to intervention-related adverse events.

"The oVRcome app involves what's called "exposure therapy," a form of CBT exposing participants to their specific phobias in short bursts, to build up their tolerance to the phobia in a clinically-approved and controlled way," Associate Professor Lacey says.

"Some participants reported significant progress in overcoming their phobias after the trial period, with one feeling confident enough to now book an overseas family holiday, another lining up for a Covid vaccine and another reporting they now felt confident not only knowing there was a spider in the house but that they could possibly remove it themselves."

The app programme consisted of standard CBT components including psychoeducation, relaxation, mindfulness, cognitive techniques, exposure through VR, and a relapse prevention model. Participants were able to select their own exposure levels to their particular phobia from a large library of VR videos.

"This means the levels of exposure therapy could be tailored to an individual's needs which is a particular strength. The more traditional in-person exposure treatment for specific phobias have a notoriously high dropout rate due to discomfort, inconvenience and a lack of motivation in people seeking out fears to expose themselves to. With this VR app treatment, triallists had increased control in exposure to their fears, as well as control over when and where exposure occurs," says Associate Professor Lacey.

The researchers say this trial was novel, due to the cost-effective availability of the app and headsets and the fact that multiple phobias were tested at once. They say most comparative VR studies to date have investigated high-end VR devices which are only available in research and limited clinical settings. One Dutch study examined a low-cost VR Dutch-language program using animated imagery that demonstrated improvement in fear-of-height symptoms, however this study only examined a single type of specific phobia.

Associate Professor Lacey says public demand to take part in the trial was unprecedented, demonstrating the increasing need and desire for phobia treatment in the community.

"An estimated ten per cent of New Zealanders have been hesitant to take part in the government's COVID-19 vaccination programme due to needle phobia. This hasn't been helped by a significant shortage of psychologists. A petition to Parliament last year claimed New Zealand is 1,000 psychologists short, causing ballooning wait times nationwide, making it difficult for people to access help if needed. We need to further research and explore the use of more cost-effective, easily-accessible, home-based solutions such as this oVRcome app, to provide people with the treatment and support they need."

Read more at Science Daily

New report aims to improve VR use in healthcare education

A new report that could help improve how immersive technologies such as Virtual Reality (VR) and Augmented Reality (AR) are used in healthcare education and training has been published with significant input from the University of Huddersfield.

Professor David Peebles, Director of the University's Centre for Cognition and Neuroscience, and Huddersfield PhD graduate Matthew Pears contributed to the report -- 'Immersive technologies in healthcare training and education: Three principles for progress' -- recently published by the University of Leeds with input from range of academics, technologists and health professionals.

The principles have also been expanded upon in a letter to the prestigious journal BMJ Simulation and Technology Enhanced Learning.

The Huddersfield contribution to the report stems from research conducted over several years, which involved another former Huddersfield PhD researcher, Yeshwanth Pulijala, and Professor Eunice Ma, now with Falmouth University.

"Yeshwanth had an interest in technology and education, and in using VR for dentistry training. Matthew was looking at soft skills and situation awareness, which could be applied to investigating how dentists were able to keep a track of what was going on around them. They were similar subjects, although with different emphases, and so it seemed a natural area for collaboration."

With only a relatively small number of dental schools in the UK, the quartet visited seven dental schools in India in early 2017, with support from travel grants from Santander Bank, to test their VR-based training materials on students. The experience gained from that visit contributed to both researchers' PhDs, and ultimately led to the involvement of Professor Peebles and Matthew Pears in the new report.

The report argues for greater standardisation of how to use immersive technologies in healthcare training and education. As Professor Peebles explains, "It's about developing a set of principles and guidelines for the use of immersive technology in medical treatment. Immersive technology is becoming increasingly popular and, as the technology is advancing, it's becoming clear that there is great potential to make training more accessible and effective.

"It is important, however, that research is driven by the needs of the user and existing evidence rather than the technology. Rather than thinking 'we have a new bit of VR or AR kit, what can we do with it?', we should be looking at the problem that needs solving -- what are the learning needs, so how do we use technology to solve it?

"Developing immersive training materials can be very time-consuming and difficult to evaluate properly. Getting surgeons and medical students to take time out to test your VR training is challenging. In our case we were lucky to have a surgeon, Professor Ashraf Ayoub, a Professor of Oral and Maxillofacial Surgery at the University of Glasgow, who granted us permission to film a surgical procedure that was then transformed into a 3D environment to train students about situation awareness while in the operating theatre."

Professor Peebles hopes the work so far will provide a basis for more investigations that could help get the most from the potential that VR and immersive technology have to offer.

Read more at Science Daily

Virtual reality helping to treat fear of heights

 Researchers from the University of Basel have developed a virtual reality app for smartphones to reduce fear of heights. Now, they have conducted a clinical trial to study its efficacy. Trial participants who spent a total of four hours training with the app at home showed an improvement in their ability to handle real height situations.

Fear of heights is a widespread phenomenon. Approximately 5% of the general population experiences a debilitating level of discomfort in height situations. However, the people affected rarely take advantage of the available treatment options, such as exposure therapy, which involves putting the person in the anxiety-causing situation under the guidance of a professional. On the one hand, people are reluctant to confront their fear of heights. On the other hand, it can be difficult to reproduce the right kinds of height situations in a therapy setting.

This motivated the interdisciplinary research team led by Professor Dominique de Quervain to develop a smartphone-based virtual reality exposure therapy app called Easyheights. The app uses 360° images of real locations, which the researchers captured using a drone. People can use the app on their own smartphones together with a special virtual reality headset.

Gradually increasing the height

During the virtual experience, the user stands on a platform that is initially one meter above the ground. After allowing acclimatization to the situation for a certain interval, the platform automatically rises. In this way, the perceived distance above the ground increases slowly but steadily without an increase in the person's level of fear.

The research team studied the efficacy of this approach in a randomized, controlled trial and published the results in the journal NPJ Digital Medicine. Fifty trial participants with a fear of heights either completed a four-hour height training program (one 60-minute session and six 30-minute sessions over the course of two weeks) using virtual reality, or were assigned to the control group, which did not complete these training sessions.

Before and after the training phase -- or the same period of time without training -- the trial participants ascended the Uetliberg lookout tower near Zurich as far as their fear of heights allowed them. The researchers recorded the height level reached by the participants along with their subjective fear level at each level of the tower. At the end of the trial, the researchers evaluated the results from 22 subjects who completed the Easyheights training and 25 from the control group.

The group that completed the training with the app exhibited less fear on the tower and was able to ascend further towards the top than they could before completing the training. The control group exhibited no positive changes. The efficacy of the Easyheights training proved comparable to that of conventional exposure therapy.

Therapy in your own living room

Researchers have already been studying the use of virtual reality for treating fear of heights for more than two decades. "What is new, however, is that smartphones can be used to produce the virtual scenarios that previously required a technically complicated type of treatment, and this makes it much more accessible," explains Dr. Dorothée Bentz, lead author of the study.

The results from the study suggest that the repeated use of a smartphone-based virtual reality exposure therapy can greatly improve the behavior and subjective state of well-being in height situations. People who suffer from a mild fear of heights will soon be able to download the free app from major app stores and complete training sessions on their own. However, the researchers recommend that people who suffer from a serious fear of heights only use the app with the supervision of a professional.

Read more at Science Daily

Specific neurons that map memories now identified in the human brain

Brain-map overlay concept illustration.

An important aspect of human memory is our ability to conjure specific moments from the vast array of experiences that have occurred in any given setting. For example, if asked to recommend a tourist itinerary for a city you have visited many times, your brain somehow enables you to selectively recall and distinguish specific memories from your different trips to provide an answer.

Studies have shown that declarative memory -- the kind of memory you can consciously recall like your home address or your mother's name -- relies on healthy medial temporal lobe structures in the brain, including the hippocampus and entorhinal cortex (EC). These regions are also important for spatial cognition, demonstrated? by the Nobel-Prize-winning discovery of "place cells" and "grid cells" in these regions -- neurons that activate to represent specific locations in the environment during navigation (akin to a GPS). However, it has not been clear if or how this "spatial map" in the brain relates to a person's memory of events at those locations, and how neuronal activity in these regions enables us to target a particular memory for retrieval among related experiences.

A team led by neuroengineers at Columbia Engineering has found the first evidence that individual neurons in the human brain target specific memories during recall. They studied recordings in neurosurgical patients who had electrodes implanted in their brains and examined how the patients' brain signals corresponded to their behavior while performing a virtual-reality (VR) object-location memory task. The researchers identified "memory-trace cells" whose activity was spatially tuned to the location where subjects remembered encountering specific objects. The study is published today in Nature Neuroscience.

"We found these memory-trace neurons primarily in the entorhinal cortex (EC), which is one of the first regions of the brain affected by the onset of Alzheimer 's disease," says Joshua Jacobs, associate professor of biomedical engineering, who directed the study. "Because the activity of these neurons is closely related to what a person is trying to remember, it is possible that their activity is disrupted by diseases like Alzheimer's, leading to memory deficits. Our findings should open up new lines of investigation into how neural activity in the entorhinal cortex and medial temporal lobe helps us target past events for recall, and more generally how space and memory overlap in the brain."

The team was able to measure the activity of single neurons by taking advantage of a rare opportunity: invasively recording from the brains of 19 neurosurgical patients at several hospitals, including the Columbia University Irving Medical Center. The patients had drug-resistant epilepsy and so had already had recording electrodes implanted in their brains for their clinical treatment. The researchers designed experiments as engaging and immersive VR computer games and the bedridden patients used laptops and handheld controllers to move through virtual environments. In performing the task, subjects first navigated through the environment to learn the locations of four unique objects. Then the researchers removed the objects and asked patients to move through the environment and mark the location of one specific object on each trial.

The team measured the activity of neurons as the patients moved through the environment and marked their memory targets. Initially, they identified purely spatially tuned neurons similar to "place cells" that always activated when patients moved through specific locations, regardless of the subjects' memory target. "These neurons seemed only to care about the person's spatial location, like a pure GPS," says Salman E. Qasim, Jacobs' PhD student and lead author of the study.

But the researchers also noticed that other neurons only activated in locations relevant to the memory the patient was recalling on that trial -- whenever patients were instructed to target a different memory for recall, these neurons changed their activity to match the new target's remembered location. What especially excited Jacobs and Qasim is that they could actually decode the specific memory a patient was targeting based on the activity of these neurons.

"Our study demonstrates that neurons in the human brain track the experiences we are willfully recalling, and can change their activity patterns to differentiate between memories. They're just like the pins on your Google map that mark the locations you remember for important events," Qasim says. "This discovery might provide a potential mechanism for our ability to selectively call upon different experiences from the past and highlights how these memories may influence our brain's spatial map."

Read more at Science Daily

Virtual reality to solve minor personal problems

A new study shows that conversation with oneself embodied as Dr. Sigmund Freud works better to improve people's mood, compared to just talking about your problems in a virtual conversation with pre-scripted comments. Researchers claimed that the method could be used by clinicians to help people dealing with minor personal problems.

People are often much better at giving useful advice to a friend in trouble than they are in dealing with their own problems. Although we typically have continuous internal dialogue, we are trapped inside our own way of thinking with our own history and point of view, and find it difficult to take an external perspective regarding our own problems. However, with friends, especially someone we know well, it is much easier to understand the bigger picture, and help them find a way through their problems.

A research team of the University of Barcelona (UB), IDIBAPS and Virtual BodyWorks, a spin-off of both institutions and ICREA, has used immersive virtual reality to observe the effects of talking to themselves as if they were another person, using virtual reality. Study results, published in the Nature Group's journal Scientific Reports, show that conversation with oneself embodied as Dr Sigmund Freud works better to improve people's mood, compared to just talking about your problems in a virtual conversation with pre-scripted comments. Researchers claimed that the method could be used by clinicians to help people dealing with minor personal problems.

The study was led by Mel Slater and Solène Neyret, researchers at the Experimental Virtual Environments Lab for Neuroscience and Technology (Event Lab), a research group of the Faculty of Psychology of the UB. Clinical psychologist Guillem Feixas, of the UB Department of Clinical Psychology and Psychobiology and the Institute of Neurosciences of the University of Barcelona (UBNeuro) also guided the study.

Changing perception and attitude thanks to Virtual Reality

Previous studies developed by this research team have shown that when we adopt a different body using virtual reality, we change our behaviour, attitude and perception of things. "We showed earlier that it is possible for people to talk to themselves as if they were another person, body swapping to two different avatars, and that participants' mood and happiness improved. However, we didn't know whether this was due to simply the participant talking about their problem or whether the virtual body swapping really made a difference," said Mel Slater, also a member of the UBNeuro.

In order to test the body swapping idea, researchers compared one group who talked to themselves first embodied as the participant and then body swapping to a virtual Sigmund Freud; and another (control) group who spoke to the virtual Freud, but in that case Freud responded with pre-scripted questions and comments (there was no body swapping).

Embodied as Sigmund Freud

For this technique to work out,researchers scanned the person to obtain an 'avatar' which is a 3D-likeness of the person. In virtual reality, when they look at themselves, at their body parts, or in a mirror, they will see a representation of themselves. When they move their real body, their virtual body will move in the same way and at the same time. Seated across the table is another virtual human, in the case of this experiment, a representation of Dr Sigmund Freud.

The participant can explain their personal problem to Dr Freud, and then switch to being embodied as Freud. Now, embodied as Freud, when they look down towards themselves, or in a mirror, they will see Freud's body rather than their own, and also this body will move in synchrony with their own movements. "They will see and hear their own likeness explaining the problem, and they see their virtual self as if this were another person. Now they themselves have become the 'friend' who is listening and trying to help," said Mel Slater.

While embodied as Freud, and after perceiving a strong likeness of themselves describing a problem, they can respond, as Freud, back to themselves and ask a question or help the person in front (themselves) to find a solution. After this, they are embodied once again in their own body and they can see and hear Freud's answer. Although it was really themselves who had spoken through Freud, they will hear their voice as disguised. They can keep switching back and forth between the two bodies, so having a conversation: in reality it is with themselves, but it appears as if it is between two different people.

Better results in dealing with personal problems

One week after the completion of the experiment more than 80% of participants in the body swapping group reported a change with respect to their problem, compared to less than 50% in the control group. "We found that those in the body swapping group got better knowledge, understanding, control, and new ideas about their problem compared to the control group (no body swapping)," said Mel Slater.

Participants were guided by clinical psychologist Tania Johnston about how to formulate their problem, so researchers do not know whether this method could be used without this prior clinical advice, and the extent to which the clinician could be incorporated into the virtual reality as part of the procedure.

Read more at Science Daily

Spherical display brings virtual collaboration closer to reality

Virtual reality spherical display.

Virtual reality can often make a user feel isolated from the world, with only computer-generated characters for company. But researchers at the University of British Columbia and University of Saskatchewan think they may have found a way to encourage a more sociable virtual reality.

The researchers have developed a ball-shaped VR display that supports up to two users at a time, using advanced calibration and graphics rendering techniques that produce a complete, distortion-free 3D image even when viewed from multiple angles.

Most spherical VR displays in the market are capable of showing a correct image only from a single viewpoint, said lead researcher Sidney Fels, an electrical and computer engineering professor at UBC.

"When you look at our globe, the 3D illusion is rich and correct from any angle," explained Fels. "This allows two users to use the display to do some sort of collaborative task or enjoy a multiplayer game, while being in the same space. It's one of the very first spherical VR systems with this capability."

The system, which the researchers are calling Crystal, includes a 24-inch (600 millimetre) hollow ball-shaped display. The display surface was custom-made to specifications in Ottawa, while four high-speed projectors and one camera used for creating the images, calibration and touch sensing were purchased off-the-shelf.

The researchers are working on a four-person system and see many potential uses for their display in the future, including multiplayer virtual reality games, virtual surgery and VR-aided learning. However, they are focusing on teleconferencing applications and computer-aided design for now.

"Imagine a remote user joining a meeting of local users. At either location you can have a Crystal globe, which is great for seeing people's heads and faces in 3D," said Ian Stavness, a computer science professor at the University of Saskatchewan and a member of the research team. "Or you can have a team of industrial designers in a room, perfecting a design with the help of VR and motion tracking technology."

While the technology is young, the researchers are forecasting a good future for it.

"We're not saying that spherical VR will replace flat screens or headsets," said Fels, adding "but we think it can be a good option for VR activities where you still want to see and talk to other people -- be it at home or in the office, for work or play."

Read more at Science Daily

Virtual reality simulation of a supermassive black hole

The black hole at the centre of our galaxy, Sagittarius A*, has been visualised in virtual reality for the first time. The details are described in an article published in the open access journal Computational Astrophysics and Cosmology.

Scientists at Radboud University, The Netherlands and Goethe University, Germany used recent astrophysical models of Sagittarius A* to create a series of images that were then put together to create a 360 degree virtual reality simulation of the black hole, that can be viewed on widely available VR consoles. The authors suggest that this virtual reality simulation could be useful for studying black holes.

Jordy Davelaar, corresponding author, said: "Our virtual reality simulation creates one of the most realistic views of the direct surroundings of the black hole and will help us to learn more about how black holes behave. Traveling to a black hole in our lifetime is impossible, so immersive visualizations like this can help us understand more about these systems from where we are."

The authors also suggest that the virtual reality simulation could help encourage the general public, including children, to take an interest in astrophysics.

Davelaar said: "The visualisations that we produced have a great potential for outreach. We used them to introduce children to the phenomenon of black holes, and they really learned something from it. This suggests that immersive virtual reality visualizations are a great tool to show our work to a broader audience, even when it involves very complicated systems like black holes."

Heino Falcke, Professor at Radboud University adds: "We all have a picture in our head of how black holes supposedly look, but science has progressed and we can now make much more accurate renderings -- and these black holes look quite different from what we are used to. These new visualisations are just the start, more to come in the future."

From Science Daily