New holographic camera sees the unseen with high precision

Northwestern University researchers have invented a new high-resolution camera that can see the unseen -- including around corners and through scattering media, such as skin, fog or potentially even the human skull.

Called synthetic wavelength holography, the new method works by indirectly scattering coherent light onto hidden objects, which then scatters again and travels back to a camera. From there, an algorithm reconstructs the scattered light signal to reveal the hidden objects. Due to its high temporal resolution, the method also has potential to image fast-moving objects, such as the beating heart through the chest or speeding cars around a street corner.

The study will be published on Nov. 17 in the journal Nature Communications.

The relatively new research field of imaging objects behind occlusions or scattering media is called non-line-of-sight (NLoS) imaging. Compared to related NLoS imaging technologies, the Northwestern method can rapidly capture full-field images of large areas with submillimeter precision. With this level of resolution, the computational camera could potentially image through the skin to see even the tiniest capillaries at work.

While the method has obvious potential for noninvasive medical imaging, early-warning navigation systems for automobiles and industrial inspection in tightly confined spaces, the researchers believe potential applications are endless.

"Our technology will usher in a new wave of imaging capabilities," said Northwestern's Florian Willomitzer, first author of the study. "Our current sensor prototypes use visible or infrared light, but the principle is universal and could be extended to other wavelengths. For example, the same method could be applied to radio waves for space exploration or underwater acoustic imaging. It can be applied to many areas, and we have only scratched the surface."

Willomitzer is a research assistant professor of electrical and computer engineering at Northwestern's McCormick School of Engineering. Northwestern co-authors include Oliver Cossairt, associate professor of computer science and electrical and computer engineering, and former Ph.D. student Fengqiang Li. The Northwestern researchers collaborated closely with Prasanna Rangarajan, Muralidhar Balaji and Marc Christensen, all researchers at Southern Methodist University.

Intercepting scattered light

Seeing around a corner versus imaging an organ inside the human body might seem like very different challenges, but Willomitzer said they are actually closely related. Both deal with scattering media, in which light hits an object and scatters in a manner that a direct image of the object can no longer be seen.

"If you have ever tried to shine a flashlight through your hand, then you have experienced this phenomenon," Willomitzer said. "You see a bright spot on the other side of your hand, but, theoretically, there should be a shadow cast by your bones, revealing the bones' structure. Instead, the light that passes the bones gets scattered within the tissue in all directions, completely blurring out the shadow image."

The goal, then, is to intercept the scattered light in order to reconstruct the inherent information about its time of travel to reveal the hidden object. But that presents its own challenge.

"Nothing is faster than the speed of light, so if you want to measure light's time of travel with high precision, then you need extremely fast detectors," Willomitzer said. "Such detectors can be terribly expensive."

Tailored waves

To eliminate the need for fast detectors, Willomitzer and his colleagues merged light waves from two lasers in order to generate a synthetic light wave that can be specifically tailored to holographic imaging in different scattering scenarios.

"If you can capture the entire light field of an object in a hologram, then you can reconstruct the object's three-dimensional shape in its entirety," Willomitzer explained. "We do this holographic imaging around a corner or through scatterers -- with synthetic waves instead of normal light waves."

Over the years, there have been many NLoS imaging attempts to recover images of hidden objects. But these methods typically have one or more problems. They either have low resolution, an extremely small angular field of regard, require a time-consuming raster scan or need large probing areas to measure the scattered light signal.

The new technology, however, overcomes these issues and is the first method for imaging around corners and through scattering media that combines high spatial resolution, high temporal resolution, a small probing area and a large angular field of view. This means that the camera can image tiny features in tightly confined spaces as well as hidden objects in large areas with high resolution -- even when the objects are moving.

Turning 'walls into mirrors'

Because light only travels on straight paths, an opaque barrier (such as a wall, shrub or automobile) must be present in order for the new device to see around corners. The light is emitted from the sensor unit (which could be mounted on top of a car), bounces off the barrier, then hits the object around the corner. The light then bounces back to the barrier and ultimately back into the detector of the sensor unit.

"It's like we can plant a virtual computational camera on every remote surface to see the world from the surface's perspective," Willomitzer said.

For people driving roads curving through a mountain pass or snaking through a rural forest, this method could prevent accidents by revealing other cars or deer just out of sight around the bend. "This technique turns walls into mirrors," Willomitzer said. "It gets better as the technique also can work at night and in foggy weather conditions."

In this manner, the high-resolution technology also could replace (or supplement) endoscopes for medical and industrial imaging. Instead of needing a flexible camera, capable of turning corners and twisting through tight spaces -- for a colonoscopy, for example -- synthetic wavelength holography could use light to see around the many folds inside the intestines.

Similarly, synthetic wavelength holography could image inside industrial equipment while it is still running -- a feat that is impossible for current endoscopes.

"If you have a running turbine and want to inspect defects inside, you would typically use an endoscope," Willomitzer said. "But some defects only show up when the device is in motion. You cannot use an endoscope and look inside the turbine from the front while it is running. Our sensor can look inside a running turbine to detect structures that are smaller than one millimeter."

Read more at Science Daily

3D holographic head-up display could improve road safety

Researchers have developed the first LiDAR-based augmented reality head-up display for use in vehicles. Tests on a prototype version of the technology suggest that it could improve road safety by 'seeing through' objects to alert of potential hazards without distracting the driver.

The technology, developed by researchers from the University of Cambridge, the University of Oxford and University College London (UCL), is based on LiDAR (light detection and ranging), and uses LiDAR data to create ultra high-definition holographic representations of road objects which are beamed directly to the driver's eyes, instead of 2D windscreen projections used in most head-up displays.

While the technology has not yet been tested in a car, early tests, based on data collected from a busy street in central London, showed that the holographic images appear in the driver's field of view according to their actual position, creating an augmented reality. This could be particularly useful where objects such as road signs are hidden by large trees or trucks, for example, allowing the driver to 'see through' visual obstructions. The results are reported in the journal Optics Express.

"Head-up displays are being incorporated into connected vehicles, and usually project information such as speed or fuel levels directly onto the windscreen in front of the driver, who must keep their eyes on the road," said lead author Jana Skirnewskaja, a PhD candidate from Cambridge's Department of Engineering. "However, we wanted to go a step further by representing real objects in as panoramic 3D projections."

Skirnewskaja and her colleagues based their system on LiDAR, a remote sensing method which works by sending out a laser pulse to measure the distance between the scanner and an object. LiDAR is commonly used in agriculture, archaeology and geography, but it is also being trialled in autonomous vehicles for obstacle detection.

Using LiDAR, the researchers scanned Malet Street, a busy street on the UCL campus in central London. Co-author Phil Wilkes, a geographer who normally uses LiDAR to scan tropical forests, scanned the whole street using a technique called terrestrial laser scanning. Millions of pulses were sent out from multiple positions along Malet Street. The LiDAR data was then combined with point cloud data, building up a 3D model.

"This way, we can stitch the scans together, building a whole scene, which doesn't only capture trees, but cars, trucks, people, signs, and everything else you would see on a typical city street," said Wilkes. "Although the data we captured was from a stationary platform, it's similar to the sensors that will be in the next generation of autonomous or semi-autonomous vehicles."

When the 3D model of Malet St was completed, the researchers then transformed various objects on the street into holographic projections. The LiDAR data, in the form of point clouds, was processed by separation algorithms to identify and extract the target objects. Another algorithm was used to convert the target objects into computer-generated diffraction patterns. These data points were implemented into the optical setup to project 3D holographic objects into the driver's field of view.

The optical setup is capable of projecting multiple layers of holograms with the help of advanced algorithms. The holographic projection can appear at different sizes and is aligned with the position of the represented real object on the street. For example, a hidden street sign would appear as a holographic projection relative to its actual position behind the obstruction, acting as an alert mechanism.

In future, the researchers hope to refine their system by personalising the layout of the head-up displays and have created an algorithm capable of projecting several layers of different objects. These layered holograms can be freely arranged in the driver's vision space. For example, in the first layer, a traffic sign at a further distance can be projected at a smaller size. In the second layer, a warning sign at a closer distance can be displayed at a larger size.

"This layering technique provides an augmented reality experience and alerts the driver in a natural way," said Skirnewskaja. "Every individual may have different preferences for their display options. For instance, the driver's vital health signs could be projected in a desired location of the head-up display.

"Panoramic holographic projections could be a valuable addition to existing safety measures by showing road objects in real time. Holograms act to alert the driver but are not a distraction."

Read more at Science Daily