Speaking without vocal cords, thanks to a new AI-assisted wearable device

People with voice disorders, including those with pathological vocal cord conditions or who are recovering from laryngeal cancer surgeries, can often find it difficult or impossible to speak. That may soon change.

A team of UCLA engineers has invented a soft, thin, stretchy device measuring just over 1 square inch that can be attached to the skin outside the throat to help people with dysfunctional vocal cords regain their voice function. Their advance is detailed this week in the journal Nature Communications.

The new bioelectric system, developed by Jun Chen, an assistant professor of bioengineering at the UCLA Samueli School of Engineering, and his colleagues, is able to detect movement in a person's larynx muscles and translate those signals into audible speech with the assistance of machine-learning technology -- with nearly 95% accuracy.

The breakthrough is the latest in Chen's efforts to help those with disabilities. His team previously developed a wearable glove capable of translating American Sign Language into English speech in real time to help users of ASL communicate with those who don't know how to sign.

The tiny new patch-like device is made up of two components. One, a self-powered sensing component, detects and converts signals generated by muscle movements into high-fidelity, analyzable electrical signals; these electrical signals are then translated into speech signals using a machine-learning algorithm. The other, an actuation component, turns those speech signals into the desired voice expression.

The two components each contain two layers: a layer of biocompatible silicone compound polydimethylsiloxane, or PDMS, with elastic properties, and a magnetic induction layer made of copper induction coils. Sandwiched between the two components is a fifth layer containing PDMS mixed with micromagnets, which generates a magnetic field.

Utilizing a soft magnetoelastic sensing mechanism developed by Chen's team in 2021, the device is capable of detecting changes in the magnetic field when it is altered as a result of mechanical forces -- in this case, the movement of laryngeal muscles. The embedded serpentine induction coils in the magnetoelastic layers help generate high-fidelity electrical signals for sensing purposes.

Measuring 1.2 inches on each side, the device weighs about 7 grams and is just 0.06 inch thick. With double-sided biocompatible tape, it can easily adhere to an individual's throat near the location of the vocal cords and can be reused by reapplying tape as needed.

Voice disorders are prevalent across all ages and demographic groups; research has shown that nearly 30% of people will experience at least one such disorder in their lifetime. Yet with therapeutic approaches, such as surgical interventions and voice therapy, voice recovery can stretch from three months to a year, with some invasive techniques requiring a significant period of mandatory postoperative voice rest.

"Existing solutions such as handheld electro-larynx devices and tracheoesophageal- puncture procedures can be inconvenient, invasive or uncomfortable," said Chen who leads the Wearable Bioelectronics Research Group at UCLA, and has been named one the world's most highly cited researchers five years in a row. "This new device presents a wearable, non-invasive option capable of assisting patients in communicating during the period before treatment and during the post-treatment recovery period for voice disorders."

How machine learning enables the wearable tech

In their experiments, the researchers tested the wearable technology on eight healthy adults. They collected data on laryngeal muscle movement and used a machine-learning algorithm to correlate the resulting signals to certain words. They then selected a corresponding output voice signal through the device's actuation component.

The research team demonstrated the system's accuracy by having the participants pronounce five sentences -- both aloud and voicelessly -- including "Hi, Rachel, how are you doing today?" and "I love you!"

The overall prediction accuracy of the model was 94.68%, with the participants' voice signal amplified by the actuation component, demonstrating that the sensing mechanism recognized their laryngeal movement signal and matched the corresponding sentence the participants wished to say.

Going forward, the research team plans to continue enlarging the vocabulary of the device through machine learning and to test it in people with speech disorders.

Read more at Science Daily

Real-time wearable human emotion recognition technology developed

A groundbreaking technology that can recognize human emotions in real time has been developed by Professor Jiyun Kim and his research team in the Department of Material Science and Engineering at UNIST. This innovative technology is poised to revolutionize various industries, including next-generation wearable systems that provide services based on emotions.

Understanding and accurately extracting emotional information has long been a challenge due to the abstract and ambiguous nature of human affects such as emotions, moods, and feelings.

To address this, the research team has developed a multi-modal human emotion recognition system that combines verbal and non-verbal expression data to efficiently utilize comprehensive emotional information.

At the core of this system is the personalized skin-integrated facial interface (PSiFI) system, which is self-powered, facile, stretchable, and transparent.

It features a first-of-its-kind bidirectional triboelectric strain and vibration sensor that enables the simultaneous sensing and integration of verbal and non-verbal expression data.

The system is fully integrated with a data processing circuit for wireless data transfer, enabling real-time emotion recognition.

Utilizing machine learning algorithms, the developed technology demonstrates accurate and real-time human emotion recognition tasks, even when individuals are wearing masks.

The system has also been successfully applied in a digital concierge application within a virtual reality (VR) environment.

The technology is based on the phenomenon of "friction charging," where objects separate into positive and negative charges upon friction.

Notably, the system is self-generating, requiring no external power source or complex measuring devices for data recognition.

Professor Kim commented, "Based on these technologies, we have developed a skin-integrated face interface (PSiFI) system that can be customized for individuals." The team utilized a semi-curing technique to manufacture a transparent conductor for the friction charging electrodes.

Additionally, a personalized mask was created using a multi-angle shooting technique, combining flexibility, elasticity, and transparency.

The research team successfully integrated the detection of facial muscle deformation and vocal cord vibrations, enabling real-time emotion recognition.

The system's capabilities were demonstrated in a virtual reality "digital concierge" application, where customized services based on users' emotions were provided.

Jin Pyo Lee, the first author of the study, stated, "With this developed system, it is possible to implement real-time emotion recognition with just a few learning steps and without complex measurement equipment. This opens up possibilities for portable emotion recognition devices and next-generation emotion-based digital platform services in the future."

The research team conducted real-time emotion recognition experiments, collecting multimodal data such as facial muscle deformation and voice.

The system exhibited high emotional recognition accuracy with minimal training.

Its wireless and customizable nature ensures wearability and convenience.

Furthermore, the team applied the system to VR environments, utilizing it as a "digital concierge" for various settings, including smart homes, private movie theaters, and smart offices.

The system's ability to identify individual emotions in different situations enables the provision of personalized recommendations for music, movies, and books.

Professor Kim emphasized, "For effective interaction between humans and machines, human-machine interface (HMI) devices must be capable of collecting diverse data types and handling complex integrated information. This study exemplifies the potential of using emotions, which are complex forms of human information, in next-generation wearable systems."

Read more at Science Daily

'Smart glove' can boost hand mobility of stroke patients

This month, a group of stroke survivors in B.C. will test a new technology designed to aid their recovery, and ultimately restore use of their limbs and hands.

Participants will wear a new groundbreaking "smart glove" capable of tracking their hand and finger movements during rehabilitation exercises supervised by Dr. Janice Eng, a leading stroke rehabilitation specialist and professor of medicine at UBC.

The glove incorporates a sophisticated network of highly sensitive sensor yarns and pressure sensors that are woven into a comfortable stretchy fabric, enabling it to track, capture and wirelessly transmit even the smallest hand and finger movements.

"With this glove, we can monitor patients' hand and finger movements without the need for cameras. We can then analyze and fine-tune their exercise programs for the best possible results, even remotely," says Dr. Eng.

Precision in a wearable device

UBC electrical and computer engineering professor Dr. Peyman Servati, PhD student Arvin Tashakori and their team at their startup, Texavie, created the smart glove for collaboration on the stroke project.

Dr. Servati highlighted a number of breakthroughs, described in a paper published last week in Nature Machine Intelligence.

"This is the most accurate glove we know of that can track hand and finger movement and grasping force without requiring motion-capture cameras. Thanks to machine learning models we developed, the glove can accurately determine the angles of all finger joints and the wrist as they move. The technology is highly precise and fast, capable of detecting small stretches and pressures and predicting movement with at least 99-per-cent accuracy -- matching the performance of costly motion-capture cameras."

Unlike other products in the market, the glove is wireless and comfortable, and can be easily washed after removing the battery.

Dr. Servati and his team have developed advanced methods to manufacture the smart gloves and related apparel at a relatively low cost locally.

Augmented reality and robotics

Dr. Servati envisions a seamless transition of the glove into the consumer market with ongoing improvements, in collaboration with different industrial partners.

The team also sees potential applications in virtual reality and augmented reality, animation and robotics.

Read more at Science Daily