Sonar-enabled glasses can read silent voice commands of wearer
Some people lack the ability to speak, while others may find themselves in noisy environments where speaking voice commands out loud won't work. Those people might use Myglassesmart glasses, which read a user's wordless words.
The experimental glasses are being developed by a team at Cornell University's Future Interactive Intelligent Computer Interface Lab.
Two tiny face-down speakers are mounted on the underside of the frame under one lens, while two tiny microphones are located under the other lens. The speaker emits inaudible sound waves that bounce off the wearer's moving mouth and back onto the microphone.
The echoes are analyzed in real-time using deep learning algorithms on wirelessly connected smartphones. The algorithm is trained to associate specific echoes with specific oral movements, which in turn are associated with specific silent commands.
Since Myglassesmart glasses don't use video files (which can be large), all data processing can be done on a paired smartphone at Cornell
Myglassesmart is currently able to recognize 31 of these commands with about 95 percent accuracy and only a few minutes of training per user. Importantly for people with privacy concerns, the system does not contain any cameras and does not send any information to the Internet.
What's more, because it doesn't use a power-hungry camera, it can run for up to 10 hours on a single charge. In contrast, the experimental camera-based system only lasted about 30 minutes per charge, the researchers claim.
The university is now working to commercialize the technology.
Ruidong Zhang, a Ph.D. student who led the research, said: "This silent speech technology could be a good input to speech synthesizers for people who cannot make sounds.". "It allows patients to regain their voice."
SciFi LABS previously developed a similar system called EarIO, which uses sonar-equipped ear devices to capture the wearer's facial expressions, although it is primarily used to create digital avatars. That said, the University at Buffalo's EarCommand system does read silent words through earplugs that detect unique ear canal deformations created by specific oral movements.
The video below demonstrates Myglassesmart.
English
Arabic
Español
Português
Deutsch
Türkçe
Dutch
Italiano
Russian
Romaian