Facebook’s Reality Labs Research wing is currently working on a number of projects designed to help us cut through noisy environments and zero in on folks we’re trying to hold a conversation with, or enhance our daily lives when used with AR glasses.
“At FRL Research, we are exploring new technologies that can extend, protect, and enhance your hearing ability – giving you the ability to increase concentration and focus, while allowing you to seamlessly interact with the people and information you care about,” said team lead Tony Miller. “At the heart of this work is a focus on building hardware that is deeply rooted in auditory perception and augmented by the latest developments in signal processing and artificial intelligence.”
The basic idea is that folks would don a pair of special glasses sporting multiple microphones. By tracking what or who the wearer is looking at, the combination of off-the-shelf hardware and custom software used for the current prototype can work out what it is the user would like to hear and enhance those sounds while turning down background noise.
The team has also built a near-field beamforming audio prototype using custom hardware developed at the lab. This technology could be of use if you’re at a rock concert and need to check in on the kids, as it effectively supercharges call clarity, focusing on the wearer’s voice and turning background noise way down. If used with active noise canceling in-ear monitors currently under development, the caller too could transport themselves into a kind of cone of silence while using a phone in a loud environment.
One of the end goals of the research is to create augmented reality glasses that can enhance both the visual and acoustic world around the wearer. The wearable could, for example, guide you through unfamiliar streets to a recommended restaurant, then work out what kinds of sounds you might not want to hear while dining (such as aircon, kitchen chaos or that annoyingly loud business party in the far corner) and remove such noise from the soundscape.
The technology could also help holiday-makers communicate with locals buy providing real-time translations, able to zero in on the native’s voice to ensure accuracy, and then provide language-appropriate responses in front of your eyes.
Beyond everyday use by those with decent enough hearing, there is also potential for the technology to help folks suffering hearing loss by essentially allowing them to tap into a supercharged hearing aid setup with augmented or virtual reality thrown in for good measure.
And the FRL Research teams are already mindful of potential privacy and security concerns and are factoring in safeguards to prevent misuse of the technology. But it’s still early days for such FRL Research innovations, and no consumer product timeline has been revealed.
Facebook is working on AR glasses with a focus on audio clarity [New Atlas]