If you were in a business meeting or a quiet library, it wouldn’t really be appropriate if you suddenly blurted out “Siri, check email.” That’s where the Speechin necklace is designed to come in, as it recognizes silent speech.
The experimental device is being developed by Cornell University’s Asst. Prof. Cheng Zhang and doctoral student Ruidong Zhang. It builds upon the NeckFace necklace that Cheng Zhang unveiled last year, which monitored the wearer’s facial expressions.
Along with a microprocessor, battery and Bluetooth module, Speechin also features an upward-facing infrared camera that images the underside of the wearer’s chin. It’s held in this orientation via a set of “wings” that extend out to either side, along with a coin that serves as a weight on its bottom. In order to address privacy concerns, it doesn’t point directly at the user’s face.
Utilizing machine-learning-based algorithms, the device is able to determine which commands its wearer is silently speaking, based on their chin movements. It can then relay those commands to a paired smartphone.
The system was initially trained by monitoring the chin movements of 20 volunteers as they silently spoke known words and phrases – 10 of those people spoke English, while the other 10 spoke Mandarin. In subsequent tests, the participants mouthed 54 commonly used commands in English, along with 44 Mandarin words and phrases.
The necklace proved to be 90.5 and 91.6 percent accurate at recognizing the English and Mandarin speech, respectively. Those figures did drop significantly when volunteers used the device while walking, as their individual walking styles caused their heads to move in an unpredictable fashion.
It is hoped that once developed further, the technology could be utilized not only in settings where people have to stay quiet, but also in noisy environments where smartphones are unable to hear their users. The Speechin necklace could additionally be used by people who lack the power of speech.
Speechin necklace recognizes its wearer’s silently spoken words [New Atlas]