Toward the end of 2020, digital artist David Li collaborated with Google’s Arts and Culture lab to release a fun machine learning experiment called the Blob Opera. Now Stuck Labs has developed a touchless interface that allows young conductors to control the operatic action with a wave of their arms.
Stuck Labs is the innovation arm of Singapore’s Stuck design studio, and is made up of hardware and software engineers, fabricators, scientists and designers. Back in January, the team created an elevator button concept that tracks the movement of a finger as it approaches and allows for the button to be pressed without physically touching it, and so Kinetic Touchless was born.
Then in March, Stuck Labs turned its attention to the kind of sliding doors seen in shopping malls, hotels and airport lounges. Rather than just opening as someone approaches, and letting out any heat from inside while also allowing cold air to flood in, the designers created a sensor that could open the door with a wave of the hand. Once through the gap, the door could then automatically close.
The third iteration of the Kinetic Touchless sensing technology, the details of which Stuck Labs is keeping close to its chest, was inspired by the clever and very addictive Blob Opera experiment created by David Li and the Google Arts and Culture lab.
But rather than using a computer mouse, Stuck Labs decided to make controlling the Blob Opera more realistic and engaging by developing a touchless interface that’s able to track the gestures of young conductors at the podium and have the AI-driven vocalists perform to the wave of a hand, the raising of an arm, and so on.
“Kinetic Touchless 3.0 mirrors and enlarges one’s body motion to give children the power to physically conduct virtual opera blobs,” Stuck Labs explained. “By correlating the natural movements of conducting to the controls needed to operate Blob Opera, Kinetic Touchless 3.0 gives you the ability to conduct as you would a real-life orchestra. Practicing conducting won’t have to be purely just an arm workout, nor will playing with Blob Opera be only a two-dimensional experience.”
Gesture-tracking tech allows kids to conduct Blob Opera with a wave [New Atlas]