While users of full-scale augmented reality (AR) setups can already “reach in” and manipulate virtual objects with their hands, users of smartphone-based AR systems are limited to utilizing the phone’s touchscreen. That could soon change, though, thanks to new technology that puts their hands in the picture.
Known as Portal-ble, the system is being developed by a team at Rhode Island’s Brown University.
In its current form, it consists of Android software (an iOS version is in the works), along with a Leap Motion infrared sensor that’s mounted on the back of the phone, and a Compute Stickthat’s plugged into the phone to provide extra processing power. The researchers hope that it will ultimately be possible to incorporate the whole system into regular off-the-shelf smartphones.
When a user extends one hand out in front of themselves, the sensor detects the position of that hand in three-dimensional space. The software responds by placing a computer-generated image of the hand within the onscreen AR environment. As the user subsequently moves their actual hand, the corresponding AR hand moves accordingly, allowing them to pick up, move, and otherwise manipulate virtual objects that appear to exist – on the phone’s screen, at least – within the real world.
Volunteers have already utilized the technology to perform functions such as stacking virtual blocks, and drawing three-dimensional virtual gardens. Interestingly, although the hand that’s holding the phone is not the one that’s performing the actions, the test subjects’ performance was improved by a feedback system in which the phone vibrates when objects are touched.
Portal-ble is now being developed further, with the Android source code available for free to groups that wish to experiment with it. The system is demonstrated in the video below.
“We wanted to make something that made AR portable so that people could use anywhere without any bulky headsets,” says lead scientist, Asst. Prof. Jeff Huang. “We also wanted people to be able to interact with the virtual world in a natural way using their hands.”
Portal-ble tech gives smartphone AR a helping hand [New Atlas]