When it comes to tracking the positions of a person’s moving hand, sensor-equipped gloves are often used. An experimental new system, however, utilizes a wrist-mounted camera … which doesn’t even “see” the user’s fingers.
The prototype setup was designed by scientists at the Tokyo Institute of Technology, working with colleagues from Carnegie Mellon University (US), the University of St. Andrews (Scotland) and the University of New South Wales (Australia).
It incorporates a small computer-linked video camera, which is attached to the top of a cuff on the user’s wrist. That camera points forward, getting a dorsal (back) view of the hand. For the most part, the fingers aren’t even in the shot.
As the video is being analyzed by a neural network known as DorsalNet, subtle yet distinctive changes in the contours of the back of the hand are matched up to corresponding finger movements. In this way, the system is able to ascertain the user’s three-dimensional “hand pose” in real time.
So far, the technology has proven to be 75-percent accurate at detecting 11 different grasp positions. That figure should rise as the system is developed further, which may include increasing its sensitivity by using a camera with a higher frame rate.
Ultimately, the setup could be incorporated into a device such as a smartwatch. It could then be utilized in applications such as the one-handed control of computers, or the tracking of users’ hands in VR environments.
The research is being presented this week via the online 33rd ACM Symposium on User Interface Software and Technology.
Wrist-camera tech tracks 3D hand pose, without seeing the fingers [New Atlas]