Researchers at the Tokyo Institute of Technology (Tokyo Tech) working in collaboration with colleagues at Carnegie Mellon University, the University of St Andrews and the University of New South Wales have developed a pulse-led device for estimating hand placement in 3D The system consists of a camera which captures images of the back of the hand and is supported by a neural network called DorsalNet which can accurately recognize dynamic gestures.
The ability to track hand gestures is crucial in advancing augmented reality (AR) and virtual reality (VR) devices that are already starting to be in high demand in the medical, sports and entertainment industries. Until now, these devices have involved the use of large data gloves which tend to prevent natural movement or controllers with a limited detection range.
Now, a research team led by Hideki Koike of Tokyo Tech has come up with a camera-based 3D wearable wrist recognition system that may in the future be on par with a smartwatch. Its system can significantly capture hand movements in mobile settings.
“This work is the first vision-based real-time 3D hand position estimator using visual characteristics of the dorsal hand region,” the researchers explain. The system consists of a camera supported by a neural network called DorsalNet that can accurately estimate the hands in 3D by detecting changes in the lower hand.
The researchers confirmed that their system outperforms previous work with a 20% higher average accuracy in dynamic gesture recognition and achieves 75% detection accuracy for eleven different grip types.
The work could advance the development of controls that support unarmed interaction. In preliminary tests, the researchers showed that it would be possible to use their system to control smart devices, for example by changing the time on a smartwatch simply by changing the angle of the finger. They also showed that it would be possible to use the system as a mouse or virtual keyboard, for example by rotating the wrist to control the position of the pointer and using a simple 8-key keyboard to enter input.
They pointed out that further improvements to the system such as using a camera with a higher frame rate to capture rapid impulse motion and being able to cope with more diverse lighting conditions will be. required for worldwide use. real.