Anticipatory Hand Glove: Understanding Human Actions for Enhanced Interaction

Perceived real-time interactions between humans and a metaverse often require synchronizing actions in the virtual and real world. Latency is a main blocking factor due to processing and transmitting data over long distances because cloud data centres that deploy metaverse locate far away. We present an anticipatory computing solution to predict precisely human actions grasping objects based on hand kinesthetics leveraging smart gloves with IMU and flexion sensors. We demonstrated that the gain from the anticipation of up to several hundred milliseconds could compensate for computing and transmission latency, enabling immersive interactions over extremely long distances. Our solution is flexible and is free from locking into particular smart glove vendors, making it easy for future extensions.