Multimodal interface for temporal pattern based interactive large volumetric visualization

PubDate: December 2017

Teams: Indian Institute of Information Technology;Nanyang Technical University

Writers: Piyush Kumar; Anupam Agrawal; Shitala Prasad

PDF: Multimodal interface for temporal pattern based interactive large volumetric visualization

Abstract

Scientific data visualization is a prominent area of research in the development of Virtual Reality Applications in order to make it more interactive and robotic. But the efficient interaction with the large size of medical data is a challenging task to diagnose virtual surgerical environment learning for a Physician. In this paper, we proposed a multimodal interface for GPU-accelerated interactive large scale volumetric data rendering to overcome this limitation. The large data has been pre-processed by octree method. An improved raycasting algorithm is used in association with a transfer function classification method for the effective rendering. The temporal data is used for defining gestures, retrieving in a pattern from the wearable device for providing multimodality with the large rendered data. A gesture vocabulary has been defined by these patterns for the navigation in visualizing the large scale medical data, which consists of five complex interactive postures used for Normal, Picking, Rotation, Dragging, and Zooming gestures. These gesture vocabularies have been categorized by kNN classification method of pattern recognition. Experimental results of the proposed approach are analyzed with the help of various ANOVA and T-testing graphs using SPSS 20 version tool and confidence interval of interaction with hand gestures vocabulary. The results of proposed approach are further compared with the existing approaches in which Microsoft Kinect and P5 dataglove have been used. The proposed system has been navigated by the DG5 VHand 2.0 Bluetooth version hand dataglove as wearable assistive device to achieve an effective interaction. The system has been tested on 10 different sizes of volume datasets ranging from 10MB to 3.15 GB. The scope of this paper is basically to develop system training with robotic arm in medical domain.