Nweon Paper https://paper.nweon.com 映维网,影响力虚拟现实(VR)、增强现实(AR)产业信息数据平台 Wed, 07 Dec 2022 02:42:33 +0000 en-US hourly 1 https://wordpress.org/?v=4.8.17 https://paper.nweon.com/wp-content/uploads/2021/04/nweon-icon.png Nweon Paper https://paper.nweon.com 32 32 Skin-integrated Haptic Interfaces enabled by Scalable Mechanical Actuators for Virtual Reality https://paper.nweon.com/13649 Wed, 07 Dec 2022 02:42:12 +0000 https://paper.nweon.com/?p=13649 ...

Skin-integrated Haptic Interfaces enabled by Scalable Mechanical Actuators for Virtual Reality最先出现在Nweon Paper

]]>
PubDate: Sep 2022

Teams:  City University of Hong Kong;Beihang University;tencent

Writers: Yiming Liu; Chun Ki Yiu; Zhao Zhao; Shiyuan Liu; Xingcan Huang; Wooyoung Park; Jingyou Su; Jingkun Zhou; Tsz Hung Wong;Kuanming Yao;Ling Zhao;Ya Huang;Jiyu Li;Pu Fan;Binbin Zhang;Yuan Dai;Zhenbao Yang;Yuhang Li;Xinge Yu;

PDFSkin-integrated Haptic Interfaces enabled by Scalable Mechanical Actuators for Virtual Reality

Abstract

The very recent concept of metaverse highlights the importance of virtual reality (VR) and augmented reality (AR), that associates with a wide variety of applications in entertainment, medical treatment, and human machine interfaces. The current VR/AR technologies mainly rely on visual interaction, while immersive experience in VR and AR highly demands sensational feedback, such as haptic and temperature with noticeable quality in wearable or even skin-integrated formarts. In this paper, we report a wearable and flexible haptic interface based on electromagnetic vibrotactile actuators with high wearability and stability. By adopting double layers of copper (Cu) coils at the top and bottom of the magnetic disc, an enhanced electromagnetic field can be generated. Additionally, the intensity of the haptic feedback can be modulated according to sensed pressure in the virtual world by adjusting the value of power input and frequency. The actuator exhibits high stability and tolerance upon environmental, cyclic, and impact resistance tests. Finally, the actuators are developed into the soft VR interfaces for mounting on forearms, fingers and hands to verify their superiority over conventional haptic actuators in the aspects of performance and applications.

Skin-integrated Haptic Interfaces enabled by Scalable Mechanical Actuators for Virtual Reality最先出现在Nweon Paper

]]>
Gaze-Vergence-Controlled See-Through Vision in Augmented Reality https://paper.nweon.com/13647 Wed, 07 Dec 2022 02:16:45 +0000 https://paper.nweon.com/?p=13647 ...

Gaze-Vergence-Controlled See-Through Vision in Augmented Reality最先出现在Nweon Paper

]]>
PubDate: Sep 2022

Teams:  Beihang University

Writers: Zhimin Wang; Yuxin Zhao; Feng Lu

PDFGaze-Vergence-Controlled See-Through Vision in Augmented Reality

Abstract

Augmented Reality (AR) see-through vision is an interesting research topic since it enables users to see through a wall and see the occluded objects. Most existing research focuses on the visual effects of see-through vision, while the interaction method is less studied. However, we argue that using common interaction modalities, e.g., midair click and speech, may not be the optimal way to control see-through vision. This is because when we want to see through something, it is physically related to our gaze depth/vergence and thus should be naturally controlled by the eyes. Following this idea, this paper proposes a novel gaze-vergence-controlled (GVC) see-through vision technique in AR. Since gaze depth is needed, we build a gaze tracking module with two infrared cameras and the corresponding algorithm and assemble it into the Microsoft HoloLens 2 to achieve gaze depth estimation. We then propose two different GVC modes for see-through vision to fit different scenarios. Extensive experimental results demonstrate that our gaze depth estimation is efficient and accurate. By comparing with conventional interaction modalities, our GVC techniques are also shown to be superior in terms of efficiency and more preferred by users. Finally, we present four example applications of gaze-vergence-controlled see-through vision.

Gaze-Vergence-Controlled See-Through Vision in Augmented Reality最先出现在Nweon Paper

]]>
FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality https://paper.nweon.com/13645 Tue, 06 Dec 2022 02:43:29 +0000 https://paper.nweon.com/?p=13645 ...

FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Shanghai Jiao Tong University;New York University; UNC

Writers: Nianchen Deng; Zhenyi He; Jiannan Ye; Budmonde Duinkharjav; Praneeth Chakravarthula; Xubo Yang; Qi Sun

PDFFoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality

Abstract

Virtual Reality (VR) is becoming ubiquitous with the rise of consumer displays and commercial VR platforms. Such displays require low latency and high quality rendering of synthetic imagery with reduced compute overheads. Recent advances in neural rendering showed promise of unlocking new possibilities in 3D computer graphics via image-based representations of virtual or physical environments. Specifically, the neural radiance fields (NeRF) demonstrated that photo-realistic quality and continuous view changes of 3D scenes can be achieved without loss of view-dependent effects. While NeRF can significantly benefit rendering for VR applications, it faces unique challenges posed by high field-of-view, high resolution, and stereoscopic/egocentric viewing, typically causing low quality and high latency of the rendered images. In VR, this not only harms the interaction experience but may also cause sickness. To tackle these problems toward six-degrees-of-freedom, egocentric, and stereo NeRF in VR, we present the first gaze-contingent 3D neural representation and view synthesis method. We incorporate the human psychophysics of visual- and stereo-acuity into an egocentric neural representation of 3D scenery. We then jointly optimize the latency/performance and visual quality while mutually bridging human perception and neural scene synthesis to achieve perceptually high-quality immersive interaction. We conducted both objective analysis and subjective studies to evaluate the effectiveness of our approach. We find that our method significantly reduces latency (up to 99% time reduction compared with NeRF) without loss of high-fidelity rendering (perceptually identical to full-resolution ground truth). The presented approach may serve as the first step toward future VR/AR systems that capture, teleport, and visualize remote environments in real-time.

FoV-NeRF: Foveated Neural Radiance Fields for Virtual Reality最先出现在Nweon Paper

]]>
Gait Energy Image-Based Human Attribute Recognition using Two-Branch Deep Convolutional Neural Network https://paper.nweon.com/13643 Tue, 06 Dec 2022 02:21:14 +0000 https://paper.nweon.com/?p=13643 ...

Gait Energy Image-Based Human Attribute Recognition using Two-Branch Deep Convolutional Neural Network最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Beihang University

Writers: Shaoxiong Zhang; Yunhong Wang; Annan Li

PDFGait Energy Image-Based Human Attribute Recognition using Two-Branch Deep Convolutional Neural Network

Abstract

Gait is an attractive biometric identifier, playing an essential role in addressing the issue of identity and attribute recognition in surveillance for its non-invasive and non-cooperative features. In this study, we propose a two-branch deep convolutional neural network for gait-based attribute recognition, including age estimation and gender recognition. We improve the estimation module by predicting a joint distribution instead of two independent distributions. In addition, several improvements are also proposed for improving the final performance of human attribute recognition, including data augmentation methods and loss functions. We implement several gait-based attribute recognition experiments on the OULP-Age and OU-MVLP datasets. Experimental results show that the proposed method outperforms existing approaches. Finally, we elicit different body regions’ contributions on attribute recognition tasks. Our conclusions can help improve the robustness of gait-based human attribute recognition systems in future.

Gait Energy Image-Based Human Attribute Recognition using Two-Branch Deep Convolutional Neural Network最先出现在Nweon Paper

]]>
Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force https://paper.nweon.com/13641 Mon, 05 Dec 2022 07:52:04 +0000 https://paper.nweon.com/?p=13641 ...

Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Harbin Institute of Technology;Ningbo Institute of Intelligent Equipment Technology;University of Vigo

Writers: Yongqing Fu; Weiyang Lin; Xinghu Yu; Juan J. Rodríguez-Andina; Huijun Gao

PDFRobot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force

Abstract

This study presents a novel method to realize an augmented reality (AR) and haptic feedback teleoperation ultrasound system, which can reduce the influence of time delay on the operator. The physician uses a haptic device to send commands to a remote site while observing the AR environment. In the virtual environment, the organ model is projected onto the corresponding position according to the human pose, and the UR robot model can respond to the physician’s action in advance. Based on force data, the dynamic environmental model is updated by the recursive least-squares method, allowing real-time force feedback. The contact force is computed from the identified parameters together with the position and velocity of the master site. An experimental platform has been built, where experiments have been conducted to evaluate the overall architecture, highlighting the role of AR and contact-force prediction models, and including clinical validation. Results show that the system can correctly perform remote ultrasound scanning tasks and ensure authenticity and synchronization.

Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force最先出现在Nweon Paper

]]>
Real-Time Construction of Stereoscopic Video Frames using Three Omnidirectional Cameras https://paper.nweon.com/13639 Mon, 05 Dec 2022 06:21:07 +0000 https://paper.nweon.com/?p=13639 ...

Real-Time Construction of Stereoscopic Video Frames using Three Omnidirectional Cameras最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  İzmir Yüksek Teknoloji Enstitüsü

Writers: Mehmet Çali; Şevket Gümüştekİn

PDFReal-Time Construction of Stereoscopic Video Frames using Three Omnidirectional Cameras

Abstract

There are many studies about acquisition of monoscopic and stereoscopic panorama in the literature. Since obtaining motion parallax accurately without having distortions is a very challenging problem, especially 360-degree stereoscopic image and video capturing has become a prevalent research topic. However, studies in this topic have focused on costly systems with many cameras and high processing power demand. In this study, which presents an efficient solution in terms of processing power and cost, stereoscopic frames are processed in real time using three consumer grade 360-degree cameras whose outputs are sampled according to view orientation. Besides, a method is developed to eliminate the distortions around the borders of the field of view with the help of blending with selected auxiliary camera frames.

Real-Time Construction of Stereoscopic Video Frames using Three Omnidirectional Cameras最先出现在Nweon Paper

]]>
3D Face Animation Generation from Audio Using Convolutional Networks https://paper.nweon.com/13637 Mon, 05 Dec 2022 05:47:30 +0000 https://paper.nweon.com/?p=13637 ...

3D Face Animation Generation from Audio Using Convolutional Networks最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:   İstanbul Teknik Üniversitesi;Nowhere Studios

Writers: Turker Unlu; Arda Inceoglu; Erkan Özgür Yılmaz; Sanem Sarıel

PDF3D Face Animation Generation from Audio Using Convolutional Networks

Abstract

3D facial animation generation from audio problem is drawing attention as it is demanded for generating artificial characters in games and movies. In the literature, several studies address this problem. However, the generated facial animations are far away from being realistic. In this work, we represent faces with Facial Action Coding System (FACS) and collect a 37-minute-long dataset. We develop convolutional and transformer based models. It is observed that the trained model is able to generate animations that can be used in video games and virtual reality applications, even with novel speaker audio data of speakers it has never seen in the training data.

3D Face Animation Generation from Audio Using Convolutional Networks最先出现在Nweon Paper

]]>
A Machine Learning Enabled mmWave RFID for Rotational Sensing in Human Gesture Recognition and Motion Capture Applications https://paper.nweon.com/13635 Mon, 05 Dec 2022 05:22:29 +0000 https://paper.nweon.com/?p=13635 ...

A Machine Learning Enabled mmWave RFID for Rotational Sensing in Human Gesture Recognition and Motion Capture Applications最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Georgia Institute of Technology;Atheraxon

Writers: Ajibayo Adeyeye; Charles Lynch; Jimmy Hester; Manos Tentzeris

PDFA Machine Learning Enabled mmWave RFID for Rotational Sensing in Human Gesture Recognition and Motion Capture Applications

Abstract

In the coming years, augmented reality (AR) and virtual reality (VR) based applications will become common place. The proliferation of radar technology and the strong performance of millimeter wave backscatter have presented a unique opportunity to develop low-cost and low-power solutions to support the advent of AR/VR. In this effort, the authors present a first of its kind millimeter wave backscatter RFID for rotational sensing. The novel RFID tag design employed takes advantage of the polarization mismatch of linearly polarized antennas as the angle between the pair is varied. A supervised learning algorithm is used to achieve extremely high accuracy <1° over an unambiguous range of ±90 ° thus opening the door for potential use in a wide variety of real-time applications.

A Machine Learning Enabled mmWave RFID for Rotational Sensing in Human Gesture Recognition and Motion Capture Applications最先出现在Nweon Paper

]]>
Control, Sensor Calibration, and Parasitic Torque Cancellation of a Dual-Rotor Haptic Actuator https://paper.nweon.com/13631 Mon, 05 Dec 2022 02:41:03 +0000 https://paper.nweon.com/?p=13631 ...

Control, Sensor Calibration, and Parasitic Torque Cancellation of a Dual-Rotor Haptic Actuator最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  University of Pittsburgh

Writers: Andrew Oldiges; Nikhil Bajaj

PDFControl, Sensor Calibration, and Parasitic Torque Cancellation of a Dual-Rotor Haptic Actuator

Abstract

Vibrotactile haptic sensations are commonly achieved by a Linear Resonant Actuator (LRA) or an Eccentric Rotating Mass (ERM). ERMs dominated the early years of the vibrotactile haptic market in large part due to their simplicity and low cost. An ERM produces output proportional to the square of its angular velocity. To achieve a desired force output, the device must accelerate to the associated velocity. In typical open-loop methods of ERM control, this acceleration takes some finite time, a limit on responsiveness. This limitation has led to increased emphasis on LRAs as manufacturers seek improved responsiveness in devices such as smart phones, game controllers, and interfaces for virtual reality systems. Another limitation of ERMs is that they cannot decouple the amplitude of output from the frequency of vibration, despite both being relevant to haptic perception. This work addresses these two deficiencies by presenting a novel closed-loop system comprised of two ERMs. Output is controlled by cascaded velocity and phase control loops–the velocities of the motors controls the output frequency, and the relative phase controls the output amplitude, decoupling the two outputs. Also, changes in relative phase can be performed faster than starting up the motor from rest, improving response time. In this implementation, the switching time from a cancellation state to maximum output is faster than thirty milliseconds. Also introduced are novel counter-balances to cancel parasitic torque, and learning algorithms that leverage symmetry and accelerometer measurements to calibrate the relative alignment of the eccentric masses with sensors.

Control, Sensor Calibration, and Parasitic Torque Cancellation of a Dual-Rotor Haptic Actuator最先出现在Nweon Paper

]]>
Towards a Robust Steerability Magnetic Catheter with Haptic Force Feedback and Virtual Reality https://paper.nweon.com/13629 Mon, 05 Dec 2022 02:15:31 +0000 https://paper.nweon.com/?p=13629 ...

Towards a Robust Steerability Magnetic Catheter with Haptic Force Feedback and Virtual Reality最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  University of Orléans;Junia HEI;Junia HEI

Writers: Hanaâ Elfakir; Nabil Amari; Masil Attou; Karim Belharet

PDFTowards a Robust Steerability Magnetic Catheter with Haptic Force Feedback and Virtual Reality

Abstract

The catheter that integrates electronic components for sensing and actuation presents some limits due to the mis-leading information consequently inducing delays and instability. Besides, the relatively high cost and the fixed dimension, strongly hold back their miniaturization for an implementation in new biomedical applications like targeted delivery. We propose a hap-tic architecture combined Virtual Reality (VR) for the magnetic navigation of catheter inside a vascular network that facilitates basic navigation with a high precision. VR was incorporated into the system, allowing the user to control the catheter from a more intuitive standpoint and provide a description of the nature, texture and intensity of collision in real-time. The kinematics, dynamics and the contact forces derived respectively from the modified Denavit-Hartenberg (DH), Euler-Lagrange and spring-damper are used to establish the closed loop inverse dynamic control of the virtual model which is validated to support the real prototype being controlled in open loop.

Towards a Robust Steerability Magnetic Catheter with Haptic Force Feedback and Virtual Reality最先出现在Nweon Paper

]]>
Collision-Aware AR Telemanipulation Using Depth Mesh https://paper.nweon.com/13624 Thu, 01 Dec 2022 07:54:18 +0000 https://paper.nweon.com/?p=13624 ...

Collision-Aware AR Telemanipulation Using Depth Mesh最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Kyushu Institute of Technology

Writers: Chanapol Piyavichayanon; Masanobu Koga; Eiji Hayashi; Sakmongkon Chumkamon

PDFCollision-Aware AR Telemanipulation Using Depth Mesh

Abstract

Remotely operating a robot in Augmented Reality (AR) is a challenging problem due to the limited information about the environment around the robot. The current AR teleoperation interface lacks the collision checking between the virtual robot model and the environment. This work aims to overcome that problem by using depth mesh generation to reconstruct the environment from a single pair of RGB and Depth images. By presenting the generated mesh with the virtual manipulator model in AR, we introduce three collision-aware features, i.e., collision checking, AR guidance, and ray casting distance calculation, to support the operator in the manipulation task. The reconstruction can be done instantly on the smartphone, allowing the system to be used on mobile AR applications. We evaluate our system with the pick-and-place task. The accuracy of the reconstruction is enough for the user to succeed in the operation. In addition, the collision-aware features reduce the task completion time, lower workload, and enhance the system’s usability.

Collision-Aware AR Telemanipulation Using Depth Mesh最先出现在Nweon Paper

]]>
Uncoupled Stability of Kinesthetic Haptic Systems Simulating Mass-Damper-Spring Environments with Complementary Filter https://paper.nweon.com/13622 Thu, 01 Dec 2022 07:26:40 +0000 https://paper.nweon.com/?p=13622 ...

Uncoupled Stability of Kinesthetic Haptic Systems Simulating Mass-Damper-Spring Environments with Complementary Filter最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Queen’s University

Writers: Leonam Pecly; Keyvan Hashtrudi-Zaad

PDFUncoupled Stability of Kinesthetic Haptic Systems Simulating Mass-Damper-Spring Environments with Complementary Filter

Abstract

Uncoupled stability, the condition by which the user is not in contact with the haptic device, is arguably a stringent stability condition for haptic simulation systems. Uncoupled stability of haptic systems simulating linear mass-spring or viscoelastic virtual environments have been analyzed. In this paper, we analytically and experimentally evaluate uncoupled stability for simulating mass-damper-spring virtual environments when only position or when position and velocity are available. In addition, the effect of using a linear combination of position and velocity in deriving acceleration estimate is also studied. Experimental results in a one degree-of-freedom device showed that the highest stiffness values are obtained when the acceleration is equally derived from position and velocity. This work will shed light on the interaction of the three dynamic components for virtual environment rendering.

Uncoupled Stability of Kinesthetic Haptic Systems Simulating Mass-Damper-Spring Environments with Complementary Filter最先出现在Nweon Paper

]]>
User Navigation Modeling, Rate-Distortion Analysis, and End-to-End Optimization for Viewport-Driven 360∘ Video Streaming https://paper.nweon.com/13620 Thu, 01 Dec 2022 06:47:25 +0000 https://paper.nweon.com/?p=13620 ...

User Navigation Modeling, Rate-Distortion Analysis, and End-to-End Optimization for Viewport-Driven 360∘ Video Streaming最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  EPFL;Cesson-Sévigné;Synmedia;Adobe

Writers: Jacob Chakareski; Xavier Corbillon; Gwendal Simon; Vishwanathan Swaminathan

PDFUser Navigation Modeling, Rate-Distortion Analysis, and End-to-End Optimization for Viewport-Driven 360∘ Video Streaming

Abstract

The emerging technologies of Virtual Reality (VR) and 360∘ video introduce new challenges for state-of-the-art video communication systems. Enormous data volume and spatial user navigation are unique characteristics of 360∘ videos that necessitate a space-time effective allocation of the available network streaming bandwidth over the 360∘ video content to maximize the Quality of Experience (QoE) delivered to the user. Towards this objective, we investigate a framework for viewport-driven rate-distortion optimized 360∘ video streaming that integrates the user view navigation patterns and the spatiotemporal rate-distortion characteristics of the 360∘ video content to maximize the delivered user viewport video quality, for the given network/system resources. The framework comprises a methodology for assigning dynamic navigation likelihoods over the 360∘ video spatiotemporal panorama, induced by the user navigation patterns, an analysis and characterization of the 360∘ video panorama’s spatiotemporal rate-distortion characteristics that leverage preprocessed spatial tilling of the content, and an optimization problem formulation and solution that capture and aim to maximize the delivered expected viewport video quality, given a user’s navigation patterns, the 360∘ video encoding/streaming decisions, and the available system/network resources. We formulate a Markov model to capture the navigation patterns of a user over the 360∘ video panorama and simultaneously extend our actual navigation datasets by synthesizing additional realistic navigation data. Moreover, we investigate the impact of using two different tile sizes for equirectangular tiling of the 360∘ video panorama. Our experimental results demonstrate the advantages of our framework over the conventional approach of streaming a monolithic uniformly-encoded 360∘ video and a state-of-the-art navigation-speed based reference method.

User Navigation Modeling, Rate-Distortion Analysis, and End-to-End Optimization for Viewport-Driven 360∘ Video Streaming最先出现在Nweon Paper

]]>
AIParsing: Anchor-Free Instance-Level Human Parsing https://paper.nweon.com/13618 Thu, 01 Dec 2022 06:23:37 +0000 https://paper.nweon.com/?p=13618 ...

AIParsing: Anchor-Free Instance-Level Human Parsing最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Tianjin University;Sun Yat-sen University;Futurewei Technologies;Tsinghua University

Writers: Sanyi Zhang; Xiaochun Cao; Guo-Jun Qi; Zhanjie Song; Jie Zhou

PDFAIParsing: Anchor-Free Instance-Level Human Parsing

Abstract

Most state-of-the-art instance-level human parsing models adopt two-stage anchor-based detectors and, therefore, cannot avoid the heuristic anchor box design and the lack of analysis on a pixel level. To address these two issues, we have designed an instance-level human parsing network which is anchor-free and solvable on a pixel level. It consists of two simple sub-networks: an anchor-free detection head for bounding box predictions and an edge-guided parsing head for human segmentation. The anchor-free detector head inherits the pixel-like merits and effectively avoids the sensitivity of hyper-parameters as proved in object detection applications. By introducing the part-aware boundary clue, the edge-guided parsing head is capable to distinguish adjacent human parts from among each other up to 58 parts in a single human instance, even overlapping instances. Meanwhile, a refinement head integrating box-level score and part-level parsing quality is exploited to improve the quality of the parsing results. Experiments on two multiple human parsing datasets ( i.e. , CIHP and LV-MHP-v2.0) and one video instance-level human parsing dataset ( i.e. , VIP) show that our method achieves the best global-level and instance-level performance over state-of-the-art one-stage top-down alternatives.

AIParsing: Anchor-Free Instance-Level Human Parsing最先出现在Nweon Paper

]]>
3D scene extraction using plane detection algorithm https://paper.nweon.com/13616 Thu, 01 Dec 2022 05:48:40 +0000 https://paper.nweon.com/?p=13616 ...

3D scene extraction using plane detection algorithm最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Technical University of Sofia

Writers: Radostina Petkova; Krasimir Tonchev; Agata Manolova; Vladimir Poulkov

PDF3D scene extraction using plane detection algorithm

Abstract

Enabling holographic type communications usually passes through the steps of objects and environment capturing, camera calibration, computer processing, data transmission, and remote side rendering. In many remote collaboration scenarios the local physical environment is meaningful to a distantly located expert. In such cases, the environment needs to be precisely recreated and virtually added. This paper presents a scenario where the interior of a dedicated controlled space with the form of a hexagonal cell is captured by three Kinect v2 sensors. Then, it is reconstructed as a single 3D object and further processed. However, the virtual representation of such a specific environment is disputed by the surroundings outside the hexagonal cell, that are also in the field of view of the cameras. The goal is to extract the virtual representation of the controlled physical space from its surroundings by proposing a devoted 3D scene extraction algorithm.

3D scene extraction using plane detection algorithm最先出现在Nweon Paper

]]>
Learning to Track Monofrequency Multiaxis 3-D Passive Magnetic Markers https://paper.nweon.com/13612 Tue, 29 Nov 2022 07:20:51 +0000 https://paper.nweon.com/?p=13612 ...

Learning to Track Monofrequency Multiaxis 3-D Passive Magnetic Markers最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Tohoku University

Writers: Yuhei Osaka; Jiawei Huang; Yoshifumi Kitamura

PDFLearning to Track Monofrequency Multiaxis 3-D Passive Magnetic Markers

Abstract

Magnetic tracking is useful in many motion tracking applications and free from the line-of-sight (occlusion) limitations that is the drawback of many popular optical tracking systems. Previously, multiaxis magnetic markers relied on time multiplex or frequency multiplex methods to separately sense inductor–capacitor (LC) coils in different axes, which introduce bandwidth or speed limitations. In this article, we propose a novel data-driven magnetic tracking approach whose markers are based on multiple LC coils in the same resonance frequency. Our approach utilizes deep learning to determine the complex flux distribution of monofrequency multiaxis magnetic markers where simulation results are the training data. Classic numerical methods struggle to achieve the same process. With this approach, we build a new magnetic tracking system that pursues multiaxis passive markers and overcomes the original tracking principle’s dead-angle limitation. With the flexibility of deep learning, we can design markers with more than three axes. Our experimental result also shows that compared to classic three-axis markers, more LC coils in different axes actually improve the tracking accuracy. Our approach not only helps build a new tracking system based on a wireless magnetic tracking principle, but can also be applied in any multiaxis magnetic tracking system.

Learning to Track Monofrequency Multiaxis 3-D Passive Magnetic Markers最先出现在Nweon Paper

]]>
Design and Development of a Novel Haptic Device with Gravity Compensation for Teleoperation https://paper.nweon.com/13610 Tue, 29 Nov 2022 06:50:33 +0000 https://paper.nweon.com/?p=13610 ...

Design and Development of a Novel Haptic Device with Gravity Compensation for Teleoperation最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Beihang University

Writers: Lingda Meng; Song Kang; Wusheng Chou

PDFDesign and Development of a Novel Haptic Device with Gravity Compensation for Teleoperation

Abstract

In the local portion of teleoperation system, haptic devices determine the reliability of the motion capture and the authenticity of virtual force feedback, which undertake the cooperation between the operator and remote environment. In order to perform large workspace of the local teleoperation robot, a novel haptic device with 6DOF force feedback is proposed in this paper. In addition, a synthetical gravity compensation system is applied for counterbalancing the gravity torques. Static balancing at arbitrary position is achieved observably which reinforces transparency in the teleoperation. Subsequently, idiographic performance including maximum exertable force and translational resolution are analyzed based on Jacobian matrix. For teleoperation application, the workspace of the master-slave based on kinematics is mapped and verified in the virtual environment. The demonstration is stably visualized to the operator and the result indicates that developed haptic device works and presents good performance.

Design and Development of a Novel Haptic Device with Gravity Compensation for Teleoperation最先出现在Nweon Paper

]]>
Saliency-Based Multiple Region of Interest Detection From a Single 360° Image https://paper.nweon.com/13608 Tue, 29 Nov 2022 06:19:49 +0000 https://paper.nweon.com/?p=13608 ...

Saliency-Based Multiple Region of Interest Detection From a Single 360° Image最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  University of Tokyo;National Institute of Informatics;

Writers: Yuuki Sawabe; Satoshi Ikehata; Kiyoharu Aizawa

PDFSaliency-Based Multiple Region of Interest Detection From a Single 360° Image

Abstract

360° images are informative – it contains omnidirectional visual information around the camera. However, the areas that cover a 360° image is much larger than the human’s field of view, therefore important information in different view directions is easily overlooked. To tackle this issue, we propose a method for predicting the optimal set of Region of Interest (RoI) from a single 360° image using the visual saliency as a clue. To deal with the scarce, strongly biased training data of existing single 360° image saliency prediction dataset, we also propose a data augmentation method based on the spherical random data rotation. From the predicted saliency map and redundant candidate regions, we obtain the optimal set of RoIs considering both the saliency within a region and the Interaction-Over-Union (IoU) between regions. We conduct the subjective evaluation to show that the proposed method can select regions that properly summarize the input 360° image.

Saliency-Based Multiple Region of Interest Detection From a Single 360° Image最先出现在Nweon Paper

]]>
Perceptual Control of Food Taste with Projection Mapping https://paper.nweon.com/13606 Tue, 29 Nov 2022 05:49:15 +0000 https://paper.nweon.com/?p=13606 ...

Perceptual Control of Food Taste with Projection Mapping最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Japan Advanced Institute of Science and Technology

Writers: Masanobu Ohyori; Ryota Takigami; Shogo Yoshida; Yichen Peng; Haoran Xie; Toshiki Sato; Kazunori Miyata

PDFPerceptual Control of Food Taste with Projection Mapping

Abstract

Spatial augmented reality techniques using projection mapping have been utilized extensively in our daily lives to augment the real world with virtual content, such as restaurant decorations that are intended to increase appetite. Although previous works have indicated that the projected visual effects on the food might change humans’ taste impressions, it is still challenging to provide perceptual control of food taste, which allows apparent taste modifications along with specified projected colors. In this study, we focused on shaved ice for a simplified verification of controlling the food taste through the projection mapping technique. Based on our user study, we verified that the proposed perceptual control is suitable for the taste control of shaved ice with the projected visual effects of different colors.

Perceptual Control of Food Taste with Projection Mapping最先出现在Nweon Paper

]]>
Investigation on Tactile Perception of Ultrasonic Haptics Devices https://paper.nweon.com/13602 Tue, 29 Nov 2022 05:18:55 +0000 https://paper.nweon.com/?p=13602 ...

Investigation on Tactile Perception of Ultrasonic Haptics Devices最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Chemnitz University of Technology

Writers: Christian Fuchs; Christian Kollatsch; Leon Gärtner; Adelina Heinz; Alexander von Kiesling; Susanne Weinhold; Franziska Klimant

PDFInvestigation on Tactile Perception of Ultrasonic Haptics Devices

Abstract

Haptics have increasingly gained importance in the field of user interface design by addressing tactile sensing in addition to visual and acoustic stimulation. Mid-air haptics allow tactile feedback without contact or wearable hardware and offer advanced control and output mechanisms as well as different sensory modalities for user interfaces (UI) and Virtual/Augmented Reality (VR/AR) applications. Using mid-air haptics virtual objects can offer tactile interaction, thus making contact-free UI interaction more viable. This paper investigates the user perception of ultrasonic haptic signals. By presenting test subjects a broad selection of haptic signals and researching the actually perceived stimuli as well as inquiring the test subjects’ thoughts about their experience, conclusions on the optimal design and application of ultrasonic haptic signals are made. This is done by comparing haptic signals of different shapes and dynamics but constant intensity and researching the subjectively perceived signal intensity and the participant’s success in detecting the correct shape. This allows better targeting of specific sensations in order to let users perceive concrete signals. It also offers insight into better design of haptic signals to achieve optimal user detection. The results of this investigation were applied in form of a demonstrator, which highlights multiple mid-air haptics and input modalities in different application scenarios.

Investigation on Tactile Perception of Ultrasonic Haptics Devices最先出现在Nweon Paper

]]>
3D Text Recognition and Localization From Point Clouds via 2D Projection and Virtual Camera https://paper.nweon.com/13600 Tue, 29 Nov 2022 04:47:38 +0000 https://paper.nweon.com/?p=13600 ...

3D Text Recognition and Localization From Point Clouds via 2D Projection and Virtual Camera最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Naval Information Warfare Center Pacific

Writers: Adrian Mai; Chelsea Mediavilla; Jane Berk; Mark Bilinski; Raymond Provost

PDF3D Text Recognition and Localization From Point Clouds via 2D Projection and Virtual Camera

Abstract

The lack of training datasets and computational complexity of 3 dimensions make text localization and recognition in point cloud environments challenging tasks, resulting in them being relatively undeveloped topics in the research community. In this paper, we introduce a method to adapt 2D text detection and recognition techniques in panoramic images with appropriate 3D mapping. This combined with heuristic methods such as a virtual camera creates a fast and efficient 3D text localization and recognition system. In real world applications, the objects of interest for a computer vision task may not be captured by the sensor from an ideal perspective; instead skewed imagery or partial occlusions are common. In a virtual 3D environment, we have full control of viewing angles and distances from the object. Therefore, by placing a virtual camera in certain positions, we can generate synthetic imagery of the object that is based on real imagery and avoids skewed views or occlusions. We use this synthetic imagery to improve the performance of text recognition when the object of interest is sufficiently close to the scanner, and hence the point density is high enough to generate quality imagery. The simplistic nature of this system is attractive and computationally inexpensive as it uses 2D data processes instead of natively 3D. The system shows promising results with over 85% accuracy for detection and localization tasks and 80% on the recognition task.

3D Text Recognition and Localization From Point Clouds via 2D Projection and Virtual Camera最先出现在Nweon Paper

]]>
Redirected Walking for Virtual Environments: Investigation and Evaluation https://paper.nweon.com/13597 Tue, 29 Nov 2022 04:14:16 +0000 https://paper.nweon.com/?p=13597 ...

Redirected Walking for Virtual Environments: Investigation and Evaluation最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Hefei University of Technology

Writers: Yuze Gao; Lin Li; Jieke Wang; Liping Zheng

PDFRedirected Walking for Virtual Environments: Investigation and Evaluation

Abstract

Redirected walking (RDW) is a significant technique to tackle the problem of limited physical space in the process of creating and experiencing an immersive virtual environment (VE). The classification and description of the RDW methods in the existing literature are biased toward the core concept, and the available evaluation system does not comprehensively consider the subjective and objective indicators. It is arduous to effectively guide the selection of solutions for practical development. This work classifies and compares the mainstream RDW technologies in virtual reality based on whether there exists a walking boundary in a VE. Four metrics from both objective and subjective aspects were selected including mean walking speed, mean reset distance, motion fidelity, and motion sickness index. An evaluation system was established based on a fuzzy comprehensive evaluation, and five groups of live user experiments were conducted to evaluate the representative RDW schemes. Sub-results analyzed the advantages and drawbacks of the RDW methods at both objective and subjective levels. Comprehensive results allowed for more intuitive outcomes that are hardly attainable with a single factor.

Redirected Walking for Virtual Environments: Investigation and Evaluation最先出现在Nweon Paper

]]>
Vibrotactile Feedback System Based on Multiple Properties Modulation https://paper.nweon.com/13595 Tue, 29 Nov 2022 02:42:14 +0000 https://paper.nweon.com/?p=13595 ...

Vibrotactile Feedback System Based on Multiple Properties Modulation最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:   Nanchang University

Writers: Pengwen Xiong; Jin Bu; Junjie Liao

PDFVibrotactile Feedback System Based on Multiple Properties Modulation

Abstract

In recent years, cross-modal retrieval and tactile rendering techniques have become a research hotspot in the virtual reality field due to their ability to provide more realistic sensations to humans. This paper presents a vibrotactile feedback system based on multiple feature modulation, which converts from texture images to tactile stimulation. We focus on tactile texture perception and propose a method to unite modulate the amplitude of raw tactile vibrations based on the physical properties of hardness and roughness to make differential modulation for different kinds of textures, and then we get driving signals that work well for the vibrotactile feedback device eventually. We designed an image-based cross-modal retrieval framework and a vibrotactile feedback device, which together with the data modulation part constitute the system that enables the modulated drive signals to output realistic tactile feedback consistent with human perception in the case of input texture images. The results of the psychophysical experiment shown that the experimenter’s recognition of texture increased from 87.5% to 93.3% comparing the virtual texture stimuli given by the vibrotactile feedback system before and after modulation.

Vibrotactile Feedback System Based on Multiple Properties Modulation最先出现在Nweon Paper

]]>
Human-UAV Collaborative Task Scheduling for 360° Video Generating in Intelligent Transportation https://paper.nweon.com/13593 Mon, 28 Nov 2022 07:53:05 +0000 https://paper.nweon.com/?p=13593 ...

Human-UAV Collaborative Task Scheduling for 360° Video Generating in Intelligent Transportation最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Dalian University of Technology

Writers: Zhiwei Yuan; Hong Tang; Pengfei Wang; Zhaohong Yan; Dongsheng Zhou; Qiang Zhang; Xiaopeng Wei

PDFHuman-UAV Collaborative Task Scheduling for 360° Video Generating in Intelligent Transportation

Abstract

With the rapid development of virtual reality technology, 360° video has been widely adopted in intelligent transportation, e.g., traffic monitoring, virtual driving, etc. Therefore, it is necessary to recruit workers and unmanned aerial vehicles (UAVs) to complete the task of video recording. However, we can not guarantee the quality of video recording by only employing workers or UAVs alone. Due to the real-time nature of traffic, task scheduling for 360 video generating is a major challenge in intelligent transportation. In this paper, we propose a human-UAV collaborative 360° video recording task scheduling framework from the perspective of crowdsensing. We build a point of interest (POI) model according to the characteristics of urban intelligent transportation to determine the locations of the video recording tasks and the number of camera stands. We introduce the relay UAV as the video streaming data transmission node to ensure the quality of video transmission and design the route planning algorithm of the relay UAV to guide the route planning of the camera UAV to ensure recording task synchronization. Meanwhile, workers are introduced into the route planning algorithm of camera UAVs to compete for tasks. The simulation experiments of the urban environment show that our proposed framework outperforms worker-mode and UAV-mode under the same environmental settings.

Human-UAV Collaborative Task Scheduling for 360° Video Generating in Intelligent Transportation最先出现在Nweon Paper

]]>
SFNet: Clothed Human 3D Reconstruction via Single Side-to-Front View RGB-D Image https://paper.nweon.com/13591 Mon, 28 Nov 2022 07:21:01 +0000 https://paper.nweon.com/?p=13591 ...

SFNet: Clothed Human 3D Reconstruction via Single Side-to-Front View RGB-D Image最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Northwestern Polytechnical University;Shadow Creator Inc;Content Production Center of Virtual Reality

Writers: Xing Li; Yangyu Fan; Di Xu; Wenqing He; Guoyun Lv; Shiya Liu

PDF:SFNet: Clothed Human 3D Reconstruction via Single Side-to-Front View RGB-D Image

Abstract

Front-view human information is critical for reconstructing a detailed 3D human body from a single RGB/RGB-D image. However, we sometimes struggle to access the front-view portrait in practice. Thus, in this work, we propose a bidirectional network (SFNet), one branch to transform side-view RGB image to front-view and another to transform side-view depth image to front-view. Since normal maps typically encode more 3D surface detail information than depth maps, we leverage an adversarial learning framework conditioned on normal maps to improve the performance of predicting front-view depth. Our method is end-to-end trainable, resulting in high fidelity front-view RGB-D estimation and 3D reconstruction.

SFNet: Clothed Human 3D Reconstruction via Single Side-to-Front View RGB-D Image最先出现在Nweon Paper

]]>
Multi-objective Optimization of Multi-video Synopsis Based on NSGA-III https://paper.nweon.com/13589 Mon, 28 Nov 2022 06:49:11 +0000 https://paper.nweon.com/?p=13589 ...

Multi-objective Optimization of Multi-video Synopsis Based on NSGA-III最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Nanjing University of Finance & Economics

Writers: Xiaoxiao Chen; Yujia Xie; Xing Wang

PDFMulti-objective Optimization of Multi-video Synopsis Based on NSGA-III

Abstract

Multi-video synopsis can display all information of surveillance video and show all sports solution of moving objects in the sight of multi-cameras. However, most of the existing multi-video synopsis can’t achieve these functions. This is because these approaches do not comprehensively consider these factors: video target display timing, maximum target display number, target display consistency, maximum moving target display density, and compression ratio. To address the comprehensive optimization problem of multiple elements for multi-video synopsis, a multi-objective optimization algorithm for multi-video synopsis is proposed. The proposed approach differs significantly from the existing methods and has several appealing properties. First, we propose five objective functions, which are consistent in time sequence, coherent in cross-camera pedestrian display, largest in number of moving objects displayed, smallest in density of moving objects, and smallest in compression ratio, then, establish a multi-objective optimization model based on the five objective functions. Second, we build a model for multi-video synopsis. Third, we use the non-dominated sorting genetic algorithm NSGA-III to optimize the multi-object model of the multi-video synopsis. Through such a solution, the poor performance of multi-video synopsis can be avoided by comprehensively considering the main influencing factors of multi-video synopsis. Besides, we present extensive experiments that demonstrate the effectiveness and efficiency of the proposed approach.

Multi-objective Optimization of Multi-video Synopsis Based on NSGA-III最先出现在Nweon Paper

]]>
Two-Stream Spatial-Temporal Fusion Graph Convolutional Network for Dynamic Gesture Recognition https://paper.nweon.com/13586 Mon, 28 Nov 2022 06:15:28 +0000 https://paper.nweon.com/?p=13586 ...

Two-Stream Spatial-Temporal Fusion Graph Convolutional Network for Dynamic Gesture Recognition最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Inner Mongolia University of Science & Technology

Writers: Ji-kai Zhang; Qi Li; Xiao-qi Lyu; Yong Liang

PDFTwo-Stream Spatial-Temporal Fusion Graph Convolutional Network for Dynamic Gesture Recognition

Abstract

As a compelling field of computer vision, dynamic gesture recognition lays the foundation for interactive interactions of virtual reality (VR) and augmented reality (AR). Compared with other body joints, hand joints feature a smaller range of movement, faster movement speed, and more movement details. It is necessary to further explore the local spatial-temporal information and global dependencies in the process of action execution. On that basis, we propose a two-stream spatial-temporal fusion graph convolutional network, 2s -STFGCN, for dynamic gesture recognition. In order to enrich detailed joint features, the second-order bone information is introduced to the model. The local spatial-temporal information is fused in the one graph to capture the complex spatial-temporal relationship. At the same time, the gated dilated convolution is employed to ensure the correlation of long sequence to be better noticed. Additionally, by simulating actions in the VR interactive applications, we collect and make the dynamic gesture skeleton data set, VR-DHG, based on different grain sizes. Experimental results suggest that the model proposed by us can achieve better recognition effects of the public data set, DHG-14/28. Compared with the DeepGRU algorithm, the recognition rate of our algorithm, when being used to recognize 14 kinds and 28 kinds of gestures, can be improved by 1.21% and 2.64%, respectively. Our algorithm also outperforms in fine-grained gesture recognition. All this provides solid evidence for the effectiveness of our algorithm.

Two-Stream Spatial-Temporal Fusion Graph Convolutional Network for Dynamic Gesture Recognition最先出现在Nweon Paper

]]>
Occluded Facial Recognition for Surviellance Using Deep Learning https://paper.nweon.com/13584 Mon, 28 Nov 2022 05:46:38 +0000 https://paper.nweon.com/?p=13584 ...

Occluded Facial Recognition for Surviellance Using Deep Learning最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:   Presidency University

Writers: Hameed Moqbel; Murali Parameswaran

PDFOccluded Facial Recognition for Surviellance Using Deep Learning

Abstract

Nowadays, due to the advancement in technology, facial recognition is becoming one of the methods to identify a person. One of the challenges arises due to occlusion or partial covering of face, especially with a facial mask or a scarf. In this work, we use deep neural networks to solve the problem of recognizing such an occluded face. For this work, we have used three publicly available facial datasets, namely Labelled Face Wild dataset, COMASK20 and Specs on Faces (with images having low illumination), cumulatively consisting more than 5000 facial images. We evaluated four existing facial detection classifiers namely OpenCV, Single Shot Detection(SSD), Multi-task Cascaded Convolutional Neural Network(MTCNN) and RetinaFace. We found MTCNN to be most relevant for our work. We proposed a new Convolutional Neural Networks (CNN) as part of this work. We got accuracy of 99.38% for LFW, 99.62% for COMASK20 and 98.33% for SOF dataset.

Occluded Facial Recognition for Surviellance Using Deep Learning最先出现在Nweon Paper

]]>
A New Dynamic Stable Treemapping Method https://paper.nweon.com/13582 Mon, 28 Nov 2022 05:19:05 +0000 https://paper.nweon.com/?p=13582 ...

A New Dynamic Stable Treemapping Method最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:   Xiangtan Institute of Technology;University of Guelph;University of Konstanz

Writers: Cong Feng; Minglun Gong; Oliver Deussen

PDFA New Dynamic Stable Treemapping Method

Abstract

Dynamic stabilility is a desired property for treemapping methods. Since in real world, the dataset is usually in a continuous way to be proceed, and the treemap we visualized will be in a dynamic flow. In this paper, We present a dynamic stable treepping method. This method has two steps, the first step is to lay out each rectangle into the treemap in a near-uniform way, and the second step is to resize each rectanle into its real size. This method is a novel method which focuses on the dynamic stable issue of treemapping methods. We use three different real world datasets to test our method, and compare our method with 14 existing methods.

A New Dynamic Stable Treemapping Method最先出现在Nweon Paper

]]>
An Efficient Anchor-Based Face Alignment Network With Transformer https://paper.nweon.com/13580 Mon, 28 Nov 2022 04:48:02 +0000 https://paper.nweon.com/?p=13580 ...

An Efficient Anchor-Based Face Alignment Network With Transformer最先出现在Nweon Paper

]]>
PubDate: Aug 2022

Teams:  Beijing Institute of Technology

Writers: Quanyu Wang; Yue Sun; Kaixiang Zhang; Uzair Saeed; Guanzhi Shen; Wenming Wang

PDF:An Efficient Anchor-Based Face Alignment Network With Transformer

Abstract

Despite significant advances have been made in facial alignment recently, face alignment remains a challenging problem due to the existence of issues like occlusion and large pose. Besides, small attention has been paid to the algorithm’s performance, efficient face landmark localization algorithm with high robustness still has room to enhance. In this work, we propose an efficient face alignment network that combines the Transformer with an anchor-based prediction method. First, we extract features of the input image by CNNs, then capture long-range relationships efficiently using Transformer encoders, at last, we use anchor points to predict landmark coordinates. We test our algorithm through experiments on WFLW, the popular face alignment benchmark. The experiments show that our algorithm can reach high accuracy with satisfactory robustness while also enjoying the high speed.

An Efficient Anchor-Based Face Alignment Network With Transformer最先出现在Nweon Paper

]]>