Nweon Paper https://paper.nweon.com 映维网,影响力虚拟现实(VR)、增强现实(AR)产业信息数据平台 Tue, 05 Jul 2022 02:49:37 +0000 en-US hourly 1 https://wordpress.org/?v=4.8.17 https://paper.nweon.com/wp-content/uploads/2021/04/nweon-icon.png Nweon Paper https://paper.nweon.com 32 32 Accommodation and Vergence Responses to Electronic Holographic Displays Compared with Those to Stereoscopic Displays https://paper.nweon.com/12481 Tue, 05 Jul 2022 02:49:37 +0000 https://paper.nweon.com/12481 PubDate: January 2022

Accommodation and Vergence Responses to Electronic Holographic Displays Compared with Those to Stereoscopic Displays最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Tokushima University;Kanazawa Institute of Technology;Tokyo University of Agriculture and Technology;National Institute of Information and Communications Technology;Advanced Telecommunications Research Institute International

Writers: Haruki Mizushina; Ippei Negishi; Junya Nakamura; Yasuhiro Takaki; Hiroshi Ando; Shinobu Masaki

PDF: Accommodation and Vergence Responses to Electronic Holographic Displays Compared with Those to Stereoscopic Displays

Abstract

Electronic holography is an ideal 3D display technique to ameliorate vergence-accommodation conflict, which is a possible cause of visual fatigue and discomfort from viewing conventional stereoscopic 3D displays. Previous studies have measured accommodative and vergence responses to holographic images and real objects, and reported that they are in good agreement. To demonstrate the effectiveness of electronic holography as a solution of vergence-accommodation conflict caused by viewing conventional stereoscopic 3D displays, we measured accommodative and vergence responses to reconstructed images of holograms and two-view stereoscopic images. We also measured responses to real objects located at various distances as a baseline condition. The results indicate that the accommodative response to the electro-holographic display changes with the vergence response in a similar manner, as is the case with real objects as compared to conventional two-view stereo display. This suggests that the holographic display technology is a promising candidate for resolving the vergence-accommodation conflict.

Accommodation and Vergence Responses to Electronic Holographic Displays Compared with Those to Stereoscopic Displays最先出现在Nweon Paper

]]>
MH Pose: 3D Human Pose Estimation based on High-quality Heatmap https://paper.nweon.com/12479 Tue, 05 Jul 2022 02:19:35 +0000 https://paper.nweon.com/12479 PubDate: January 2022

MH Pose: 3D Human Pose Estimation based on High-quality Heatmap最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Xiamen University of Technology

Writers: Huifen Zhou; Chaoqun Hong; Yong Han; Pengcheng Huang; Yanhui Zhuang

PDF: MH Pose: 3D Human Pose Estimation based on High-quality Heatmap

Abstract

Human pose estimation is a key technology in the field of human action recognition. It aims at recognizing human poses by extracting features in images or videos. At present, human body occlusion in multi-camera views is a major challenge for human body pose estimation. To tackle the problem, we proposed the MaxHeatmap Pose model. The model obtains high-quality heatmap estimation through the MH structure module, in order to locate the target person more accurately. Then the HumanPose regression network is proposed to estimate the detailed 3D human pose. The experimental results on the data set show that the performance of the model is better than previous methods.

MH Pose: 3D Human Pose Estimation based on High-quality Heatmap最先出现在Nweon Paper

]]>
Comparative Study of 3D Point Cloud Compression Methods https://paper.nweon.com/12477 Thu, 30 Jun 2022 07:49:21 +0000 https://paper.nweon.com/12477 PubDate: January 2022

Comparative Study of 3D Point Cloud Compression Methods最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: The Catholic University of America

Writers: Mai Bui; Lin-Ching Chang; Hang Liu; Qi Zhao; Genshe Chen

PDF: Comparative Study of 3D Point Cloud Compression Methods

Abstract

3D sensors such as LiDAR, stereo cameras, and radar have been used in many applications, for instance, virtual or augmented reality, real-time immersive communications, and autonomous driving systems. The output of 3D sensors is often represented in the form of point clouds. However, the massive amount of point cloud data generated from 3D sensors poses big challenges in data storage and transmission. Therefore, effective compression schemes are needed for reducing the bandwidth of wireless networks or storage space of 3D point cloud data. Several point cloud compression (PCC) algorithms have been proposed using signal processing or neural network techniques. In this study, we investigate four state-of-the-art PCC methods using two different datasets with various configurations. The objective of this study is to provide a comprehensive understanding of various approaches in PCC. The results of this paper will be helpful in developing an adaptive 3D point cloud stream compression benchmark that is efficient and benefited from different PCC techniques.

Comparative Study of 3D Point Cloud Compression Methods最先出现在Nweon Paper

]]>
FESTH: Visual Tracking with Feature Enhancement and Space-time History Frame Networks https://paper.nweon.com/12475 Thu, 30 Jun 2022 07:22:23 +0000 https://paper.nweon.com/12475 PubDate: January 2022

FESTH: Visual Tracking with Feature Enhancement and Space-time History Frame Networks最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Xiamen University of Technology

Writers: Yanhui Zhuang; Chaoqun Hong; Xuebai Zhang; Chaohui Tang; Huifen Zhou

PDF: FESTH: Visual Tracking with Feature Enhancement and Space-time History Frame Networks

Abstract

Siamese network is widely used in object tracking. However, it suffers the problem of poor generalization. The existing tracker based on template update mechanism uses complex calculation strategies and time-consuming optimization to achieve good tracking performance, but it does not meet the requirements of real-time tracking. In this paper, we first propose a tracking framework based on space-time history frames, and use feature enhancement to enrich the features of history frames. Then, we propose EnhanceNet which is an offline trained network for performing online data augmentation. It can enhance the tracking accuracy while preserving high speeds of the state-of-the-art online learning. In challenging large-scale datasets, such as LaSOT, VOT2018 and OTB-2015, our framework is better than state-of-the-art real-time trackers and has achieved excellent results while running at 32 FPS.

FESTH: Visual Tracking with Feature Enhancement and Space-time History Frame Networks最先出现在Nweon Paper

]]>
Multi-input-output Fusion Attention Module for Deblurring Networks https://paper.nweon.com/12473 Thu, 30 Jun 2022 06:55:33 +0000 https://paper.nweon.com/12473 PubDate: January 2022

Multi-input-output Fusion Attention Module for Deblurring Networks最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Xiamen University of Technology

Writers: Yiqing Fan; Chaoqun Hong; Xiaodong Wang; Zhiqiang Zeng; Zetian Guo

PDF: Multi-input-output Fusion Attention Module for Deblurring Networks

Abstract

In recent years, coarse-to-fine networks have shown good performance in image deblurring. The image details recovered better by inputting multi-scale images, while the computation will be high. To design a efficient deblurring network, we use a new coarse-to-fine strategy for image deblurring. We use UNet as the backbone network, feeding one image of different scales at each layer of the network to obtain sharp images with better details. In addition, we have designed information supplement blocks that can effectively supplement the information of different scale images. Finally, to be able to obtain better information about the image, we introduce an attention mechanism. It is experimentally shown that our network performs well on different metrics.

Multi-input-output Fusion Attention Module for Deblurring Networks最先出现在Nweon Paper

]]>
Robust Speech Emotion Recognition System Through Novel ER-CNN and Spectral Features https://paper.nweon.com/12471 Thu, 30 Jun 2022 06:22:27 +0000 https://paper.nweon.com/12471 PubDate: January 2022

Robust Speech Emotion Recognition System Through Novel ER-CNN and Spectral Features最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: University of Engineering and Technology

Writers: Muhammad Zeeshan; Huma Qayoom; Farman Hassan

PDF: Robust Speech Emotion Recognition System Through Novel ER-CNN and Spectral Features

Abstract

The speech is most fundamental way of communication among the humans and an important method for human computer interaction (HCI) by employing the microphone. Measurable emotion recognition from the speech signal by employing microphone is an emerging and interesting area of research in HCI such as human reboot interaction, healthcare, virtual reality, emergency call, and behavior assessment. In this paper, we proposed a novel integration of spectral features comprises of mel-spectral frequency coefficients (MFCC), root mean square energy (RMSE), and zero crossing rate (ZCR) to represent complex audio signal. For the classification purpose, we designed a novel convolutional neural network called emotion recognition neural network (ER-CNN) to classify different emotions such as angry, disgust, fear, happy, neutral, and sad. The proposed method Speech emotion recognition (SER-CNN) obtained an equal error rate (EER) of 1.34%, an accuracy of 94.99%, precision of 94.96%, recall of 94.98%, and F1-score of 94.96%. We evaluated the performance of the proposed system SER-CNN on the standard dataset crowd-sourced emotional multimodal actors (CREMA-D). Experimental results of the proposed method and comparative analysis against the existing methods show that our method has superior performance and can reliably be used for the emotion detection.

Robust Speech Emotion Recognition System Through Novel ER-CNN and Spectral Features最先出现在Nweon Paper

]]>
Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering https://paper.nweon.com/12469 Thu, 30 Jun 2022 05:49:22 +0000 https://paper.nweon.com/12469 PubDate: January 2022

Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: University of Surrey;AWS Tübingen

Writers: Guillaume Rochette; Chris Russell; Richard Bowden

PDF: Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering

Abstract

We present a new approach for synthesizing novel views of people in new poses. Our novel differentiable renderer enables the synthesis of highly realistic images from any viewpoint. Rather than operating over mesh-based structures, our renderer makes use of diffuse Gaussian primitives that directly represent the underlying skeletal structure of a human. Rendering these primitives gives results in a high-dimensional latent image, which is then transformed into an RGB image by a decoder network. The formulation gives rise to a fully differentiable framework that can be trained end-to-end. We demonstrate the effectiveness of our approach to image reconstruction on both the Human3.6M and Panoptic Studio datasets. We show how our approach can be used for motion transfer between individuals; novel view synthesis of individuals captured from just a single camera; to synthesize individuals from any virtual viewpoint; and to re-render people in novel poses. Code and video results are available at https://github.com/GuillaumeRochette/HumanViewSynthesis.

Human Pose Manipulation and Novel View Synthesis using Differentiable Rendering最先出现在Nweon Paper

]]>
360ViewPET: View Based Pose EsTimation for Ultra-Sparse 360-Degree Cameras https://paper.nweon.com/12467 Tue, 28 Jun 2022 07:55:35 +0000 https://paper.nweon.com/12467 PubDate: January 2022

360ViewPET: View Based Pose EsTimation for Ultra-Sparse 360-Degree Cameras最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: University of Illinois Urbana-Champaign;

Writers: Qian Zhou; Bo Chen; Zhe Yang; Hongpeng Guo; Klara Nahrstedt

PDF: 360ViewPET: View Based Pose EsTimation for Ultra-Sparse 360-Degree Cameras

Abstract

Immersive virtual tours based on 360-degree cameras, showing famous outdoor scenery, are becoming more and more desirable due to travel costs, pandemics and other constraints. To feel immersive, a user must receive the view accurately corresponding to her position and orientation in the virtual space when she moves inside, and this requires cameras’ orientations to be known. Outdoor tour contexts have numerous, ultra-sparse cameras deployed across a wide area, making camera pose estimation challenging. As a result, pose estimation techniques like SLAM, which require mobile or dense cameras, are not applicable. In this paper we present a novel strategy called 360ViewPET, which automatically estimates the relative poses of two stationary, ultra-sparse (15 meters apart) 360-degree cameras using one equirectangular image taken by each camera. Our experiments show that it achieves accurate pose estimation, with a mean error as low as 0.9 degree.

360ViewPET: View Based Pose EsTimation for Ultra-Sparse 360-Degree Cameras最先出现在Nweon Paper

]]>
Head Rotation Model for Virtual Reality System Level Simulations https://paper.nweon.com/12465 Tue, 28 Jun 2022 07:25:20 +0000 https://paper.nweon.com/12465 PubDate: January 2022

Head Rotation Model for Virtual Reality System Level Simulations最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: National Institute of Standards and Technology;New Jersey Institute of Technology;York College of Pennsylvania

Writers: Steve Blandino; Tanguy Ropitault; Raied Caromi; Jacob Chakareski; Mahmudur Khan; Nada Golmie

PDF: Head Rotation Model for Virtual Reality System Level Simulations

Abstract

Virtual Reality (VR) promises immersive experiences in diverse areas such as gaming, entertainment, education, healthcare, and remote monitoring. In VR environments, users can navigate 360-degree content by moving or looking around in all directions, by rotating their heads, as in real life. A rapid head rotation can corrupt the wireless link, degrading the user experience. Due to the lack of proper head rotation models, testbeds are usually required to analyze VR systems. In this paper, we propose an open source code package that generates realistic head rotation traces. The code package is based on a simple, yet flexible, time-correlated mathematical model, which is extrapolated from a publicly available VR head rotation measurement-based dataset. We show that the probability density function of head rotation pitch and roll angles can be modeled as Gaussian distributions, while the probability density function of yaw angles can be modeled as a Gaussian mixture distribution. To introduce temporal correlation, we extrapolate the power spectral density of the angular processes, which are modeled with a bi-exponential decay. Finally, we show how the model can support and accelerate the design of future VR systems by proposing the analysis of a distributed Multiple Input Multiple Output (MIMO) system and the design of a situational awareness Machine Learning (ML) based beamforming training for millimeter wave networks.

Head Rotation Model for Virtual Reality System Level Simulations最先出现在Nweon Paper

]]>
360 Degree Video Caching with LRU & LFU https://paper.nweon.com/12463 Tue, 28 Jun 2022 06:55:24 +0000 https://paper.nweon.com/12463 PubDate: January 2022

360 Degree Video Caching with LRU & LFU最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: University of Texas at Tyler

Writers: Md Milon Uddin; Jounsup Park

PDF: 360 Degree Video Caching with LRU & LFU

Abstract

360-degree videos, which provide a means to enjoy virtual reality, have gained in popularity among people around the world. It allows users to view video scenes at any angles while watching videos. 360-degree video caching at the edge server can be a good solution to minimize the bandwidth cost and to deliver the video with less latency. Popular video contents can be divided into tiles which are cached at the edge server in a potential 360-degree video streaming system. In this research, a system architecture for 360 video caching has been proposed, and video caching has been performed using the Least Recently Used (LRU) and Least Frequently Used (LFU) algorithms. Recency and frequency are used for cache eviction. In the experiment, 48 users’ head movement data is utilized in a sequential and randomized order for two 360-degree videos, and caching is compared between the LRU cache and LFU cache by varying cache size. The results show that average cache hit rate is greater when using LFU caching as compared to LRU caching for a varying cache size.

360 Degree Video Caching with LRU & LFU最先出现在Nweon Paper

]]>
Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation https://paper.nweon.com/12461 Tue, 28 Jun 2022 06:19:19 +0000 https://paper.nweon.com/12461 PubDate: January 2022

Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Delhi Technological University

Writers: Raman Goel; Seba Susan; Sachin Vashisht; Armaan Dhanda

PDF: Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation

Abstract

Modern day conversational agents are trained to emulate the manner in which humans communicate. To emotionally bond with the user, these virtual agents need to be aware of the affective state of the user. Transformers are the recent state of the art in sequence-to-sequence learning that involves training an encoder-decoder model with word embeddings from utterance-response pairs. We propose an emotion-aware transformer encoder for capturing the emotional quotient in the user utterance in order to generate human-like empathetic responses. The contributions of our paper are as follows: 1) An emotion detector module trained on the input utterances determines the affective state of the user in the initial phase 2) A novel transformer encoder is proposed that adds and normalizes the word embedding with emotion embedding thereby integrating the semantic and affective aspects of the input utterance 3) The encoder and decoder stacks belong to the Transformer-XL architecture which is the recent state of the art in language modeling. Experimentation on the benchmark Facebook AI empathetic dialogue dataset confirms the efficacy of our model from the higher BLEU-4 scores achieved for the generated responses as compared to existing methods. Emotionally intelligent virtual agents are now a reality and inclusion of affect as a modality in all human-machine interfaces is foreseen in the immediate future.

Emotion-Aware Transformer Encoder for Empathetic Dialogue Generation最先出现在Nweon Paper

]]>
A Database for Perceived Quality Assessment of User-Generated VR Videos https://paper.nweon.com/12459 Tue, 28 Jun 2022 05:55:27 +0000 https://paper.nweon.com/12459 PubDate: Jun 2022

A Database for Perceived Quality Assessment of User-Generated VR Videos最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: Jiangxi University of Finance and Economics;City University of
Hong Kong

Writers: Yuming Fang, Yiru Yao, Xiangjie Sui, Kede Ma

PDF: A Database for Perceived Quality Assessment of User-Generated VR Videos

Abstract

Virtual reality (VR) videos (typically in the form of 360∘ videos) have gained increasing attention due to the fast development of VR technologies and the remarkable popularization of consumer-grade 360∘ cameras and displays. Thus it is pivotal to understand how people perceive user-generated VR videos, which may suffer from commingled authentic distortions, often localized in space and time. In this paper, we establish one of the largest 360∘ video databases, containing 502 user-generated videos with rich content and distortion diversities. We capture viewing behaviors (i.e., scanpaths) of 139 users, and collect their opinion scores of perceived quality under four different viewing conditions (two starting points × two exploration times). We provide a thorough statistical analysis of recorded data, resulting in several interesting observations, such as the significant impact of viewing conditions on viewing behaviors and perceived quality. Besides, we explore other usage of our data and analysis, including evaluation of computational models for quality assessment and saliency detection of 360∘ videos. We have made the dataset and code available at this https URL.

A Database for Perceived Quality Assessment of User-Generated VR Videos最先出现在Nweon Paper

]]>
i-FlatCam: A 253 FPS, 91.49 μJ/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR https://paper.nweon.com/12457 Tue, 28 Jun 2022 05:34:21 +0000 https://paper.nweon.com/12457 PubDate: Jun 2022

i-FlatCam: A 253 FPS, 91.49 μJ/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: Rice University;Me

Writers: Yang Zhao, Ziyun Li, Yonggan Fu, Yongan Zhang, Chaojian Li, Cheng Wan, Haoran You, Shang Wu, Xu Ouyang, Vivek Boominathan, Ashok Veeraraghavan, Yingyan Lin

PDF: i-FlatCam: A 253 FPS, 91.49 μJ/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR

Abstract

We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 μJ/Frame, and 6.7mm x 8.9mm x 1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.

i-FlatCam: A 253 FPS, 91.49 μJ/Frame Ultra-Compact Intelligent Lensless Camera for Real-Time and Efficient Eye Tracking in VR/AR最先出现在Nweon Paper

]]>
ARHome: Object Selection and Manipulation using Raycasting Technique with 3D-model Tracking in Handheld Augmented Reality https://paper.nweon.com/12455 Mon, 27 Jun 2022 07:52:27 +0000 https://paper.nweon.com/12455 PubDate: January 2022

ARHome: Object Selection and Manipulation using Raycasting Technique with 3D-model Tracking in Handheld Augmented Reality最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Universiti Teknologi Malaysia

Writers: Nur Ameerah Binti Abdul Halim; Ajune Wanis Binti Ismail

PDF: ARHome: Object Selection and Manipulation using Raycasting Technique with 3D-model Tracking in Handheld Augmented Reality

Abstract

In Augmented Reality (AR), virtual objects are superimposed over the real world when viewed on a display device. It allows users to manipulate and engage with virtual objects. This paper explores the virtual object selection and manipulation using the raycasting technique when well suited with 3D model-based tracking technique in AR workspace of a handheld application called ARHome. This paper will explain the design stage involves in achieving the aim. It will then integrate with 3D model-based tracking. The significance of this study is the interaction technique can be applied in other fields such as education, training, commercial or simulation. Based on the results, the aim of this paper was achieved.

ARHome: Object Selection and Manipulation using Raycasting Technique with 3D-model Tracking in Handheld Augmented Reality最先出现在Nweon Paper

]]>
Human Performance Capture from Monocular Video in the Wild https://paper.nweon.com/12451 Mon, 27 Jun 2022 07:01:21 +0000 https://paper.nweon.com/12451 PubDate: January 2022

Human Performance Capture from Monocular Video in the Wild最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: ETH Zürich

Writers: Chen Guo; Xu Chen; Jie Song; Otmar Hilliges

PDF: Human Performance Capture from Monocular Video in the Wild

Abstract

Capturing the dynamically deforming 3D shape of clothed human is essential for numerous applications, including VR/AR, autonomous driving, and human-computer interaction. Existing methods either require a highly specialized capturing setup, such as expensive multi-view imaging systems, or they lack robustness to challenging body poses. In this work, we propose a method capable of capturing the dynamic 3D human shape from a monocular video featuring challenging body poses, without any additional input. We first build a 3D template human model of the subject based on a learned regression model. We then track this template model’s deformation under challenging body articulations based on 2D image observations. Our method outperforms state-of-the-art methods on an in-the-wild human video dataset 3DPW. Moreover, we demonstrate its efficacy in robustness and generalizability on videos from iPER datasets.

Human Performance Capture from Monocular Video in the Wild最先出现在Nweon Paper

]]>
SceneFormer: Indoor Scene Generation with Transformers https://paper.nweon.com/12449 Mon, 27 Jun 2022 06:22:27 +0000 https://paper.nweon.com/12449 PubDate: January 2022

SceneFormer: Indoor Scene Generation with Transformers最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Technical University of Munich

Writers: Xinpeng Wang; Chandan Yeshwanth; Matthias Nießner

PDF: SceneFormer: Indoor Scene Generation with Transformers

Abstract

We address the task of indoor scene generation by generating a sequence of objects, along with their locations and orientations conditioned on a room layout. Large-scale indoor scene datasets allow us to extract patterns from user-designed indoor scenes, and generate new scenes based on these patterns. Existing methods rely on the 2D or 3D appearance of these scenes in addition to object positions, and make assumptions about the possible relations between objects. In contrast, we do not use any appearance information, and implicitly learn object relations using the self-attention mechanism of transformers. We show that our model design leads to faster scene generation with similar or improved levels of realism compared to previous methods. Our method is also flexible, as it can be conditioned not only on the room layout but also on text descriptions of the room, using only the cross-attention mechanism of transformers. Our user study shows that our generated scenes are preferred to the state-of-the-art FastSynth scenes 53.9% and 56.7% of the time for bedroom and living room scenes, respectively. At the same time, we generate a scene in 1.48 seconds on average, 20% faster than FastSynth.

SceneFormer: Indoor Scene Generation with Transformers最先出现在Nweon Paper

]]>
RealisticHands: A Hybrid Model for 3D Hand Reconstruction https://paper.nweon.com/12447 Mon, 27 Jun 2022 06:01:21 +0000 https://paper.nweon.com/12447 PubDate: January 2022

RealisticHands: A Hybrid Model for 3D Hand Reconstruction最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: ETH Zurich;University of Haifa

Writers: Michael Seeber; Roi Poranne; Marc Polleyfeys; Martin R. Oswald

PDF: RealisticHands: A Hybrid Model for 3D Hand Reconstruction

Abstract

Estimating 3D hand meshes from RGB images robustly is a highly desirable task, made challenging due to the numerous degrees of freedom, and issues such as self-similarity and occlusions. Previous methods generally either use parametric 3D hand models or follow a model-free approach. While the former can be considered more robust, e.g. to occlusions, they are less expressive. We propose a hybrid approach, utilizing a deep neural network and differential rendering based optimization to demonstrably achieve the best of both worlds. In addition, we explore Virtual Reality (VR) as an application. Most VR headsets are nowadays equipped with multiple cameras, which we can leverage by extending our method to the egocentric stereo domain. This extension proves to be more resilient to the above mentioned issues. Finally, as a use-case, we show that the improved image-model alignment can be used to acquire the user’s hand texture, which leads to a more realistic virtual hand representation.

RealisticHands: A Hybrid Model for 3D Hand Reconstruction最先出现在Nweon Paper

]]>
Towards a Robust New Radio Compatible With XR https://paper.nweon.com/12445 Mon, 27 Jun 2022 05:22:26 +0000 https://paper.nweon.com/12445 PubDate: January 2022

Towards a Robust New Radio Compatible With XR最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: ZTE Corperation

Writers: Yuzhou Hu; Jiajun Xu; Xiaoying Ma; Mengzhu Chen; Hong Tang; Jun Xu

PDF: Towards a Robust New Radio Compatible With XR

Abstract

Extended Reality (XR), an umbrella abbreviation for Cloud Gaming/Augmented Reality/Virtual Reality, is supposed to deliver the commercial outlook for 5G networks. This paper discusses the challenges incurred by this service due to its inherent stringent requirements from a data rate, reliability and latency perspective. To address the challenges, a novel priority-based adaptive preemption/cancellation strategy is proposed to guarantee the new radio quality of service under both the multi-streams XR service and XR in concurrent transmission with other services scenarios. Simulation results demonstrate that a scheduler devised based on the principle of the proposed strategy is quite useful from a capacity perspective. In the meantime, a synthesis of the industrial consideration and vision is provided featuring the ongoing the 3rd Group Partnership Project Release-17 XR study item.

Towards a Robust New Radio Compatible With XR最先出现在Nweon Paper

]]>
Cybersickness Measurement and Evaluation During Flying a Helicopter in Different Weather Conditions in Virtual Reality https://paper.nweon.com/12443 Mon, 27 Jun 2022 04:55:33 +0000 https://paper.nweon.com/12443 PubDate: January 2022

Cybersickness Measurement and Evaluation During Flying a Helicopter in Different Weather Conditions in Virtual Reality最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Deakin University

Writers: Wadhah Al-Ashwal; Houshyar Asadi; Shady Mohamed; Shehab Alsanwy; Lars Kooijman; Darius Nahavandi; Ahmad Abu Alqumsan;Saeid Nahavandi

PDF: Cybersickness Measurement and Evaluation During Flying a Helicopter in Different Weather Conditions in Virtual Reality

Abstract

The conflicts between the perceived sensation of the different sensory systems can cause adverse effects which is known as motion sickness (MS) and the side effects of MS include nausea, dizziness, stomach awareness etc. Virtual reality sickness (also called Cybersickness or visually induced motion sickness (VIMS)) happens during exposure to a virtual environment when senses transfer conflicting sensation signals to the brain. The symptoms of Cybersickness are similar to motion sickness symptoms. The adverse effects of this common phenomenon can negatively affect the training outcome and benefits using VR, undermine users’ health and usefulness of simulators as it involves health risk and contributes to the increase of dropout rates. Therefore, to mitigate these issues, MS should be detected and measured. The primary objective of this study is to subjectively and objectively detect and quantify cybersickness level using a helicopter simulator. This study has also investigated the change in cybersickness self-reported scores in different weather conditions such as clear and stormy. Simulator sickness questionnaire (SSQ) has been employed for subjective scoring. This research also aimed to correlate SSQ scores with physiological data such as Galvanic Skin Response (GSR). The findings demonstrated that the SSQ total score (TS) has increased significantly from clear weather to stormy for the participants. There is also a positive correlation found between the change in TS and the amount of GSR but not significant.

Cybersickness Measurement and Evaluation During Flying a Helicopter in Different Weather Conditions in Virtual Reality最先出现在Nweon Paper

]]>
Subjective Evaluation of Filter- and Optimization-Based Motion Cueing Algorithms for a Hybrid Kinematics Driving Simulator https://paper.nweon.com/12441 Mon, 27 Jun 2022 04:19:31 +0000 https://paper.nweon.com/12441 PubDate: January 2022

Subjective Evaluation of Filter- and Optimization-Based Motion Cueing Algorithms for a Hybrid Kinematics Driving Simulator最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Heinz Nixdorf Institute

Writers: Patrick Biemelt; Sabrina Böhm; Sandra Gausemeier; Ansgar Trächtler

PDF: Subjective Evaluation of Filter- and Optimization-Based Motion Cueing Algorithms for a Hybrid Kinematics Driving Simulator

Abstract

Interactive driving simulation has become a key technology to support the development and optimization process of modern vehicle components and driver assistance systems both in academic research and in the automotive industry. However, the validity of the results obtained within the virtual environment depends essentially on the adequate reproduction of the simulated vehicle movements and the corresponding immersion of the driver. For that reason, specific motion platform control strategies, so-called Motion Cueing Algorithms (MCA), are used to replicate the simulated accelerations and angular velocities within the physical limitations of the driving simulator best possible. In this paper, we present the design and evaluation of a subjective comparison of three different filter- and optimization-based MCA resulting from previous research. For that purpose, a Human-in-the-Loop experiment was conducted with 27 participants in four typical driving situations, using a hybrid kinematics motion system as an application example. The statistical analysis of the study proves that the optimization-based algorithm is preferred by the subjects regardless of the presentation sequence and the respective driving maneuver, while the filter-based approaches differ only insignificantly in their ranking. Results thus correlate with the findings of an objective evaluation of the control quality and identify further potentials for improving the driving experience.

Subjective Evaluation of Filter- and Optimization-Based Motion Cueing Algorithms for a Hybrid Kinematics Driving Simulator最先出现在Nweon Paper

]]>
A high-accuracy, low-budget Sensor Glove for Trajectory Model Learning https://paper.nweon.com/12439 Mon, 27 Jun 2022 02:46:20 +0000 https://paper.nweon.com/12439 PubDate: January 2022

A high-accuracy, low-budget Sensor Glove for Trajectory Model Learning最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: University of Luebeck;Friedrich-Alexander-University

Writers: Robin Denz; Rabia Demirci; M. Ege Cansev; Adna Bliek; Philipp Beckerle; Elmar Rueckert; Nils Rottmann

PDF: A high-accuracy, low-budget Sensor Glove for Trajectory Model Learning

Abstract

Sensor gloves are gaining importance in tracking hand and finger movements in virtual reality applications as well as in scientific research. They introduce an unrestricted way of capturing motion without the dependence on direct line of sight as for visual tracking systems. With such sensor gloves, data of complex motion tasks can be recorded and used for modeling probabilistic trajectories or teleoperation of robotic arms. While a multitude of sensor glove designs relying on different functional principles exist, these approaches require either sensitive calibration and sensor fusion methods or complex manufacturing processes. In this paper, we propose a low-budget, yet accurate sensor glove system that uses flex sensors for fast and efficient motion tracking. We evaluate the performance of our sensor glove, such as accuracy and latency, and demonstrate the functionality by recording motion data for learning probabilistic movement models.

A high-accuracy, low-budget Sensor Glove for Trajectory Model Learning最先出现在Nweon Paper

]]>
Towards a Twisted String Actuated Haptic Device: Experimental Testing of a 2-D Virtual Environment and Teleoperation Interface https://paper.nweon.com/12437 Mon, 27 Jun 2022 02:16:19 +0000 https://paper.nweon.com/12437 PubDate: January 2022

Towards a Twisted String Actuated Haptic Device: Experimental Testing of a 2-D Virtual Environment and Teleoperation Interface最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Eindhoven University of Technology;University of Bologna

Writers: Linda Feenstra; Umberto Scarcia; Riccardo Zanella; Roberto Meattini; Davide Chiaravalli; Gianluca Palli; Claudio Melchiorri

PDF: Towards a Twisted String Actuated Haptic Device: Experimental Testing of a 2-D Virtual Environment and Teleoperation Interface

Abstract

In the article, a first stage implementation of a haptic device towards a complete 3-D workspace twisted-string actuated haptic interface is discussed. In the present work, a 2-D setup is presented, with the aim of preliminarly testing the behaviour of this novel haptic system, especially with respect to the adopted cable-based actuation solution. In particular, the component descriptions, kinematics of the planar device and the controller for teleoperation purposes are illustrated. Results regarding the behaviour of the system in rendering a virtual environment and in a robot teleoperation scenario with haptic force feedback are reported. The experimental outcomes show that the designed and implemented system is suitable for teleoperation with haptic interfaces, providing positive perspectives for the realization of the fully functional 3-D haptic interface in the future work.

Towards a Twisted String Actuated Haptic Device: Experimental Testing of a 2-D Virtual Environment and Teleoperation Interface最先出现在Nweon Paper

]]>
Dynamic Object Grasping in Human-Robot Cooperation Based on Mixed-Reality https://paper.nweon.com/12435 Mon, 27 Jun 2022 01:55:27 +0000 https://paper.nweon.com/12435 PubDate: January 2022

Dynamic Object Grasping in Human-Robot Cooperation Based on Mixed-Reality最先出现在Nweon Paper

]]>
PubDate: January 2022

Teams: Innopolis University

Writers: Albert Demian; Mikhail Ostanin; Alexandr Klimchik

PDF: Dynamic Object Grasping in Human-Robot Cooperation Based on Mixed-Reality

Abstract

Static Object grasping is a challenging task that has been studied for decades. The difficulty of the task comes back to the reason that a grasping attempt can have many solutions or due to the uncertainty about the targeted object’s features and characteristics. This makes the fact about dynamic object grasping with un-modeled dynamics even more challenging. In this paper, an approach for dynamic object grasping is presented. The approach considers human-robot handover operation where the robot should be able to track human’s holding-object hand and plan a successful grasp of the object in hand. The system was implemented with the help of Mixed-Reality using HoloLens glasses for human’s hand tracking. A serial manipulator was used to execute the operation mounted with end-effector-mounted camera to perform computer vision operations for grasp planning and correction. The main task is robot at random configuration can be able to find hand-holding object and plan grasp on object in hand. The implemented system shows success and was able to perform most of the grasping tasks successfully.

Dynamic Object Grasping in Human-Robot Cooperation Based on Mixed-Reality最先出现在Nweon Paper

]]>
DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers https://paper.nweon.com/12433 Mon, 27 Jun 2022 01:22:19 +0000 https://paper.nweon.com/12433 PubDate: Jun 2022

DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: KAIST

Writers: Mustafa B. Yaldiz, Andreas Meuleman, Hyeonjoong Jang, Hyunho Ha, Min H. Kim

PDF: DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers

Abstract

Fiducial markers have been broadly used to identify objects or embed messages that can be detected by a camera. Primarily, existing detection methods assume that markers are printed on ideally planar surfaces. Markers often fail to be recognized due to various imaging artifacts of optical/perspective distortion and motion blur. To overcome these limitations, we propose a novel deformable fiducial marker system that consists of three main parts: First, a fiducial marker generator creates a set of free-form color patterns to encode significantly large-scale information in unique visual codes. Second, a differentiable image simulator creates a training dataset of photorealistic scene images with the deformed markers, being rendered during optimization in a differentiable manner. The rendered images include realistic shading with specular reflection, optical distortion, defocus and motion blur, color alteration, imaging noise, and shape deformation of markers. Lastly, a trained marker detector seeks the regions of interest and recognizes multiple marker patterns simultaneously via inverse deformation transformation. The deformable marker creator and detector networks are jointly optimized via the differentiable photorealistic renderer in an end-to-end manner, allowing us to robustly recognize a wide range of deformable markers with high accuracy. Our deformable marker system is capable of decoding 36-bit messages successfully at ~29 fps with severe shape deformation. Results validate that our system significantly outperforms the traditional and data-driven marker methods. Our learning-based marker system opens up new interesting applications of fiducial markers, including cost-effective motion capture of the human body, active 3D scanning using our fiducial markers’ array as structured light patterns, and robust augmented reality rendering of virtual objects on dynamic surfaces.

DeepFormableTag: End-to-end Generation and Recognition of Deformable Fiducial Markers最先出现在Nweon Paper

]]>
Immersion Metrics for Virtual Reality https://paper.nweon.com/12431 Mon, 27 Jun 2022 00:58:20 +0000 https://paper.nweon.com/12431 PubDate: Jun 2022

Immersion Metrics for Virtual Reality最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: Universidad Nacional del Sur

Writers: Matias N. Selzer, Silvia M. Castro

PDF: Immersion Metrics for Virtual Reality

Abstract

Technological advances in recent years have promoted the development of virtual reality systems that have a wide variety of hardware and software characteristics, providing varying degrees of immersion. Immersion is an objective property of the virtual reality system that depends on both its hardware and software characteristics. Virtual reality systems are currently attempting to improve immersion as much as possible. However, there is no metric to measure the level of immersion of a virtual reality system based on its characteristics. To date, the influence of these hardware and software variables on immersion has only been considered individually or in small groups. The way these system variables simultaneously affect immersion has not been analyzed either. In this paper, we propose immersion metrics for virtual reality systems based on their hardware and software variables, as well as the development process that led to their formulation. From the conducted experiment and the obtained data, we followed a methodology to find immersion models based on the variables of the system. The immersion metrics presented in this work offer a useful tool in the area of virtual reality and immersive technologies, not only to measure the immersion of any virtual reality system but also to analyze the relationship and importance of the variables of these systems.

Immersion Metrics for Virtual Reality最先出现在Nweon Paper

]]>
Learning Audio-Visual Dereverberation https://paper.nweon.com/12426 Sat, 25 Jun 2022 00:10:23 +0000 https://paper.nweon.com/12426 PubDate: June 2022

Learning Audio-Visual Dereverberation最先出现在Nweon Paper

]]>
PubDate: June 2022

Teams: UT Austin,Facebook AI Research

Writers: Changan Chen, Wei Sun, David Harwath, Kristen Grauman

PDF: Learning Audio-Visual Dereverberation

Project: Learning Audio-Visual Dereverberation

Abstract

Reverberation from audio reflecting off surfaces and objects in the environment not only degrades the quality of speech for human perception, but also severely impacts the accuracy of automatic speech recognition. Prior work attempts to remove reverberation based on the audio modality only. Our idea is to learn to dereverberate speech from audio-visual observations. The visual environment surrounding a human speaker reveals important cues about the room geometry, materials, and speaker location, all of which influence the precise reverberation effects in the audio stream. We introduce Visually-Informed Dereverberation of Audio (VIDA), an end-to-end approach that learns to remove reverberation based on both the observed sounds and visual scene. In support of this new task, we develop a large-scale dataset that uses realistic acoustic renderings of speech in real-world 3D scans of homes offering a variety of room acoustics. Demonstrating our approach on both simulated and real imagery for speech enhancement, speech recognition, and speaker identification, we show it achieves state-of-the-art performance and substantially improves over traditional audio-only methods.

Learning Audio-Visual Dereverberation最先出现在Nweon Paper

]]>
Visual Acoustic Matching https://paper.nweon.com/12424 Sat, 25 Jun 2022 00:07:27 +0000 https://paper.nweon.com/12424 PubDate: Jun 2022

Visual Acoustic Matching最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: UT Austin, 2Stanford University, 3Reality Labs at Meta, 4Meta AI

Writers: Changan Chen, Ruohan Gao, Paul Calamia, Kristen Grauman

PDF: Visual Acoustic Matching

Project: Visual Acoustic Matching

Abstract

We introduce the visual acoustic matching task, in which an audio clip is transformed to sound like it was recorded in a target environment. Given an image of the target environment and a waveform for the source audio, the goal is to re-synthesize the audio to match the target room acoustics as suggested by its visible geometry and materials. To address this novel task, we propose a cross-modal transformer model that uses audio-visual attention to inject visual properties into the audio and generate realistic audio output. In addition, we devise a self-supervised training objective that can learn acoustic matching from in-the-wild Web videos, despite their lack of acoustically mismatched audio. We demonstrate that our approach successfully translates human speech to a variety of real-world environments depicted in images, outperforming both traditional acoustic matching and more heavily supervised baselines.

Visual Acoustic Matching最先出现在Nweon Paper

]]>
High-speed acoustic holography with arbitrary scattering objects https://paper.nweon.com/12422 Wed, 22 Jun 2022 23:49:20 +0000 https://paper.nweon.com/12422 PubDate: Jun 2022

High-speed acoustic holography with arbitrary scattering objects最先出现在Nweon Paper

]]>
PubDate: Jun 2022

Teams: University College London

Writers: RYUJI HIRAYAMA ;GIORGOS CHRISTOPOULOS;MARTINEZ PLASENCIA ;SRIRAM SUBRAMANIAN

PDF: High-speed acoustic holography with arbitrary scattering objects

Abstract

Recent advances in high-speed acoustic holography have enabled levitation-based volumetric displays with tactile and audio sensations. However, current approaches do not compute sound scattering of objects’ surfaces; thus, any physical object inside can distort the sound field. Here, we present a fast computational technique that allows high-speed multipoint levitation even with arbitrary sound-scattering surfaces and demonstrate a volumetric display that works in the presence of any physical object. Our technique has a two-step scattering model and a simplified levitation solver, which together can achieve more than 10,000 updates per second to create volumetric images above and below static sound-scattering objects. The model estimates transducer contributions in real time by reformulating the boundary element method for acoustic holography, and the solver creates multiple levitation traps. We explain how our technique achieves its speed with minimum loss in the trap quality and illustrate how it brings digital and physical content together by demonstrating mixed-reality interactive applications.

High-speed acoustic holography with arbitrary scattering objects最先出现在Nweon Paper

]]>
Exploration of Multi-dimensional Sensing in Human Machine Interactions https://paper.nweon.com/12418 Mon, 20 Jun 2022 07:25:29 +0000 https://paper.nweon.com/12418 PubDate: December 2021

Exploration of Multi-dimensional Sensing in Human Machine Interactions最先出现在Nweon Paper

]]>
PubDate: December 2021

Teams: National University of Singapore

Writers: Minglu Zhu; Zhongda Sun; Chengkuo Lee

PDF: Exploration of Multi-dimensional Sensing in Human Machine Interactions

Abstract

The advancements of human machine interface (HMI) with diversified sensory systems are essential to the intuitive and dexterous interaction with robotics and virtual world, which greatly affect our industrial, healthcare, and entertainment, etc. The manipulations done by human motions requires the multi-dimensional sensors to detect the complex movements. In this paper, we proposed a grating patterned sensor for both dual directional rotation sensors and tactile sensor which can be integrated into the exoskeleton and robot for motion monitoring. With the aid of the switch, the rotation, twisting, and bending sensing are achieved by the proposed designs with high customizability. The motion of upper limbs of users is projected into the virtual space or applied to control the robotics intuitively.

Exploration of Multi-dimensional Sensing in Human Machine Interactions最先出现在Nweon Paper

]]>
Analysis and Evaluation of Application Technology of Smart Transportation System https://paper.nweon.com/12416 Mon, 20 Jun 2022 07:01:21 +0000 https://paper.nweon.com/12416 PubDate:

Analysis and Evaluation of Application Technology of Smart Transportation System最先出现在Nweon Paper

]]>
PubDate:

Teams: Wuhan University

Writers: Bokang Li

PDF: Analysis and Evaluation of Application Technology of Smart Transportation System

Abstract

Smart transportation is a further development of the concept of intelligent transportation. It collects traffic information through high technology methods, conducts a large amount of data mining and model construction, to realize the systematic, real-time, and interactive characteristics of the system. There are many cutting-edge technologies that need to be used in the development and construction of smart transportation systems. The application of the Internet and block chain technology makes sure that the smart transportation system is equipped with enough safety and convenience. Big data and virtual reality technology can incorporate the experience of passers-by into the design of the transportation system. Artificial intelligence and building information modeling technology make the construction and operation of the transportation system a project that can be visually observed and predicted. With the integrating of these multiple key technologies to gradually build a complete smart transportation construction and management system, analyzing the causes of traffic problems, and moreover, providing reliable analysis results used in all aspects of transportation decision-making and citizen service, these applications will generate huge social value and economic benefits. This paper analyzes the technical characteristics, application status and development trend of intelligent transportation system construction. The main technical features of the intelligent transportation system are comprehensive perception, multi-faceted information fusion computing and intelligent decision. From a technical point of view, the comprehensive application of these technologies in the future may be reflected in the establishment of a holographic perception system and an online deduction system for reproducing and predicting traffic scenes in complex environments. At the same time, the building of a personalized smart travel service system, and the creation of smart road operations for the future of autonomous driving are also key areas for the development of smart transportation systems.

Analysis and Evaluation of Application Technology of Smart Transportation System最先出现在Nweon Paper

]]>