Nweon Paper https://paper.nweon.com 映维网,影响力虚拟现实(VR)、增强现实(AR)产业信息数据平台 Thu, 22 Jul 2021 07:31:27 +0000 en-US hourly 1 https://wordpress.org/?v=4.8.17 https://paper.nweon.com/wp-content/uploads/2021/04/nweon-icon.png Nweon Paper https://paper.nweon.com 32 32 Interactive Characters for Virtual Reality Stories https://paper.nweon.com/10679 Thu, 22 Jul 2021 07:31:27 +0000 https://paper.nweon.com/10679 PubDate: June 2021

Interactive Characters for Virtual Reality Stories最先出现在Nweon Paper

]]>
PubDate: June 2021

Teams: Artanim Foundation

Writers: Joan Llobera;Caecilia Charbonnier

PDF: Interactive Characters for Virtual Reality Stories

Abstract

Virtual Reality (VR) content production is now a flourishing industry. The specifics of VR, as opposed to videogames or movies, allow for a content format where users experience, at the same time, the narrative richness characteristic of movies and theatre plays with interactive engagement. To create such a content format some technical challenges still need to be solved, the main being the need for a new generation of animation engines that can deliver interactive characters appropriate for narrative-focused VR interactive content. We review the main assumptions of this approach and recent progress in interactive character animation techniques that seems promising to realise this goal.

Interactive Characters for Virtual Reality Stories最先出现在Nweon Paper

]]>
Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality https://paper.nweon.com/10677 Thu, 22 Jul 2021 07:13:44 +0000 https://paper.nweon.com/10677 PubDate: June 2021

Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality最先出现在Nweon Paper

]]>
PubDate: June 2021

Teams: Clemson University

Writers: Guo Freeman;Dane Acena

PDF: Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality

Abstract

This paper focuses on how the emerging social VR systems, as new and unique social interaction spaces that afford high-fidelity and multidimensional physical presence, may support building interpersonal relationships in a more nuanced, immersive, and embodied way. Based on 30 interviews, our investigation focuses on 1) the main reasons why people build and foster interpersonal relationships in social VR; 2) various novel activities through which users can foster relationships in social VR; and 3) the complicated influences of social VR mediated relationships on users’ online and offline social lives. We contribute to better understanding mediated interactive experiences by shedding light on the novel role of social VR in transforming how people meet, interact, and establish connections with others compared to other forms of media. We also provide potential directions to inform the design of future social VR systems to better afford healthy, fulfilling, and supportive interpersonal relationships.

Hugging from A Distance: Building Interpersonal Relationships in Social Virtual Reality最先出现在Nweon Paper

]]>
Effects of Task Type and Wall Appearance on Collision Behavior in Virtual Environments https://paper.nweon.com/10675 Thu, 22 Jul 2021 06:46:20 +0000 https://paper.nweon.com/10675 PubDate: Jul 2021

Effects of Task Type and Wall Appearance on Collision Behavior in Virtual Environments最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: University of Duisburg-Essen

Writers: Sebastian Cmentowski, Jens Krüger

PDF: Effects of Task Type and Wall Appearance on Collision Behavior in Virtual Environments

Abstract

Driven by the games community, virtual reality setups have lately evolved into affordable and consumer-ready mobile headsets. However, despite these promising improvements, it remains challenging to convey immersive and engaging VR games as players are usually limited to experience the virtual world by vision and hearing only. One prominent example of such open challenges is the disparity between the real surroundings and the virtual environment. As virtual obstacles usually do not have a physical counterpart, players might walk through walls enclosing the level. Thus, past research mainly focussed on multisensory collision feedback to deter players from ignoring obstacles. However, the underlying causative reasons for such unwanted behavior have mostly remained unclear.
Our work investigates how task types and wall appearances influence the players’ incentives to walk through virtual walls. Therefore, we conducted a user study, confronting the participants with different task motivations and walls of varying opacity and realism. Our evaluation reveals that players generally adhere to realistic behavior, as long as the experience feels interesting and diverse. Furthermore, we found that opaque walls excel in deterring subjects from cutting short, whereas different degrees of realism had no significant influence on walking trajectories. Finally, we use collected player feedback to discuss individual reasons for the observed behavior.

Effects of Task Type and Wall Appearance on Collision Behavior in Virtual Environments最先出现在Nweon Paper

]]>
Evaluating Performance and Gameplay of Virtual Reality Sickness Techniques in a First-Person Shooter Game https://paper.nweon.com/10673 Thu, 22 Jul 2021 06:19:33 +0000 https://paper.nweon.com/10673 PubDate: Jul 2021

Evaluating Performance and Gameplay of Virtual Reality Sickness Techniques in a First-Person Shooter Game最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Xi’an Jiaotong-Liverpool University;LaTrobe University;Birmingham City University

Writers: Diego Monteiro, Hao Chen, Hai-Ning Liang, Huawei Tu, Henry Dub

PDF: Evaluating Performance and Gameplay of Virtual Reality Sickness Techniques in a First-Person Shooter Game

Abstract

In virtual reality (VR) games, playability and immersion levels are important because they affect gameplay, enjoyment, and performance. However, they can be adversely affected by VR sickness (VRS) symptoms. VRS can be minimized by manipulating users’ perception of the virtual environment via the head-mounted display (HMD). One extreme example is the Teleport mitigation technique, which lets users navigate discretely, skipping sections of the virtual space. Other techniques are less extreme but still rely on controlling what and how much users see via the HMD. This research examines the effect on players’ performance and gameplay of these mitigation techniques in fast-paced VR games. Our focus is on two types of visual reduction techniques. This study aims to identify specifically the trade-offs these techniques have in a first-person shooter game regarding immersion, performance, and VRS. The main contributions in this paper are (1) a deeper understanding of one of the most popular techniques (Teleport) when it comes to gameplay; (2) the replication and validation of a novel VRS mitigation technique based on visual reduction; and (3) a comparison of their effect on players’ performance and gameplay.

Evaluating Performance and Gameplay of Virtual Reality Sickness Techniques in a First-Person Shooter Game最先出现在Nweon Paper

]]>
Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance https://paper.nweon.com/10671 Thu, 22 Jul 2021 06:01:20 +0000 https://paper.nweon.com/10671 PubDate: Jul 2021

Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Xi’an Jiaotong-Liverpool University;The University of Liverpool;

Writers: Yiming Luo, Jialin Wang, Hai-Ning Liang, Shan Luo, Eng Gee Lim

PDF: Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance

Abstract

Virtual reality (VR) head-mounted displays (HMD) have recently been used to provide an immersive, first-person vision/view in real-time for manipulating remotely-controlled unmanned ground vehicles (UGV). The teleoperation of UGV can be challenging for operators when it is done in real time. One big challenge is for operators to perceive quickly and rapidly the distance of objects that are around the UGV while it is moving. In this research, we explore the use of monoscopic and stereoscopic views and display types (immersive and non-immersive VR) for operating vehicles remotely. We conducted two user studies to explore their feasibility and advantages. Results show a significantly better performance when using an immersive display with stereoscopic view for dynamic, real-time navigation tasks that require avoiding both moving and static obstacles. The use of stereoscopic view in an immersive display in particular improved user performance and led to better usability.

Monoscopic vs. Stereoscopic Views and Display Types in the Teleoperation of Unmanned Ground Vehicles for Object Avoidance最先出现在Nweon Paper

]]>
SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance https://paper.nweon.com/10669 Thu, 22 Jul 2021 05:31:19 +0000 https://paper.nweon.com/10669 PubDate: Jul 2021

SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: University of Toronto;Sunnybrook Health Sciences Centre;Sunnybrook Research Institute

Writers: Mitchell Doughty, Karan Singh, Nilesh R. Ghugre

PDF: SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance

Abstract

We present SurgeonAssist-Net: a lightweight framework making action-and-workflow-driven virtual assistance, for a set of predefined surgical tasks, accessible to commercially available optical see-through head-mounted displays (OST-HMDs). On a widely used benchmark dataset for laparoscopic surgical workflow, our implementation competes with state-of-the-art approaches in prediction accuracy for automated task recognition, and yet requires 7.4x fewer parameters, 10.2x fewer floating point operations per second (FLOPS), is 7.0x faster for inference on a CPU, and is capable of near real-time performance on the Microsoft HoloLens 2 OST-HMD. To achieve this, we make use of an efficient convolutional neural network (CNN) backbone to extract discriminative features from image data, and a low-parameter recurrent neural network (RNN) architecture to learn long-term temporal dependencies. To demonstrate the feasibility of our approach for inference on the HoloLens 2 we created a sample dataset that included video of several surgical tasks recorded from a user-centric point-of-view. After training, we deployed our model and cataloged its performance in an online simulated surgical scenario for the prediction of the current surgical task. The utility of our approach is explored in the discussion of several relevant clinical use-cases. Our code is publicly available at this https URL.

SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance最先出现在Nweon Paper

]]>
Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion https://paper.nweon.com/10667 Thu, 22 Jul 2021 05:10:35 +0000 https://paper.nweon.com/10667 PubDate: Jul 2021

Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Ryerson University

Writers: Zeeshan Ahmad, Suha Rabbani, Muhammad Rehman Zafar, Syem Ishaque, Sridhar Krishnan, Naimul Khan

PDF: Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion

Abstract

ECG is an attractive option to assess stress in serious Virtual Reality (VR) applications due to its non-invasive nature. However, the existing Machine Learning (ML) models perform poorly. Moreover, existing studies only perform a binary stress assessment, while to develop a more engaging biofeedback-based application, multi-level assessment is necessary. Existing studies annotate and classify a single experience (e.g. watching a VR video) to a single stress level, which again prevents design of dynamic experiences where real-time in-game stress assessment can be utilized. In this paper, we report our findings on a new study on VR stress assessment, where three stress levels are assessed. ECG data was collected from 9 users experiencing a VR roller coaster. The VR experience was then manually labeled in 10-seconds segments to three stress levels by three raters. We then propose a novel multimodal deep fusion model utilizing spectrogram and 1D ECG that can provide a stress prediction from just a 1-second window. Experimental results demonstrate that the proposed model outperforms the classical HRV-based ML models (9% increase in accuracy) and baseline deep learning models (2.5% increase in accuracy). We also report results on the benchmark WESAD dataset to show the supremacy of the model.

Multi-level Stress Assessment from ECG in a Virtual Reality Environment using Multimodal Fusion最先出现在Nweon Paper

]]>
Investigating the Sense of Presence Between Handcrafted and Panorama Based Virtual Environments https://paper.nweon.com/10665 Thu, 22 Jul 2021 04:22:21 +0000 https://paper.nweon.com/10665 PubDate: Jul 2021

Investigating the Sense of Presence Between Handcrafted and Panorama Based Virtual Environments最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: TU Kaiserslautern;German Research Center for Artificial Intelligence

Writers: Alexander Schäfer, Gerd Reis, Didier Stricker

PDF: Investigating the Sense of Presence Between Handcrafted and Panorama Based Virtual Environments

Abstract

Virtual Reality applications are becoming increasingly mature. The requirements and complexity of such systems is steadily increasing. Realistic and detailed environments are often omitted in order to concentrate on the interaction possibilities within the application. Creating an accurate and realistic virtual environment is not a task for laypeople, but for experts in 3D design and modeling. To save costs and avoid hiring experts, panorama images are often used to create realistic looking virtual environments. These images can be captured and provided by non-experts. Panorama images are an alternative to handcrafted 3D models in many cases because they offer immersion and a scene can be captured in great detail with the touch of a button. This work investigates whether it is advisable to recreate an environment in detail by hand or whether it is recommended to use panorama images for virtual environments in certain scenarios. For this purpose, an interactive virtual environment was created in which a handmade 3D environment is almost indistinguishable from an environment created with panorama images. Interactive elements were added and a user study was conducted to investigate the effect of both environments to the user. The study conducted indicates that panorama images can be a useful substitute for 3D modeled environments.

Investigating the Sense of Presence Between Handcrafted and Panorama Based Virtual Environments最先出现在Nweon Paper

]]>
An Overview of Low latency for Wireless Communications: an Evolutionary Perspective https://paper.nweon.com/10663 Thu, 22 Jul 2021 03:01:22 +0000 https://paper.nweon.com/10663 PubDate: Jul 2021

An Overview of Low latency for Wireless Communications: an Evolutionary Perspective最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Jiaotong University

Writers: Xin Fan, Yan Huo

PDF: An Overview of Low latency for Wireless Communications: an Evolutionary Perspective

Abstract

Ultra-low latency supported by the fifth generation (5G) give impetus to the prosperity of many wireless network applications, such as autonomous driving, robotics, telepresence, virtual reality and so on. Ultra-low latency is not achieved in a moment, but requires long-term evolution of network structure and key enabling communication technologies. In this paper, we provide an evolutionary overview of low latency in mobile communication systems, including two different evolutionary perspectives: 1) network architecture; 2) physical layer air interface technologies. We firstly describe in detail the evolution of communication network architecture from the second generation (2G) to 5G, highlighting the key points reducing latency. Moreover, we review the evolution of key enabling technologies in the physical layer from 2G to 5G, which is also aimed at reducing latency. We also discussed the challenges and future research directions for low latency in network architecture and physical layer.

An Overview of Low latency for Wireless Communications: an Evolutionary Perspective最先出现在Nweon Paper

]]>
Online and Offline Robot Programming via Augmented Reality Workspaces https://paper.nweon.com/10661 Thu, 22 Jul 2021 02:22:31 +0000 https://paper.nweon.com/10661 PubDate: Jul 2021

Online and Offline Robot Programming via Augmented Reality Workspaces最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Idiap Research Institute

Writers: Yong Joon Thoo, Jérémy Maceiras, Philip Abbet, Mattia Racca, Hakan Girgin, Sylvain Calinon

PDF: Online and Offline Robot Programming via Augmented Reality Workspaces

Abstract

Robot programming methods for industrial robots are time consuming and often require operators to have knowledge in robotics and programming. To reduce costs associated with reprogramming, various interfaces using augmented reality have recently been proposed to provide users with more intuitive means of controlling robots in real-time and programming them without having to code. However, most solutions require the operator to be close to the real robot’s workspace which implies either removing it from the production line or shutting down the whole production line due to safety hazards. We propose a novel augmented reality interface providing the users with the ability to model a virtual representation of a workspace which can be saved and reused to program new tasks or adapt old ones without having to be co-located with the real robot. Similar to previous interfaces, the operators then have the ability to program robot tasks or control the robot in real-time by manipulating a virtual robot. We evaluate the intuitiveness and usability of the proposed interface with a user study where 18 participants programmed a robot manipulator for a disassembly task.

Online and Offline Robot Programming via Augmented Reality Workspaces最先出现在Nweon Paper

]]>
Telelife: The Future of Remote Living https://paper.nweon.com/10659 Thu, 22 Jul 2021 02:01:20 +0000 https://paper.nweon.com/10659 PubDate: Jul 2021

Telelife: The Future of Remote Living最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Osaka University;University of California;University of St. Gallen;University of Maryland;Texas A&M University;Massachusetts Institute of Technology;University College London;Nara Institute of Science and Technology;Augusta University

Writers: Jason Orlosky, Misha Sra, Kenan Bektaş, Huaishu Peng, Jeeeun Kim, Nataliya Kos’myna, Tobias Hollerer, Anthony Steed, Kiyoshi Kiyokawa, Kaan Akşit

PDF: Telelife: The Future of Remote Living

Abstract

In recent years, everyday activities such as work and socialization have steadily shifted to more remote and virtual settings. With the COVID-19 pandemic, the switch from physical to virtual has been accelerated, which has substantially affected various aspects of our lives, including business, education, commerce, healthcare, and personal life. This rapid and large-scale switch from in-person to remote interactions has revealed that our current technologies lack functionality and are limited in their ability to recreate interpersonal interactions. To help address these limitations in the future, we introduce “Telelife,” a vision for the near future that depicts the potential means to improve remote living better aligned with how we interact, live and work in the physical world. Telelife encompasses novel synergies of technologies and concepts such as digital twins, virtual prototyping, and attention and context-aware user interfaces with innovative hardware that can support ultrarealistic graphics, user state detection, and more. These ideas will guide the transformation of our daily lives and routines soon, targeting the year 2035. In addition, we identify opportunities across high-impact applications in domains related to this vision of Telelife. Along with a recent survey of relevant fields such as human-computer interaction, pervasive computing, and virtual reality, the directions outlined in this paper will guide future research on remote living.

Telelife: The Future of Remote Living最先出现在Nweon Paper

]]>
Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality https://paper.nweon.com/10657 Wed, 21 Jul 2021 07:07:20 +0000 https://paper.nweon.com/10657 PubDate: Jul 2021

Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: University of Verona;University of Trento; Fondazione Bruno Kessler

Writers: Ariel Caputo, Andrea Giachetti, Salwa Abkal, Chiara Marchesini, Massimo Zancanaro

PDF: Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality

Abstract

In this paper, a study aimed at investigating the effects of real (using eye tracking to determine the fixation) and simulated foveated blurring in immersive Virtual Reality is presented. Techniques to reduce the optical flow perceived at the visual field margins are often employed in immersive Virtual Reality environments to alleviate discomfort experienced when the visual motion perception does not correspond to the body’s acceleration. Although still preliminary, our results suggest that for participants with higher self-declared sensitivity to sickness, there might be an improvement for nausea when using blurring. The (perceived) difficulty of the task seems to improve when the real foveated method is used.

Real vs Simulated Foveated Rendering to Reduce Visual Discomfort in Virtual Reality最先出现在Nweon Paper

]]>
Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning https://paper.nweon.com/10655 Wed, 21 Jul 2021 06:13:24 +0000 https://paper.nweon.com/10655 PubDate: Jul 2021

Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Singapore University of Technology and Design;Beihang University

Writers: Peng Yang, Tony Q. S. Quek, Jingxuan Chen, Chaoqun You, Xianbin Cao

PDF: Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning

Abstract

This paper investigates the problem of providing ultra-reliable and energy-efficient virtual reality (VR) experiences for wireless mobile users. To ensure reliable ultra-high-definition (UHD) video frame delivery to mobile users and enhance their immersive visual experiences, a coordinated multipoint (CoMP) transmission technique and millimeter wave (mmWave) communications are exploited. Owing to user movement and time-varying wireless channels, the wireless VR experience enhancement problem is formulated as a sequence-dependent and mixed-integer problem with a goal of maximizing users’ feeling of presence (FoP) in the virtual world, subject to power consumption constraints on access points (APs) and users’ head-mounted displays (HMDs). The problem, however, is hard to be directly solved due to the lack of users’ accurate tracking information and the sequence-dependent and mixed-integer characteristics. To overcome this challenge, we develop a parallel echo state network (ESN) learning method to predict users’ tracking information by training fresh and historical tracking samples separately collected by APs. With the learnt results, we propose a deep reinforcement learning (DRL) based optimization algorithm to solve the formulated problem. In this algorithm, we implement deep neural networks (DNNs) as a scalable solution to produce integer decision variables and solving a continuous power control problem to criticize the integer decision variables. Finally, the performance of the proposed algorithm is compared with various benchmark algorithms, and the impact of different design parameters is also discussed. Simulation results demonstrate that the proposed algorithm is more 4.14% energy-efficient than the benchmark algorithms.

Feeling of Presence Maximization: mmWave-Enabled Virtual Reality Meets Deep Reinforcement Learning最先出现在Nweon Paper

]]>
Passing a Non-verbal Turing Test: Evaluating Gesture Animations Generated from Speech https://paper.nweon.com/10653 Wed, 21 Jul 2021 01:43:22 +0000 https://paper.nweon.com/10653 PubDate: Jul 2021

Passing a Non-verbal Turing Test: Evaluating Gesture Animations Generated from Speech最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Graz University of Technology;American University

Writers: Manuel Rebol, Christian Gütl, Krzysztof Pietroszek

PDF:

Abstract

In real life, people communicate using both speech and non-verbal signals such as gestures, face expression or body pose. Non-verbal signals impact the meaning of the spoken utterance in an abundance of ways. An absence of non-verbal signals impoverishes the process of communication. Yet, when users are represented as avatars, it is difficult to translate non-verbal signals along with the speech into the virtual world without specialized motion-capture hardware. In this paper, we propose a novel, data-driven technique for generating gestures directly from speech. Our approach is based on the application of Generative Adversarial Neural Networks (GANs) to model the correlation rather than causation between speech and gestures. This approach approximates neuroscience findings on how non-verbal communication and speech are correlated. We create a large dataset which consists of speech and corresponding gestures in a 3D human pose format from which our model learns the speaker-specific correlation. We evaluate the proposed technique in a user study that is inspired by the Turing test. For the study, we animate the generated gestures on a virtual character. We find that users are not able to distinguish between the generated and the recorded gestures. Moreover, users are able to identify our synthesized gestures as related or not related to a given utterance.

Passing a Non-verbal Turing Test: Evaluating Gesture Animations Generated from Speech最先出现在Nweon Paper

]]>
Resilient and Latency-aware Orchestration of Network Slices Using Multi-connectivity in MEC-enabled 5G Networks https://paper.nweon.com/10651 Wed, 21 Jul 2021 01:19:25 +0000 https://paper.nweon.com/10651 PubDate: Jul 2021

Resilient and Latency-aware Orchestration of Network Slices Using Multi-connectivity in MEC-enabled 5G Networks最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Indian Institute of Technology Madras

Writers: Prabhu Kaliyammal Thiruvasagam, Abhishek Chakraborty, C Siva Ram Murthy

PDF: Resilient and Latency-aware Orchestration of Network Slices Using Multi-connectivity in MEC-enabled 5G Networks

Abstract

Network slicing (NS) and multi-access edge computing (MEC) are new paradigms which play key roles in 5G and beyond networks. NS allows network operators (NOs) to divide the available network resources into multiple logical NSs for providing dedicated virtual networks tailored to the specific service/business requirements. MEC enables NOs to provide diverse ultra-low latency services for supporting the needs of different industry verticals by moving computing facilities to the network edge. NS can be constructed by instantiating a set of virtual network functions (VNFs) on top of MEC cloud servers for provisioning diverse latency-sensitive communication services (e.g., autonomous driving and augmented reality) on demand at a lesser cost and time. However, VNFs, MEC cloud servers, and communication links are subject to failures due to software bugs, misconfiguration, overloading, hardware faults, cyber attacks, power outage, and natural/man-made disaster. Failure of a critical network component disrupts services abruptly and leads to users’ dissatisfaction, which may result in revenue loss for the NOs. In this paper, we present a novel approach based on multi-connectivity in 5G networks to tackle this problem and our proposed approach is resilient against i) failure of VNFs, ii) failure of local servers within MEC, iii) failure of communication links, and iv) failure of an entire MEC cloud facility in regional level. To this end, we formulate the problem as a binary integer programming (BIP) model in order to optimally deploy NSs with the minimum cost, and prove it is NP-hard. To overcome time complexity, we propose an efficient genetic algorithm based heuristic to obtain near-optimal solution in polynomial time. By extensive simulations, we show that our proposed approach not only reduces resource wastage, but also improves throughput while providing high resiliency against failures.

Resilient and Latency-aware Orchestration of Network Slices Using Multi-connectivity in MEC-enabled 5G Networks最先出现在Nweon Paper

]]>
VREUD – An End-User Development Tool to Simplify the Creation of Interactive VR Scenes https://paper.nweon.com/10648 Tue, 20 Jul 2021 08:01:22 +0000 https://paper.nweon.com/10648 PubDate: Jul 2021

VREUD – An End-User Development Tool to Simplify the Creation of Interactive VR Scenes最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: Paderborn University

Writers: Enes Yigitbas, Jonas Klauke, Sebastian Gottschalk, Gregor Engels

PDF: VREUD – An End-User Development Tool to Simplify the Creation of Interactive VR Scenes

Abstract

Recent advances in Virtual Reality (VR) technology and the increased availability of VR-equipped devices enable a wide range of consumer-oriented applications. For novice developers, however, creating interactive scenes for VR applications is a complex and cumbersome task that requires high technical knowledge which is often missing. This hinders the potential of enabling novices to create, modify, and execute their own interactive VR scenes. Although recent authoring tools for interactive VR scenes are promising, most of them focus on expert professionals as the target group and neglect the novices with low programming knowledge. To lower the entry barrier, we provide an open-source web-based End-User Development (EUD) tool, called VREUD, that supports the rapid construction and execution of interactive VR scenes. Concerning construction, VREUD enables the specification of the VR scene including interactions and tasks. Furthermore, VREUD supports the execution and immersive experience of the created interactive VR scenes on VR head-mounted displays. Based on a user study, we have analyzed the effectiveness, efficiency, and user satisfaction of VREUD which shows promising results to empower novices in creating their interactive VR scenes.

VREUD – An End-User Development Tool to Simplify the Creation of Interactive VR Scenes最先出现在Nweon Paper

]]>
UrbanVR: An immersive analytics system for context-aware urban design https://paper.nweon.com/10646 Tue, 20 Jul 2021 07:19:39 +0000 https://paper.nweon.com/10646 PubDate: Jul 2021

UrbanVR: An immersive analytics system for context-aware urban design最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: University of Science and Technology of China;Chinese Academy of Sciences

Writers: Chi Zhang, Wei Zeng, Ligang Liu

PDF: UrbanVR: An immersive analytics system for context-aware urban design

Abstract

Urban design is a highly visual discipline that requires visualization for informed decision making. However, traditional urban design tools are mostly limited to representations on 2D displays that lack intuitive awareness. The popularity of head-mounted displays (HMDs) promotes a promising alternative with consumer-grade 3D displays. We introduce UrbanVR, an immersive analytics system with effective visualization and interaction techniques, to enable architects to assess designs in a virtual reality (VR) environment. Specifically, UrbanVR incorporates 1) a customized parallel coordinates plot (PCP) design to facilitate quantitative assessment of high-dimensional design metrics, 2) a series of egocentric interactions, including gesture interactions and handle-bar metaphors, to facilitate user interactions, and 3) a viewpoint optimization algorithm to help users explore both the PCP for quantitative analysis, and objects of interest for context awareness. Effectiveness and feasibility of the system are validated through quantitative user studies and qualitative expert feedbacks.

UrbanVR: An immersive analytics system for context-aware urban design最先出现在Nweon Paper

]]>
Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning https://paper.nweon.com/10644 Tue, 20 Jul 2021 06:40:23 +0000 https://paper.nweon.com/10644 PubDate: Jul 2021

Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning最先出现在Nweon Paper

]]>
PubDate: Jul 2021

Teams: German Aerospace Center;Hamburg University;Technical University of Munich

Writers: Hadi Beik-Mohammadi, Matthias Kerzel, Benedikt Pleintinger, Thomas Hulin, Philipp Reisich, Annika Schmidt, Aaron Pereira, Stefan Wermter, Neal Y. Lii

PDF: Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning

Abstract

Telerobotic systems must adapt to new environmental conditions and deal with high uncertainty caused by long-time delays. As one of the best alternatives to human-level intelligence, Reinforcement Learning (RL) may offer a solution to cope with these issues. This paper proposes to integrate RL with the Model Mediated Teleoperation (MMT) concept. The teleoperator interacts with a simulated virtual environment, which provides instant feedback. Whereas feedback from the real environment is delayed, feedback from the model is instantaneous, leading to high transparency. The MMT is realized in combination with an intelligent system with two layers. The first layer utilizes Dynamic Movement Primitives (DMP) which accounts for certain changes in the avatar environment. And, the second layer addresses the problems caused by uncertainty in the model using RL methods. Augmented reality was also provided to fuse the avatar device and virtual environment models for the teleoperator. Implemented on DLR’s Exodex Adam hand-arm haptic exoskeleton, the results show RL methods are able to find different solutions when changes are applied to the object position after the demonstration. The results also show DMPs to be effective at adapting to new conditions where there is no uncertainty involved.

Model Mediated Teleoperation with a Hand-Arm Exoskeleton in Long Time Delays Using Reinforcement Learning最先出现在Nweon Paper

]]>
Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments https://paper.nweon.com/10642 Tue, 20 Jul 2021 06:01:34 +0000 https://paper.nweon.com/10642 PubDate: Jun 2021

Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Universität Oldenburg;Technical University of Munich

Writers: Stefan Fichna, Thomas Biberger, Bernhard U. Seeber, Stephan D. Ewert

PDF: Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments

Abstract

In daily life, social interaction and acoustic communication often take place in complex acoustic environments (CAE) with a variety of interfering sounds and reverberation. For hearing research and evaluation of hearing systems simulated CAEs using virtual reality techniques have gained interest in the context of ecologically validity. In the current study, the effect of scene complexity and visual representation of the scene on psychoacoustic measures like sound source location, distance perception, loudness, speech intelligibility, and listening effort in a virtual audio-visual environment was investigated. A 3-dimensional, 86-channel loudspeaker array was used to render the sound field in combination with or without a head-mounted display (HMD) to create an immersive stereoscopic visual representation of the scene. The scene consisted of a ring of eight (virtual) loudspeakers which played a target speech stimulus and non-sense speech interferers in several spatial conditions. Either an anechoic (snowy outdoor scenery) or echoic environment (loft apartment) with a reverberation time (T60) of about 1.5 s was simulated. In addition to varying the number of interferers, scene complexity was varied by assessing the psychoacoustic measures in isolated consecutive measurements or simultaneously. Results showed no significant effect of wearing the HMD on the data. Loudness and distance perception showed significantly different results when they were measured simultaneously instead of consecutively in isolation. The advantage of the suggested setup is that it can be directly transferred to a corresponding real room, enabling a 1:1 comparison and verification of the perception experiments in the real and virtual environment.

Effect of acoustic scene complexity and visual scene representation on auditory perception in virtual audio-visual environments最先出现在Nweon Paper

]]>
Deep Learning for Multi-View Stereo via Plane Sweep: A Survey https://paper.nweon.com/10640 Tue, 20 Jul 2021 04:31:19 +0000 https://paper.nweon.com/10640 PubDate: Jun 2021

Deep Learning for Multi-View Stereo via Plane Sweep: A Survey最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Peking University

Writers: Qingtian Zhu

PDF: Deep Learning for Multi-View Stereo via Plane Sweep: A Survey

Abstract

3D reconstruction has lately attracted increasing attention due to its wide application in many areas, such as autonomous driving, robotics and virtual reality. As a dominant technique in artificial intelligence, deep learning has been successfully adopted to solve various computer vision problems. However, deep learning for 3D reconstruction is still at its infancy due to its unique challenges and varying pipelines. To stimulate future research, this paper presents a review of recent progress in deep learning methods for Multi-view Stereo (MVS), which is considered as a crucial task of image-based 3D reconstruction. It also presents comparative results on several publicly available datasets, with insightful observations and inspiring future research directions.

Deep Learning for Multi-View Stereo via Plane Sweep: A Survey最先出现在Nweon Paper

]]>
FOVQA: Blind Foveated Video Quality Assessment https://paper.nweon.com/10638 Tue, 20 Jul 2021 02:25:36 +0000 https://paper.nweon.com/10638 PubDate: Jun 2021

FOVQA: Blind Foveated Video Quality Assessment最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: The University of Texas at Austin;Facebook Reality Labs

Writers: Yize Jin, Anjul Patney, Richard Webb, Alan Bovik

PDF: FOVQA: Blind Foveated Video Quality Assessment

Abstract

Previous blind or No Reference (NR) video quality assessment (VQA) models largely rely on features drawn from natural scene statistics (NSS), but under the assumption that the image statistics are stationary in the spatial domain. Several of these models are quite successful on standard pictures. However, in Virtual Reality (VR) applications, foveated video compression is regaining attention, and the concept of space-variant quality assessment is of interest, given the availability of increasingly high spatial and temporal resolution contents and practical ways of measuring gaze direction. Distortions from foveated video compression increase with increased eccentricity, implying that the natural scene statistics are space-variant. Towards advancing the development of foveated compression / streaming algorithms, we have devised a no-reference (NR) foveated video quality assessment model, called FOVQA, which is based on new models of space-variant natural scene statistics (NSS) and natural video statistics (NVS). Specifically, we deploy a space-variant generalized Gaussian distribution (SV-GGD) model and a space-variant asynchronous generalized Gaussian distribution (SV-AGGD) model of mean subtracted contrast normalized (MSCN) coefficients and products of neighboring MSCN coefficients, respectively. We devise a foveated video quality predictor that extracts radial basis features, and other features that capture perceptually annoying rapid quality fall-offs. We find that FOVQA achieves state-of-the-art (SOTA) performance on the new 2D LIVE-FBT-FCVR database, as compared with other leading FIQA / VQA models. we have made our implementation of FOVQA available at: this http URL.

FOVQA: Blind Foveated Video Quality Assessment最先出现在Nweon Paper

]]>
Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos https://paper.nweon.com/10636 Tue, 20 Jul 2021 02:01:29 +0000 https://paper.nweon.com/10636 PubDate: Jun 2021

Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Shanghai Jiao Tong University;

Writers: Guangming Wang, Honghao Zeng, Ziliang Wang, Zhe Liu, Hesheng Wang

PDF: Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos

Abstract

Real-time 3D human pose estimation is crucial for human-computer interaction. It is cheap and practical to estimate 3D human pose only from monocular video. However, recent bone splicing based 3D human pose estimation method brings about the problem of cumulative error. In this paper, the concept of virtual bones is proposed to solve such a challenge. The virtual bones are imaginary bones between non-adjacent joints. They do not exist in reality, but they bring new loop constraints for the estimation of 3D human joints. The proposed network in this paper predicts real bones and virtual bones, simultaneously. The final length of real bones is constrained and learned by the loop constructed by the predicted real bones and virtual bones. Besides, the motion constraints of joints in consecutive frames are considered. The consistency between the 2D projected position displacement predicted by the network and the captured real 2D displacement by the camera is proposed as a new projection consistency loss for the learning of 3D human pose. The experiments on the Human3.6M dataset demonstrate the good performance of the proposed method. Ablation studies demonstrate the effectiveness of the proposed inter-frame projection consistency constraints and intra-frame loop constraints.

Motion Projection Consistency Based 3D Human Pose Estimation with Virtual Bones from Monocular Videos最先出现在Nweon Paper

]]>
SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition https://paper.nweon.com/10634 Thu, 15 Jul 2021 07:46:26 +0000 https://paper.nweon.com/10634 PubDate: Jun 2021

SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Queensland University of Technology

Writers: Sourav Garg, Michael Milford

PDF: SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition

Abstract

Place Recognition is a crucial capability for mobile robot localization and navigation. Image-based or Visual Place Recognition (VPR) is a challenging problem as scene appearance and camera viewpoint can change significantly when places are revisited. Recent VPR methods based on sequential representations” have shown promising results as compared to traditional sequence score aggregation or single image based techniques. In parallel to these endeavors, 3D point clouds based place recognition is also being explored following the advances in deep learning based point cloud processing. However, a key question remains: is an explicit 3D structure based place representation always superior to an implicit spatialrepresentation based on sequence of RGB images which can inherently learn scene structure. In this extended abstract, we attempt to compare these two types of methods by considering a similar “metric span” to represent places. We compare a 3D point cloud based method (PointNetVLAD) with image sequence based methods (SeqNet and others) and showcase that image sequence based techniques approach, and can even surpass, the performance achieved by point cloud based methods for a given metric span. These performance variations can be attributed to differences in data richness of input sensors as well as data accumulation strategies for a mobile robot. While a perfect apple-to-apple comparison may not be feasible for these two different modalities, the presented comparison takes a step in the direction of answering deeper questions regarding spatial representations, relevant to several applications like Autonomous Driving and Augmented/Virtual Reality. Source code available publicly this https URL.

SeqNetVLAD vs PointNetVLAD: Image Sequence vs 3D Point Clouds for Day-Night Place Recognition最先出现在Nweon Paper

]]>
Directions for 3D User Interface Research from Consumer VR Games https://paper.nweon.com/10632 Thu, 15 Jul 2021 07:25:20 +0000 https://paper.nweon.com/10632 PubDate: Jun 2021

Directions for 3D User Interface Research from Consumer VR Games最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: University College London;Waseda University;Virginia Tech;University of Canterbury

Writers: Anthony Steed, Tuukka M. Takala, Daniel Archer, Wallace Lages, Robert W. Lindeman

PDF: Directions for 3D User Interface Research from Consumer VR Games

Abstract

With the continuing development of affordable immersive virtual reality (VR) systems, there is now a growing market for consumer content. The current form of consumer systems is not dissimilar to the lab-based VR systems of the past 30 years: the primary input mechanism is a head-tracked display and one or two tracked hands with buttons and joysticks on hand-held controllers. Over those 30 years, a very diverse academic literature has emerged that covers design and ergonomics of 3D user interfaces (3DUIs). However, the growing consumer market has engaged a very broad range of creatives that have built a very diverse set of designs. Sometimes these designs adopt findings from the academic literature, but other times they experiment with completely novel or counter-intuitive mechanisms. In this paper and its online adjunct, we report on novel 3DUI design patterns that are interesting from both design and research perspectives: they are highly novel, potentially broadly re-usable and/or suggest interesting avenues for evaluation. The supplemental material, which is a living document, is a crowd-sourced repository of interesting patterns. This paper is a curated snapshot of those patterns that were considered to be the most fruitful for further elaboration.

Directions for 3D User Interface Research from Consumer VR Games最先出现在Nweon Paper

]]>
Latency-aware and Survivable Mapping of VNFs in 5G Network Edge Cloud https://paper.nweon.com/10630 Thu, 15 Jul 2021 07:07:21 +0000 https://paper.nweon.com/10630 PubDate: Jun 2021

Latency-aware and Survivable Mapping of VNFs in 5G Network Edge Cloud最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Indian Institute of Technology Madras

Writers: Prabhu Kaliyammal Thiruvasagam, Abhishek Chakraborty, C. Siva Ram Murthy

PDF: Latency-aware and Survivable Mapping of VNFs in 5G Network Edge Cloud

Abstract

Network Functions Virtualization (NFV) and Multi-access Edge Computing (MEC) play crucial roles in 5G networks for dynamically provisioning diverse communication services with heterogeneous service requirements. In particular, while NFV improves flexibility and scalability by softwarizing physical network functions as Virtual Network Functions (VNFs), MEC enables to provide delay-sensitive/time-critical services by moving computing facilities to the network edge. However, these new paradigms introduce challenges in terms of latency, availability, and resource allocation. In this paper, we first explore MEC cloud facility location selection and then latency-aware placement of VNFs in different selected locations of NFV enabled MEC cloud facilities in order to meet the ultra-low latency requirements of different applications (e.g., Tactile Internet, virtual reality, and mission-critical applications). Furthermore, we also aim to guarantee the survivability of VNFs and an edge server against failures in resource limited MEC cloud facility due to software bugs, configuration faults, etc. To this end, we formulate the problem of latency-aware and survivable mapping of VNFs in different MEC cloud facilities as an Integer Linear Programming (ILP) to minimize the overall service provisioning cost, and show that the problem is NP-hard. Owing to the high computational complexity of solving the ILP, we propose a simulated annealing based heuristic algorithm to obtain near-optimal solution in polynomial time. With extensive simulations, we show the effectiveness of our proposed solution in a real-world network topology, which performs close to the optimal solution.

Latency-aware and Survivable Mapping of VNFs in 5G Network Edge Cloud最先出现在Nweon Paper

]]>
Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh https://paper.nweon.com/10628 Thu, 15 Jul 2021 06:19:27 +0000 https://paper.nweon.com/10628 PubDate: Jun 2021

Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Binghamton University

Writers: Na Li, Yao Liu

PDF: Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh

Abstract

With the emerging of 360-degree image/video, augmented reality (AR) and virtual reality (VR), the demand for analysing and processing spherical signals get tremendous increase. However, plenty of effort paid on planar signals that projected from spherical signals, which leading to some problems, e.g. waste of pixels, distortion. Recent advances in spherical CNN have opened up the possibility of directly analysing spherical signals. However, they pay attention to the full mesh which makes it infeasible to deal with situations in real-world application due to the extremely large bandwidth requirement. To address the bandwidth waste problem associated with 360-degree video streaming and save computation, we exploit Focused Icosahedral Mesh to represent a small area and construct matrices to rotate spherical content to the focused mesh area. We also proposed a novel VertexShuffle operation that can significantly improve both the performance and the efficiency compared to the original MeshConv Transpose operation introduced in UGSCNN. We further apply our proposed methods on super resolution model, which is the first to propose a spherical super-resolution model that directly operates on a mesh representation of spherical pixels of 360-degree data. To evaluate our model, we also collect a set of high-resolution 360-degree videos to generate a spherical image dataset. Our experiments indicate that our proposed spherical super-resolution model achieves significant benefits in terms of both performance and inference time compared to the baseline spherical super-resolution model that uses the simple MeshConv Transpose operation. In summary, our model achieves great super-resolution performance on 360-degree inputs, achieving 32.79 dB PSNR on average when super-resoluting 16x vertices on the mesh.

Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh最先出现在Nweon Paper

]]>
Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence https://paper.nweon.com/10626 Thu, 15 Jul 2021 06:01:26 +0000 https://paper.nweon.com/10626 PubDate: Jun 2021

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: University of Oulu;The Hong Kong University of Science and Technology;KAIST;University of Helsinki

Writers: Jacky Cao, Kit-Yung Lam, Lik-Hang Lee, Xiaoli Liu, Pan Hui, Xiang Su

PDF: Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence

Abstract

Mobile Augmented Reality (MAR) integrates computer-generated virtual objects with physical environments for mobile devices. MAR systems enable users to interact with MAR devices, such as smartphones and head-worn wearables, and performs seamless transitions from the physical world to a mixed world with digital entities. These MAR systems support user experiences by using MAR devices to provide universal accessibility to digital contents. Over the past 20 years, a number of MAR systems have been developed, however, the studies and design of MAR frameworks have not yet been systematically reviewed from the perspective of user-centric design. This article presents the first effort of surveying existing MAR frameworks (count: 37) and further discusses the latest studies on MAR through a top-down approach: 1) MAR applications; 2) MAR visualisation techniques adaptive to user mobility and contexts; 3) systematic evaluation of MAR frameworks including supported platforms and corresponding features such as tracking, feature extraction plus sensing capabilities; and 4) underlying machine learning approaches supporting intelligent operations within MAR systems. Finally, we summarise the development of emerging research fields, current state-of-the-art, and discuss the important open challenges and possible theoretical and technical directions. This survey aims to benefit both researchers and MAR system developers alike.

Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence最先出现在Nweon Paper

]]>
Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering https://paper.nweon.com/10624 Thu, 15 Jul 2021 05:19:26 +0000 https://paper.nweon.com/10624 PubDate: Jun 2021

Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Hong Kong Baptist University;A*STAR

Writers: Wenpeng Xing, Jie Chen, Zaifeng Yang, Qiang Wang

PDF: Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering

Abstract

Image-based geometric modeling and novel view synthesis based on sparse, large-baseline samplings are challenging but important tasks for emerging multimedia applications such as virtual reality and immersive telepresence. Existing methods fail to produce satisfactory results due to the limitation on inferring reliable depth information over such challenging reference conditions. With the popularization of commercial light field (LF) cameras, capturing LF images (LFIs) is as convenient as taking regular photos, and geometry information can be reliably inferred. This inspires us to use a sparse set of LF captures to render high-quality novel views globally. However, fusion of LF captures from multiple angles is challenging due to the scale inconsistency caused by various capture settings. To overcome this challenge, we propose a novel scale-consistent volume rescaling algorithm that robustly aligns the disparity probability volumes (DPV) among different captures for scale-consistent global geometry fusion. Based on the fused DPV projected to the target camera frustum, novel learning-based modules have been proposed (i.e., the attention-guided multi-scale residual fusion module, and the disparity field guided deep re-regularization module) which comprehensively regularize noisy observations from heterogeneous captures for high-quality rendering of novel LFIs. Both quantitative and qualitative experiments over the Stanford Lytro Multi-view LF dataset show that the proposed method outperforms state-of-the-art methods significantly under different experiment settings for disparity inference and LF synthesis.

Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Rendering最先出现在Nweon Paper

]]>
QoE Driven VR 360 Video Massive MIMO Transmission https://paper.nweon.com/10622 Thu, 15 Jul 2021 02:01:27 +0000 https://paper.nweon.com/10622 PubDate: Jun 2021

QoE Driven VR 360 Video Massive MIMO Transmission最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Institute of Image Communication and Information Processing;University of California at Davis;Lehigh University

Writers: Long Teng, Guangtao Zhai, Yongpeng Wu, Xiongkuo Min, Wenjun Zhang, Zhi Ding, Chengshang Xiao

PDF: QoE Driven VR 360 Video Massive MIMO Transmission

Abstract

Massive multiple-input and multiple-output (MIMO) enables ultra-high throughput and low latency for tile-based adaptive virtual reality (VR) 360 video transmission in wireless network. In this paper, we consider a massive MIMO system where multiple users in a single-cell theater watch an identical VR 360 video. Based on tile prediction, base station (BS) deliveries the tiles in predicted field of view (FoV) to users. By introducing practical supplementary transmission for missing tiles and unacceptable VR sickness, we propose the first stable transmission scheme for VR video. we formulate an integer non-linear programming (INLP) problem to maximize users’ average quality of experience (QoE) score. Moreover, we derive the achievable spectral efficiency (SE) expression of predictive tile groups and the approximately achievable SE expression of missing tile groups, respectively. Analytically, the overall throughput is related to the number of tile groups and the length of pilot sequences. By exploiting the relationship between the structure of viewport tiles and SE expression, we propose a multi-lattice multi-stream grouping method aimed at improving the overall throughput for VR video transmission. Moreover, we analyze the relationship between QoE objective and number of predictive tile. We transform the original INLP problem into an integer linear programming problem by setting the predictive tiles groups as some constants. With variable relaxation and recovery, we obtain the optimal average QoE. Extensive simulation results validate that the proposed algorithm effectively improves QoE.

QoE Driven VR 360 Video Massive MIMO Transmission最先出现在Nweon Paper

]]>
Adaptive Streaming Perception using Deep Reinforcement Learning https://paper.nweon.com/10620 Tue, 13 Jul 2021 06:40:33 +0000 https://paper.nweon.com/10620 PubDate: Jun 2021

Adaptive Streaming Perception using Deep Reinforcement Learning最先出现在Nweon Paper

]]>
PubDate: Jun 2021

Teams: Microsoft Research India

Writers: Anurag Ghosh, Akshay Nambi, Aditya Singh, Harish YVS, Tanuja Ganu

PDF: Adaptive Streaming Perception using Deep Reinforcement Learning

Abstract

Executing computer vision models on streaming visual data, or streaming perception is an emerging problem, with applications in self-driving, embodied agents, and augmented/virtual reality. The development of such systems is largely governed by the accuracy and latency of the processing pipeline. While past work has proposed numerous approximate execution frameworks, their decision functions solely focus on optimizing latency, accuracy, or energy, etc. This results in sub-optimum decisions, affecting the overall system performance. We argue that the streaming perception systems should holistically maximize the overall system performance (i.e., considering both accuracy and latency simultaneously). To this end, we describe a new approach based on deep reinforcement learning to learn these tradeoffs at runtime for streaming perception. This tradeoff optimization is formulated as a novel deep contextual bandit problem and we design a new reward function that holistically integrates latency and accuracy into a single metric. We show that our agent can learn a competitive policy across multiple decision dimensions, which outperforms state-of-the-art policies on public datasets.

Adaptive Streaming Perception using Deep Reinforcement Learning最先出现在Nweon Paper

]]>