跳至内容
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

CUBE360: Learning Cubic Field Representation for Monocular 360 Depth Estimation for Virtual Reality

编辑:广东客   |   分类:CV   |   2025年3月6日

Note: We don't have the ability to review paper

PubDate: Oct 2024

Teams:USTC; HKUST(GZ)

Writers:Wenjie Chang, Hao Ai, Tianzhu Zhang, Lin Wang

PDF:CUBE360: Learning Cubic Field Representation for Monocular 360 Depth Estimation for Virtual Reality

Abstract

Panoramic images provide comprehensive scene information and are suitable for VR applications. Obtaining corresponding depth maps is essential for achieving immersive and interactive experiences. However, panoramic depth estimation presents significant challenges due to the severe distortion caused by equirectangular projection (ERP) and the limited availability of panoramic RGB-D datasets. Inspired by the recent success of neural rendering, we propose a novel method, named CUBE360, that learns a cubic field composed of multiple MPIs from a single panoramic image for continuous depth estimation at any view direction. Our CUBE360 employs cubemap projection to transform an ERP image into six faces and extract the MPIs for each, thereby reducing the memory consumption required for MPI processing of high-resolution data. Additionally, this approach avoids the computational complexity of handling the uneven pixel distribution inherent to equirectangular projectio. An attention-based blending module is then employed to learn correlations among the MPIs of cubic faces, constructing a cubic field representation with color and density information at various depth levels. Furthermore, a novel sampling strategy is introduced for rendering novel views from the cubic field at both cubic and planar scales. The entire pipeline is trained using photometric loss calculated from rendered views within a self-supervised learning approach, enabling training on 360 videos without depth annotations. Experiments on both synthetic and real-world datasets demonstrate the superior performance of CUBE360 compared to prior SSL methods. We also highlight its effectiveness in downstream applications, such as VR roaming and visual effects, underscoring CUBE360's potential to enhance immersive experiences.

本文链接:https://paper.nweon.com/16230

您可能还喜欢...

  • cd5427171b8ce01d756628d58e6fec31-thumb-medium

    ACTION-Net: Multipath Excitation for Action Recognition

    2021年07月01日 映维

  • STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering

    2021年06月24日 映维

  • SEGA: Drivable 3D Gaussian Head Avatar from a Single Image

    2025年07月29日 广东客

关注:

最新AR/VR行业分享

  • ★ 映维日报:Valve申请Steam Frame商标,谷歌XR专利申请波导光栅边缘切趾化技术 2025年9月5日
  • ★ Meta专利申请XR头戴显示器多状态面板技术 2025年9月5日
  • ★ 谷歌XR专利申请波导光栅边缘切趾化技术 2025年9月5日
  • ★ 安徽工程大学100万元采购沉浸式虚拟现实教学系统 2025年9月5日
  • ★ 新疆兵团疾控中心276万元采购传染病监测VR平台 2025年9月5日

最新AR/VR专利

  • ★ Meta Patent | Systems and methods for improved spatial computing systems 2025年9月4日
  • ★ Apple Patent | Methods for sharing content and interacting with physical devices in a three-dimensional environment 2025年9月4日
  • ★ MagicLeap Patent | Matching content to a spatial 3d environment 2025年9月4日
  • ★ Meta Patent | Techniques for coordinating artificial-reality interactions using augmented-reality interaction guides for performing interactions with physical objects within a user's physical surroundings, and systems and methods for using such techniques 2025年9月4日
  • ★ Meta Patent | Solution of body-garment collisions in avatars for immersive reality applications 2025年9月4日

最新AR/VR行业招聘

  • ★ Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • ★ Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • ★ Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • ★ Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • ★ Apple AR/VR Job | iOS Perception Engineer 2025年3月27日
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

备案粤公网安备:44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper