跳至内容
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

编辑:广东客   |   分类:CV   |   2025年3月27日

Note: We don't have the ability to review paper

PubDate: Otc 2024

Teams:Carnegie Mellon University, Meta Reality Labs Research, TSMC Corporate Research, New York University

Writers:Yiwei Zhao, Ziyun Li, Win-San Khwa, Xiaoyu Sun, Sai Qian Zhang, Syed Shakib Sarwar, Kleber Hugo Stangherlin, Yi-Lun Lu, Jorge Tomas Gomez, Jae-Sun Seo, Phillip B. Gibbons, Barbara De Salvo, Chiao Liu

PDF:Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

Abstract

Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems.

本文链接:https://paper.nweon.com/16247

您可能还喜欢...

  • a221f08c61f2682cdef48180b9c8654d-thumb-medium

    DDT: A Diffusion-Driven Transformer-based Framework for Human Mesh Recovery from a Video

    2023年03月29日 映维

  • ec63ad77983e6bfe638e9273827d6720-thumb-medium

    Hand Gesture Tracking and Recognition based Human-Computer Interaction System and Its Applications

    2020年11月06日 映维

  • fb9fb55ce6b8852f788b9f3437590392-thumb-medium

    A lightweight and efficient system for tracking handheld objects in virtual reality

    2020年06月09日 映维

关注:

最新AR/VR行业分享

  • ★ 映维日报:Valve申请Steam Frame商标,谷歌XR专利申请波导光栅边缘切趾化技术 2025年9月5日
  • ★ Meta专利申请XR头戴显示器多状态面板技术 2025年9月5日
  • ★ 谷歌XR专利申请波导光栅边缘切趾化技术 2025年9月5日
  • ★ 安徽工程大学100万元采购沉浸式虚拟现实教学系统 2025年9月5日
  • ★ 新疆兵团疾控中心276万元采购传染病监测VR平台 2025年9月5日

最新AR/VR专利

  • ★ Meta Patent | Systems and methods for improved spatial computing systems 2025年9月4日
  • ★ Apple Patent | Methods for sharing content and interacting with physical devices in a three-dimensional environment 2025年9月4日
  • ★ MagicLeap Patent | Matching content to a spatial 3d environment 2025年9月4日
  • ★ Meta Patent | Techniques for coordinating artificial-reality interactions using augmented-reality interaction guides for performing interactions with physical objects within a user's physical surroundings, and systems and methods for using such techniques 2025年9月4日
  • ★ Meta Patent | Solution of body-garment collisions in avatars for immersive reality applications 2025年9月4日

最新AR/VR行业招聘

  • ★ Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • ★ Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • ★ Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • ★ Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • ★ Apple AR/VR Job | iOS Perception Engineer 2025年3月27日
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

备案粤公网安备:44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper