跳至内容
  • 首页
  • 资讯
  • 行业方案
  • 付费阅读
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction
空 挡 广 告 位 | 空 挡 广 告 位

Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

小编 广东客   |   分类:CV   |   发布日期 2025年3月27日

Note: We don't have the ability to review paper

PubDate: Otc 2024

Teams:Carnegie Mellon University, Meta Reality Labs Research, TSMC Corporate Research, New York University

Writers:Yiwei Zhao, Ziyun Li, Win-San Khwa, Xiaoyu Sun, Sai Qian Zhang, Syed Shakib Sarwar, Kleber Hugo Stangherlin, Yi-Lun Lu, Jorge Tomas Gomez, Jae-Sun Seo, Phillip B. Gibbons, Barbara De Salvo, Chiao Liu

PDF:Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

Abstract

Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems.

本文链接:https://paper.nweon.com/16247

您可能还喜欢...

  • StereoTrack: 180-degree Low-cost Acoustic Positional Tracking for Mobile VR Platforms

    2020年06月11日 映维

  • Zillow Indoor Dataset:Annotated Floor Plans With 360o Panoramas and 3D Room Layouts

    2021年06月28日 映维

  • Consistency Guided Scene Flow Estimation

    2020年08月25日 映维

关注:

RSS 最新AR/VR行业分享

  • XR日报:Meta自称是AR真正的信徒,Vision Pro开放摄像头API+意念操控 2025年5月14日
  • 印度混合现实初创公司Flam获得1400万美元A轮融资 2025年5月14日
  • 三星发布5000 ppi OLEDoS:亮度15000尼特,8K VR显示质量 2025年5月14日
  • Meta CTO表示和扎克伯格都是“AR眼镜的真正信徒” 2025年5月14日
  • Google I/O 2025前瞻:AI眼镜原型亮相,Gemini多模态交互重塑无界面计算 2025年5月14日

RSS 最新AR/VR专利

  • Sony Patent | Information processing device and information processing method 2025年5月8日
  • Niantic Patent | Maintaining object alignment in 3d map segments 2025年5月8日
  • Samsung Patent | Deposition mask and method for manufacturing the same 2025年5月8日
  • ARM Patent | Firearm training system 2025年5月8日
  • Apple Patent | Electronic device system with supplemental lenses 2025年5月8日

RSS 最新AR/VR行业招聘

  • Apple AR/VR Job | Senior Software QA Engineer - Apple Vision Pro 2024年11月12日
  • Apple AR/VR Job | System Product Design Engineer - Apple Vision Pro 2024年11月12日
  • Microsoft AR/VR Job | Principal Software Engineer -Teams Premium Services 2024年11月12日
  • Meta AR/VR Job | Software Engineer - XR Codec Interactions and Avatars Team 2024年11月12日
  • Meta AR/VR Job | Product Cost Engineer 2024年11月12日

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

粤公网安备 44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper