跳至内容
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

编辑:广东客   |   分类:CV   |   2025年3月27日

Note: We don't have the ability to review paper

PubDate: Otc 2024

Teams:Carnegie Mellon University, Meta Reality Labs Research, TSMC Corporate Research, New York University

Writers:Yiwei Zhao, Ziyun Li, Win-San Khwa, Xiaoyu Sun, Sai Qian Zhang, Syed Shakib Sarwar, Kleber Hugo Stangherlin, Yi-Lun Lu, Jorge Tomas Gomez, Jae-Sun Seo, Phillip B. Gibbons, Barbara De Salvo, Chiao Liu

PDF:Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

Abstract

Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems.

本文链接:https://paper.nweon.com/16247

您可能还喜欢...

  • Polarmask-Tracker: Lightweight Multi-Object Tracking and Segmentation Model for Edge Device

    2022年05月30日 映维

  • 341123b0b184a4e9e43dd52b68aa0b0c-thumb-medium

    DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion

    2021年06月28日 映维

  • 681e7ab164a0638658126a5f4d0ed0a8-thumb-medium

    Tiled Multiplane Images for Practical 3D Photography

    2023年10月17日 映维

关注:

最新AR/VR行业分享

  • ★ 映维日报:Meta战略调整元宇宙团队重组,谷歌XR专利提出热补偿特性波导 2025年10月28日
  • ★ 媒体评论:三星Galaxy XR是行业迈向下一阶段的关键头显 2025年10月28日
  • ★ 研究员提出新型电子墨水XR显示屏,分辨率远超现有头显设备 2025年10月28日
  • ★ Meta战略调整:元宇宙团队重组,高管驰援AI部门 2025年10月28日
  • ★ Snap提出XR设备可调节显示布置方案 2025年10月28日

最新AR/VR专利

  • ★ Snap Patent | Selecting an audio track in association with multi-video clip capture 2025年10月23日
  • ★ Meta Patent | Systems and methods for seamless user interface rendering using multiple processors of a device 2025年10月23日
  • ★ LG Patent | Cooperation platform between edge and cloud for providing signage 2025年10月23日
  • ★ Snap Patent | Generative ai experience with movement 2025年10月23日
  • ★ Meta Patent | Coordination between independent rendering frameworks 2025年10月23日

最新AR/VR行业招聘

  • ★ Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • ★ Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • ★ Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • ★ Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • ★ Apple AR/VR Job | iOS Perception Engineer 2025年3月27日
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

备案粤公网安备:44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper