跳至内容
  • 首页
  • 资讯
  • 行业方案
  • 付费阅读
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

小编 广东客   |   分类:CV   |   发布日期 2025年3月27日

Note: We don't have the ability to review paper

PubDate: Otc 2024

Teams:Carnegie Mellon University, Meta Reality Labs Research, TSMC Corporate Research, New York University

Writers:Yiwei Zhao, Ziyun Li, Win-San Khwa, Xiaoyu Sun, Sai Qian Zhang, Syed Shakib Sarwar, Kleber Hugo Stangherlin, Yi-Lun Lu, Jorge Tomas Gomez, Jae-Sun Seo, Phillip B. Gibbons, Barbara De Salvo, Chiao Liu

PDF:Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices

Abstract

Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) and transformers (ViT), often achieve superior accuracy/performance tradeoff on various computer vision and machine learning (ML) tasks. However, hybrid ML models can pose system challenges for latency and energy-efficiency due to their diverse nature in dataflow and memory access patterns. In this work, we leverage the architecture heterogeneity from Neural Processing Units (NPU) and Compute-In-Memory (CIM) and perform diverse execution schemas to efficiently execute these hybrid models. We also introduce H4H-NAS, a Neural Architecture Search framework to design efficient hybrid CNN/ViT models for heterogeneous edge systems with both NPU and CIM. Our H4H-NAS approach is powered by a performance estimator built with NPU performance results measured on real silicon, and CIM performance based on industry IPs. H4H-NAS searches hybrid CNN/ViT models with fine granularity and achieves significant (up to 1.34%) top-1 accuracy improvement on ImageNet dataset. Moreover, results from our Algo/HW co-design reveal up to 56.08% overall latency and 41.72% energy improvements by introducing such heterogeneous computing over baseline solutions. The framework guides the design of hybrid network architectures and system architectures of NPU+CIM heterogeneous systems.

本文链接:https://paper.nweon.com/16247

您可能还喜欢...

  • Language-Driven Interactive Shadow Detection

    2025年01月23日 广东客

  • LitAR: Visually Coherent Lighting for Mobile Augmented Reality

    2023年01月18日 映维

  • ContactOpt: Optimizing Contact to Improve Grasps

    2021年06月24日 映维

关注:

RSS 最新AR/VR行业分享

  • XR日报:小米AI眼镜1999元起,全息技术Swave Photonics获600万欧元追加投资 2025年6月27日
  • 谷歌XR专利提出纳米颗粒增强衍射波导制造技术 2025年6月27日
  • 谷歌专利提出基于准直器的AR显示系统视场校准技术 2025年6月27日
  • 欧亚研究团队推出OSLO-IC技术实现360度图像高效压缩 2025年6月27日
  • 悉尼大学研发TA-GNN模型实现VR环境高精度手指运动预测 2025年6月27日

RSS 最新AR/VR专利

  • Samsung Patent | Deposition equipment 2025年6月26日
  • Snap Patent | Determining gaze direction to generate augmented reality content 2025年6月26日
  • Apple Patent | Eye characteristic determination 2025年6月26日
  • Snap Patent | Eyewear device charging case 2025年6月26日
  • Qualcomm Patent | Vehicle and mobile device interface for vehicle occupant assistance 2025年6月26日

RSS 最新AR/VR行业招聘

  • Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • Apple AR/VR Job | iOS Perception Engineer 2025年3月27日

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

粤公网安备 44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper