跳至内容
  • 首页
  • 资讯
  • 行业方案
  • 付费阅读
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details

小编 广东客   |   分类:CV   |   2025年2月27日

Note: We don't have the ability to review paper

PubDate: Feb 2025

Teams:Beihang University, Beijing Academy of Blockchain and Edge Computing, Psyche AI,The University of Alabama at Birmingham

Writers:Jian Yang, Xukun Wang, Wentao Wang, Guoming Li, Qihang Fang, Ruihong Yuan, Tianyang Wang, Jason Zhaoxin Fan

PDF:LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details

Abstract

Audio-driven talking head generation is a pivotal area within film-making and Virtual Reality. Although existing methods have made significant strides following the end-to-end paradigm, they still encounter challenges in producing videos with high-frequency details due to their limited expressivity in this domain. This limitation has prompted us to explore an effective post-processing approach to synthesize photo-realistic talking head videos. Specifically, we employ a pretrained Wav2Lip model as our foundation model, leveraging its robust audio-lip alignment capabilities. Drawing on the theory of Lipschitz Continuity, we have theoretically established the noise robustness of Vector Quantised Auto Encoders (VQAEs). Our experiments further demonstrate that the high-frequency texture deficiency of the foundation model can be temporally consistently recovered by the Space-Optimised Vector Quantised Auto Encoder (SOVQAE) we introduced, thereby facilitating the creation of realistic talking head videos. We conduct experiments on both the conventional dataset and the High-Frequency TalKing head (HFTK) dataset that we curated. The results indicate that our method, LaDTalk, achieves new state-of-the-art video quality and out-of-domain lip synchronization performance.

本文链接:https://paper.nweon.com/16220

您可能还喜欢...

  • Robust and real-time pose tracking for augmented reality on mobile devices

    2021年05月31日 映维

  • Real-time Hair Segmentation and Recoloring on Mobile GPUs

    2020年08月11日 映维

  • Multi-user, Scalable 3D Object Detection in AR Cloud

    2020年06月22日 映维

关注:

RSS 最新AR/VR行业分享

  • XR日报: Cellid开发60° FOV AR投影仪,苹果调研用户规划XR产品路线 2025年7月1日
  • 武当山推出基于PICO VR的"入境武当"VR大空间文旅项目 2025年7月1日
  • Cellid开发60度视场AR眼镜微型投影仪 2025年7月1日
  • PICO推出暑期科幻季内容分享活动 2025年7月1日
  • HealthpointCapital收购XR医疗公司ImmersiveTouch多数股权 2025年7月1日

RSS 最新AR/VR专利

  • Samsung Patent | Deposition equipment 2025年6月26日
  • Snap Patent | Determining gaze direction to generate augmented reality content 2025年6月26日
  • Apple Patent | Eye characteristic determination 2025年6月26日
  • Snap Patent | Eyewear device charging case 2025年6月26日
  • Qualcomm Patent | Vehicle and mobile device interface for vehicle occupant assistance 2025年6月26日

RSS 最新AR/VR行业招聘

  • Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • Apple AR/VR Job | iOS Perception Engineer 2025年3月27日

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

粤公网安备 44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper