跳至内容
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群
  • All
  • XR
  • CV
  • CG
  • HCI
  • Video
  • Optics
  • Perception
  • Reconstruction

LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details

小编 广东客   |   分类:CV   |   2025年2月27日

Note: We don't have the ability to review paper

PubDate: Feb 2025

Teams:Beihang University, Beijing Academy of Blockchain and Edge Computing, Psyche AI,The University of Alabama at Birmingham

Writers:Jian Yang, Xukun Wang, Wentao Wang, Guoming Li, Qihang Fang, Ruihong Yuan, Tianyang Wang, Jason Zhaoxin Fan

PDF:LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details

Abstract

Audio-driven talking head generation is a pivotal area within film-making and Virtual Reality. Although existing methods have made significant strides following the end-to-end paradigm, they still encounter challenges in producing videos with high-frequency details due to their limited expressivity in this domain. This limitation has prompted us to explore an effective post-processing approach to synthesize photo-realistic talking head videos. Specifically, we employ a pretrained Wav2Lip model as our foundation model, leveraging its robust audio-lip alignment capabilities. Drawing on the theory of Lipschitz Continuity, we have theoretically established the noise robustness of Vector Quantised Auto Encoders (VQAEs). Our experiments further demonstrate that the high-frequency texture deficiency of the foundation model can be temporally consistently recovered by the Space-Optimised Vector Quantised Auto Encoder (SOVQAE) we introduced, thereby facilitating the creation of realistic talking head videos. We conduct experiments on both the conventional dataset and the High-Frequency TalKing head (HFTK) dataset that we curated. The results indicate that our method, LaDTalk, achieves new state-of-the-art video quality and out-of-domain lip synchronization performance.

本文链接:https://paper.nweon.com/16220

您可能还喜欢...

  • 3D Face Reconstruction From Radar Images

    2025年04月24日 广东客

  • Human Shape and Clothing Estimation

    2024年08月07日 广东客

  • A Robust 6-D Pose Tracking Approach by Fusing a Multi-Camera Tracking Device and an AHRS Module

    2022年07月28日 映维

关注:

RSS 最新AR/VR行业分享

  • 《哪吒之魔童闹海》将于8月2日上午10点登陆PICO 2025年8月2日
  • 尽管Apple Vision Pro销量不佳,但库克坚信这是未来的发展方向 2025年8月2日
  • 2025年08月02日美国专利局新申请AR/VR专利摘选 2025年8月2日
  • 映维日报:三星XR头显将2025年底上市,Android Studio新增XR开发工具模板 2025年8月1日
  • 西安市卫生学校120万元招标康养VR实训基地建设项目 2025年8月1日

RSS 最新AR/VR专利

  • Samsung Patent | Worktable system and manufacturing method of display panel using the same 2025年7月31日
  • Nvidia Patent | Ego-machine simulation using hardware in-loop 2025年7月31日
  • Samsung Patent | Universal controller for use with either hand 2025年7月31日
  • Sony Patent | Systems and methods for electronic game control and game controller configurations with emg sensing 2025年7月31日
  • Meta Patent | Wearable band structure with flexible printed circuit strain-relief integration, and systems and methods of use thereof 2025年7月31日

RSS 最新AR/VR行业招聘

  • Microsoft AR/VR Job | High Performance Compute, Director 2025年6月5日
  • Microsoft AR/VR Job | Data Center Technician/ Technicien de Centre de Données 2025年6月3日
  • Microsoft AR/VR Job | Senior Product Designer 2025年5月16日
  • Apple AR/VR Job | AirPlay Audio Engineer 2025年3月27日
  • Apple AR/VR Job | iOS Perception Engineer 2025年3月27日
  • 首页
  • 资讯
  • 资源下载
  • 行业方案
  • Job招聘
  • Paper论文
  • Patent专利
  • 映维会员
  • 导航收录
  • 合作
  • 关于
  • 微信群

联系微信:ovalics

版权所有:广州映维网络有限公司 © 2025

备案许可:粤ICP备17113731号-2

粤公网安备 44011302004835号

友情链接: AR/VR行业导航

读者QQ群:251118691

Quest QQ群:526200310

开发者QQ群:688769630

Paper