Audio Driven Facial Animation For Audio-Visual Reality

Title: Audio Driven Facial Animation For Audio-Visual Reality

Teams: Microsoft

Writers: Tanveer A. Faruquie Ashish Kapoor Rohit J. Kate Nitendra Rajput L. Venkat Subramaniam

Publication date: August 2001

Abstract

In this paper, we demonstrate a morphing based automated audio driven facial animation system. Based on an incoming audio stream, a face image is animated with full lip synchronization and expression. An animation sequence using optical flow between visemes is constructed, given an incoming audio stream and still pictures of a face speaking different visemes. Rules are formulated based on coarticulation and the duration of a viseme to control the continuity in terms of shape and extent of lip opening. In addition to this new viseme-expression combinations are synthesized to be able to generate animations with new facial expressions. Finally various applications of this system are discussed in the context of creating audiovisual reality.