A Generative Audio-Visual Prosodic Model for Virtual Actors

Adela Barbulescu, Rémi Ronfard, Gérard Bailly.

IEEE Computer Graphics and Applications, 37 (6), 2017, pp.40-51.

Rows present corresponding frames extracted from (a) the video, (b) ground-truth animation, and
(c) synthetic animation. From left to right, the images correspond to comforting, fascinated, thinking
(male actor), fascinated, ironic, and scandalized (female actor) attitudes.


An important problem in the animation of virtual characters is the expression of complex mental states using the coordinated prosody of voice, rhythm, facial expressions, and head and gaze motion. We propose a method for generating natural speech and facial animation in various attitudes using neutral speech and animation as input.