It has recently become possible to capture time-varying 3D point clouds at high spatial and temporal resolution. However, tools to process and analyze these data robustly and automatically are still missing. Such tools are critical to learning generative models of dynamic human motion, which can in turn be leveraged to create plausible synthetic humans. This has the potential to influence virtual reality applications, such as virtual change rooms or crowd simulations.
The goal of this Ph.D. is to automatically compute high-quality generative models from a database of raw dense 3D motion sequences for human bodies and faces by leveraging deep learning techniques. In particular, the work will focus on two aspects. First, we will study whether co-registration approaches developed for static human scans [1,2] can be extended to dense 3D geometric models in motion. These approaches work based on the realization that the alignment of static 3D data and the generative models learned based on this alignment are interdependent problems, and propose solutions by performing a groupwise optimization over the full training dataset to compute a generative model of high quality. These approaches have received little attention in practice as the optimization is heavy and impractically slow. Hence, extending these works to motion data is challenging due to the mere size of the data. We will investigate whether deep learning techniques can be leveraged to accelerate the groupwise computation. Recently deep learning techniques have started to be used to learn generative models of static 3D human data, e.g. [3,4].
Second, the resulting generative models will be evaluated through user studies. When synthesizing human motion, standard distance measurements used in computer vision are limited, as the same motion performed by the same individual exhibits variation. For this reason, we will investigate how to quantify the realism of a synthesized motion using perceptual studies, similar to works that have been performed for collision handling in crowds by involving users in the evaluation loop. We plan to develop both perception and perception/action experimental paradigms, similar to [5,6].
- Hirshberg, M. Loper, M. Black. Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape, In European Conference on Computer Vision 2012.
- Bolkart and S. Wuhrer. A groupwise multilinear correspondence optimization for 3d faces. In International Conference on Computer Vision, 2015.
- Bagautdinov, C. Wu, J. Saragih, P. Fua, Y. Sheikh. Modeling facial geometry using compositional VAEs, In Conference on Computer Vision and Pattern Recognition, 2018.
- Fernàndez Abrevaya, A. Boukhayma, S. Wuhrer, and E. Boyer. A generative 3d facial model by adversarial training. In International Conference on Computer Vision, 2019.
- -H. Olivier, J. Bruneau, R. Kulpa, and J. Pettré. Walking with virtual people: Evaluation of locomotion interfaces in dynamic environments. IEEE Transactions on Visualization and Computer Graphics, 2018.
- Hoyet, A.-H. Olivier, R. Kulpa, J. Pettré. Perceptual Effect of Shoulder Motions on Crowd Animations. SIGGRAPH 2016.
The Ph.D. will start in October 2021 and its duration will be 3 years. The Ph.D. will be supervised by Stefanie Wuhrer, Inria Grenoble-Alpes, and Anne-Hélène Olivier, Université Rennes 2.
The Ph.D. will take place within the Morpheo research team at Inria Grenoble Rhône-Alpes. The team deals with the capture and analysis of humans from multi-camera studios, and operates its own 68 camera acquisition platform and cluster, http://kinovis.inrialpes.fr. (Example acquisitions are shown in the figure above.) The work is co-supervised with MimeTIC at Inria Rennes, whose research focus includes the simulation of realistic virtual humans.
Inria is a leading French research centre in computer science. The Grenoble centre is located at the heart of the French Alps, a very dynamic region for new technologies offering a large range of recreational activities.
- Master in Computer Science or Applied Mathematics.
- Solid programming skills, e.g. python and/or C++.
- Solid mathematical knowledge in linear algebra and statistics.
- Experience with deep learning and shape modeling is a plus.
- Experience with user studies is a plus.
- Good English level. French is not required.
Only applications submitted here are guaranteed to be considered.
For more information on this position, contact Stefanie Wuhrer (email@example.com).