The Morpheo INRIA team and Microsoft are setting up a collaboration on the capture and modelling of moving shapes using multiple videos. This PhD proposal is part of this collaboration with the objective to develop tools for animation synthesis and 4D content generation based on captured 4D sequences with multi-view videos. The PhD will take place at Inria Grenoble Rhône-Alpes and will involve regular visits and stays at Microsoft in Redmond (USA) and Cambridge (UK).
Recent 4D shape capture system have the ability to model dynamic scenes in a precise and realistic ways,hence providing a meaningful way to produce contents for VAR applications. Besides the properties of the 4D Models, e.g. its precision, that can still be improved, an interesting issue is how to recombinerecorded 4D models in order to produce new animations and with respect to constraints, e.g. interactivesystems or motion transfer. In addition to broaden creative possibilities with 4D models, this also reduces the need for exhaustive acquisitions when large datasets or complex dynamic scenes (e.g. a crowd) are targeted. While some preliminary works have already investigated the concatenation or the motion transfer between 4D models there are still many directions to explore:
- How to recombine, morph or transfer information in a hierarchical way between body subjects and between their body parts, in both the geometry and appearance domains.
- Time re-basing of sequences, time speed alteration
- Retrieve style characteristics of captured humans and use for improvement of synthesis
- How to design interactive systems where animations are produced in real-time.
- How to model the motion and appearance information of dynamic shapes in order to enable interpolation (e.g pose spaces for motion).
- How to create complex scenes with interacting shapes, e.g. crowds.
- How to preserve individual style when creating or reproducing a given motion.
We propose to explore some of the above issues with a particular emphasize on interactive setups for VR or AR experiences with HMD and to take advantage of the Kinovis platform to produce data for that purpose.
The PhD candidate should hold a master’s degree in computer science. Very good background in computer vision, 3D vision, and/or machine learning are expected. The candidate will be co-supervised by Jean-Sébastien Franco and Edmond Boyer at Inria Grenoble, France, with the involvement of Steve Sullivan, Andrew Fitzgibbon, Jamie Shotton and Marta Wilczkowiak at Microsoft.
Inria is a leading French research centre in computer science, with an international culture – the English language being widely adopted. The Grenoble centre is located at the heart of the French Alps, a very dynamic region for new technologies offering a large range of recreational activities.
The PhD will involve regular visits and stays to the Microsoft centres at Redmond (USA) and Cambridge (UK).
Informal inquires can be addressed to firstname.lastname@example.org and email@example.com. Please apply on the INRIA website.
- Real-time human pose recognition in parts from single depth images. Jamie Shotton, Toby Sharp, Alex Kipman, Andrew Fitzgibbon, Mark Finocchio, Andrew Blake, Mat Cook, Richard Moore, Communications of the ACM, 2013, 56 (1), 116-124
- High-Quality Streamable Free-Viewpoint Video. Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Steve Sullivan. ACM Trans. Graphics (SIGGRAPH), 2015, 34, 4
- Fusion4D: Real-time Performance Capture of Challenging Scenes. Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, Pushmeet Kohli, Vladimir Tankovich, Shahram Izadi. ACM Transactions on Graphics (TOG), 2016, 35 (4), 114
- Eigen Appearance Maps of Dynamic Shapes. Adnane Boukhayma, Vagia Tsiminaki, Jean-Sébastien Franco, Edmond Boyer. ECCV 2016-European Conference on Computer Vision
- Surface Motion Capture Transfer with Gaussian Process Regression. Adnane Boukhayma, Jean-Sébastien Franco, Edmond Boyer. CVPR 2017 – IEEE Conference on Computer Vision and Pattern Recognition, 2017.
- An Efficient Volumetric Framework for Shape Tracking. Benjamin Allain, Jean-Sébastien Franco, Edmond Boyer. CVPR 2015 – IEEE International Conference on Computer Vision and Pattern Recognition, 2015