Research
Please read our scientific project plan for more information.
ANIMA is a multi-disciplinary team with a common research interest in story understanding, story world authoring and story world directing. We plan to confront those emerging topics from multiple perspectives, by organizing the team into four research themes, where we have the necessary experience and expertise to make significant contributions. Each research theme will examine the same challenges of understanding stories, authoring story worlds, and directing story worlds, from its own perspective and with its own research agenda.
The four research themes pursued by ANIMA are (i) the geometry of story worlds; (ii) the physics of story worlds; (iii) the semantics of story worlds; and (iv) the aesthetics of story worlds. In each theme, significant advances in the state of the art are needed to propose computational models of stories, and build the necessary tools for translating stories to 3D graphics and animation.
Those four research themes are of course not independent. For all practical purposes, we will need a combination of geometry, physics, semantics and aesthetics to understand stories and storyboards; author the necessary story worlds; and direct the stories into compelling animation. We therefore expect a very strong inter-connection between the four research directions pursued by the ANIMA team members.
Geometric modeling
We aim to create intuitive tools for designing 3D shapes and animations which can be used to populate interactive, animated story worlds, rather than inert and static virtual worlds. In many different application scenarios such as preparing a product design review, teaching human anatomy with a MOOC, composing a theatre play, directing a movie, showing a sports event, 3D shapes must be modeled for the specific requirements of the animation and interaction scenarios (stories) of the application.
We will need to invent novel shape modeling methods to support the necessary affordances for interaction and maintain consistency and plausibility of the shape appearances and behaviors during animation and interaction. Compared to our previous work, we will therefore focus increasingly on designing shapes and motions simultaneously, rather than separately, based on the requirements of the stories to be told.
Previous work in the IMAGINE team has emphasized the usefulness of space-time constructions for sketching and sculpting animation both in 2D and 3D. Future work in the ANIMA team will further develop this line of research, with the long-term goal of choreographing complex multi-character animation and providing full authorial and directorial control to the user.
Physical modeling
When authoring and directing story worlds, physics is important to obtain believable and realistic behaviors, e.g. to determine how a garment should deform when a character moves, or how the branches of a tree bend when the wind start to blow. In practice, while deformation rules could be defined a priori (e.g. procedurally), relying on physics-based simulation is more efficient in many cases as this means that we do not need to think in advance about all possible scenarii. In ANIMA, we want to go a step further. Not only do we want to be able to \emph{predict} how the shape of deformable objects will change, but we also want to be able to \emph{control} their deformation. In short, we are interested in solving inverse problems where we adjust some parameters of the simulation, yet to be defined so that the output of the simulation matches what the user wants.
By optimizing design parameters, we can get realistic results on input scenarii, but we can also extrapolate to new settings. For example, solving inverse problems corresponding to static cases can be useful to obtain realistic behaviors when looking at dynamics. E.g. if we can optimize the cloth material and the shape of the patterns of a dress such that it matches what an artist designed for the first frame of an animation, then we can use the same parameters for the rest of the animation. Of course, matching dynamics is also one of our goals.
Compared to more traditional approaches, this raises several challenges. It is not clear what the best way is for the user to specify constraints, i.e. how to define what she wants (we do not necessarily want to specify the positions of all nodes of the meshes for all frames, for example). We want the shape to deform according to physical laws, but also according to what the user specified, which means that the objectives may conflict and that the problem can be over-constrained or under-constrained.
Physics may not be satisfied exactly in all story worlds i.e. input may be cartoonish, for example. In such cases, we may need to adapt the laws of physics or even to invent new ones. In computational fabrication, the designer may want to design an object that cannot be fabricated using traditional materials for example. But in this case, we cannot cheat with the physics. One idea is to extend the range of things that we can do by creating new materials (meta-materials), creating 3D shapes from flat patterns, increasing the extensibility of materials, etc.
To achieve these goals, we will need to find effective metrics (how to define objective functions that we can minimize); develop efficient models (that can be inverted); find suitable parameterizations; and develop efficient numerical optimization schemes (that can account for our specific constraints).
Semantic modeling
Beyond geometry and physics, we aim at representing the semantics of story worlds. We use ontologies to organize story worlds into entities described by well defined concepts and relations between them. Especially important to us is the ability to “depict” story world objects and their properties during the design process while their geometric and material properties are not yet defined. Another important aspect of this research axis is to make it possible to quickly create interactive 3D scenes and movies by assembling existing geometric objects and animations. This requires a conceptual model for semantic annotations, and high level query languages where the result of a semantic query can be a 3D scene or 3D movie.
One important application area for this research axis will be the teaching of human anatomy. The Phd thesis of Ameya Murukutla focuses on automatic generation of augmented reality lessons and exercises for teaching anatomy to medical students and sports students using the prose storyboard language which we introduced during Vineet Gandhi’s PhD thesis. By specializing to this particular area, we are hoping to obtain a formal validation of the proposed methods before we attempt to generalize them to other domains such as interactive storytelling and computer games.
Aesthetic modeling
Data-driven methods for shape modeling and animation are becoming increasingly popular in computer graphics, due to the recent success of deep learning methods. In the context of the ANIMA team, we are particularly interested in methods that can help us capture artistic styles from examples and transfer them to new content. This has important implications in authoring and directing story worlds because it is important to offer artistic control to the author or director, and to maintain a stylistic coherence while generating new content. Ideally, we would like to learn models of our user’s authoring and directing styles, and create contents that matches those styles.
