Localization and geometric reasoning with high level features
Our goal is to push forward vision-based scene understanding and localization through the joint use of learning-based methods with geometrical reasoning. Our hypothesis is that the use of intermediate representations instead or in addition to the classical point feature will lead to increased capacity in terms of scale and robustness to changing conditions. These intermediate representations can be concrete objects which are recognized and used directly in the global pose computation, in the continuity of our works on ellipsoid modeling of objects, or conceptual objects such as vanishing points (VP) or horizon lines that are of specific interest both for localization and modeling of urban or industrial scenes.
A first goal is to improve our method for localization from sets of ellipse/ellipsoid correspondences 1. Besides the need to have more accurate prediction of ellipses, another objective is to elaborate robust strategies and associated numerical schemes for refining the initial pose from a set of objects. This requires to develop appropriate metrics for characterizing good reprojection of 3D objects onto 2D ones and study their impact on minimization issues in localization. Another important line of research will aim at defining strategies to integrate into the localization procedure various features such as points, objects and VPs, which each bring information at different levels. We especially want to investigate how predictive uncertainty and explainability mechanisms can be used to select and weight these various features in the estimation process.
Building dedicated models
In this line of research, our goal is to build physically coherent models with a good accuracy vs. efficiency compromise despite the interactive time constraint set in some targeted applications. Though general purpose solutions exist for building models, such techniques are still greatly challenged in more complex cases when specific constraints on the shape or its deformation must be met. This is especially the case in medical imaging of thin deformable organs, such as the diaphragm, the mitral valve or blood vessels, but also for classical scene modeling where constraints, such as ellipsoidal abstraction of objects, must be introduced. The use of mechanical models have become increasingly important in the team activities in medical imaging, especially for handling organs with large deformations. Our goal is to push forward the development of such models with image-guided procedures or predictive simulation in view.
Facing difficulties of meshing complex geometries, especially thin ones, we want to promote mesh free methods such as implicit models. In the continuity of past works, automatic adaptation of node locations and sizes to the image will be investigated to improve compactness, and computational efficiency of implicit models. As the fidelity of a mechanical model is often impaired by approximations required to solve its dynamical system equations at interactive frame rates, a second goal is to take advantage of our implicit models to improve contact and deformation resolution.
The second axis is about investigating shape-aware methods either for shape segmentation or shape recognition, in order to be able to enforce global shape constraints or geometric shape priors in the output of CNNs. This topic is still addressed in the team in the context of localization from 3D ellipsoidal abstraction of objects 7. Two applications are especially targeted: (i) We aim at improving the detection of pathologies (e.g. brain aneurysms that are mostly located at vessel bifurcations), through adapted and guided sampling of input data during training, as well as through mechanisms inspired by visual attention modules (ii) In the context of fluid structure simulation methods for patient-based mitral valve simulation, it often appears that the geometric segmented model leads to divergence of the numerical scheme. Our intention is to identify geometric conditions under which simulation works well with the idea to incorporate them in the segmentation process.
Estimation and inverse problems
Most aforementioned tasks lead to image-based inverse, possibly ill-posed, problems. While some of them can be solved with well-established estimation techniques, others will necessitate the design of new strategies. In this perspective, we consider in this research axis several fundamental aspects of estimation, common to our problems, such as sampling methods,traditional optimization methods, or end-to-end learning methods for pose estimation.
- Optimization, variational calculus and numerical schemes
We are interested in non-convex optimization problems, especially those raised by variational calculus. While the convergence of numerical schemes is well established for convex problems, this is not always the case for non-convex functionals. Our aim is to continue the work already carried out in the biconvex framework 6, and extend it to primal-dual algorithms. We especially want to address energy minimization problems where the energy is convex with respect to each variable, but non-convex with respect to the pair of variables.
Another research topic is to investigate new neural architectures adapted to non-Euclidean data, and also to plug variational methods into deep learning approaches to regularize the results. The obtained theoretical results will be applied to image colorization, with the idea to reduce artefacts caused both by a lack of regularization and by the non-Euclidean structure of color information as perceived by the human visual system.
- Machine learning for physical problems
We aim at continuing our efforts towards supervised and unsupervised learning for estimation problems. Concerning supervised learning, we intend to investigate further the opportunities offered by neural network estimation of displacement and strain fields in experimental mechanics that we have recently introduced with colleagues in mechanics and signal processing 8. Besides, we also aim at developing unsupervised learning in problems where a quantity has to be estimated over a spatio-temporal domain, which is a recent trend in several application domains. Neural networks are indeed universal approximators whose derivative can be exactly computed with the backpropagation algorithm, which is supposed to make them robust to acquisition noise.
