Deadline for sending applications: 30 November 2013. Project proposed by Georgios Evangelidis and Radu Horaud.
 A stereoscopic image pair |
 The dense depth map obtained by our method |
The PERCEPTION team at INRIA investigates stereoscopic vision. Over years we have studied the geometry of binocular vision, e.g., [1] and developed several stereo matching algorithms based on seed-growing [2,3,4]. The seed-growing principle is very simple: sparse disparities owing to a small number of control points (seeds) is propagated to yield a dense disparity map. This yields a maximum a posteriori (MAP) formulation that leads to a series of local minimization problems that are solved hierarchically by a range growing process. While this method is sub-optimal, it allowed us to devise an intrinsically efficient methodology that bridges the gap between MRF-based global optimizers and locally optimal winner-take-all strategies. These developments formed the basis of a fast stereo implementation that runs on the NAO humanoid at 10 frames/second. These two master projects will study the seed-growing stereo matching paradigm from two distinct perspectives:
- Humanoid vision. Stereo matching has been thoroughly investigated in the framework of Markov random fields (MRF) and the most successful stereo methods are based on belief propagation [5]. We propose to study these methods and to re-formulate seed-growing in terms of a thorough probabilistic graphical model [6,7] with its associated inference procedures. Based on this novel formulation, it should be possible to implement a method that produces as output a dense and accurate depth map, even in the absence of textured surfaces. The associated algorithms will then be implemented onto the NAO’s head which is equipped with a synchronized camera pair and with a middleware allowing the software to run on a remote computer.
- Human vision. Recently, it has been postulated that the visual cortex implements Bayesian inference [8] and that MRF could form the basis of a biological model of stereopsis [9]. We propose to investigate the biological plausibility of seed growing, the links between stereopsis and the perception of texture [10], and the role played by the complex cells for representing local image structure [11,12]. The basic idea of seed-growing is that a small number of “strong” and high-probability correspondences are established and then these correspondences are propagated using some kind of message-passing mechanism. One can postulates that these initial strong matches are the result of an attentional process that remains to be studied and characterized. Hence the importance of top-down information pipelines that feed data into visual areas V1 and V2 from other “higher-level” cortical areas.
These two ambitious research projects will go well beyond the master theses. Both projects will then be addressed in two PHD theses that will immediately follow the master projects. We seek two very motivated candidates. The first candidate should have strong background in Probability, Statistics, Optimization, very good knowledge in Computer Vision, and excellent programming skills (C and Matlab). The second candidate should have strong background in Applied Mathematics, Probability theory, Statistics, good programming skills in Matlab, and good knowledge in Neuroscience. References [1] M. Hansard and R. Horaud. Cyclopean geometry of binocular vision. Journal of the Optical Society of America A, Volume 25, Number 9, 2357–2369, September 2008. [2] J. Cech, J. Sanchez-Rieira, and R. Horaud. Scene Flow Estimation by Growing Correspondence Seeds. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, June 2011. [3] J. Cech, J. Sanchez-Rieira, and R. Horaud. Robust Spatiotemporal Stereo for Dynamic Scenes. Proceedings of the 21st International Conference on Pattern Recognition, November 2012. [4] V. Gandhi, J. Cech, and R. Horaud. High Resolution Depth Maps from TOF-stereo Fusion. Proceedings of the IEEE International Conference on Robotics and Automation, 4742 –4749, May 2012. [5] Q. Yang et al.. Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 31, number 3, March 2009. [6] C. M. Bishop. Pattern recognition and machine Learning, Springer-Verlag, New York, NJ, 2006. [7] M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, volume 1, number 1-2, 1-305, 2008. [8] T. S. Lee and D. Mumford. Hierarchical Bayesian inference in the visual cortex. Journal of the Optical Society of America A, Volume 20, number 7, July 2003. [9] Y. Ming and Z. Hu. Modeling Stereopsis via Markov Random Field. Neural Computation, volume 22, 2161-2191, 2010. [10] P. Chossat and O. Faugeras. Hyperbolic Planforms in Relation to Visual Edges and Textures Perception. PLOS Computational Biology, volume 5, issue 12, December 2009. [11] J. Petitot. Neurogéométrie de la vision. Les Editions de l’Ecole Polytechnique, 2008. [12] M. Hansard and R. Horaud. A Differential Model of the Complex Cell. Neural Computation, Volume 23, Number 9, 2324–2357, September 2011.
