Manuel J. Marin-Jimenez, University of Cordoba, Spain
Thursday, 7 July 2022, 14:00-15:00, room F107, Inria Montbonnot Saint-Martin
Attend online: https://inria.webex.com/inria/j.php?MTID=mb256349fcf231701cb7e004536b4f398
Abstract: Capturing the ‘mutual gaze’ of people is essential for understanding and interpreting the social interactions between them. To this end, this paper addresses the problem of detecting people Looking At Each Other (LAEO) in video sequences. For this purpose, we propose LAEO-Net++, a new deep CNN for determining LAEO in videos. In contrast to previous works, LAEO-Net++ takes spatio-temporal tracks as input and reasons about the whole track. It consists of three branches, one for each character’s tracked head and one for their relative position. Moreover, we introduce two new LAEO datasets: UCO-LAEO and AVA-LAEO. A thorough experimental evaluation demonstrates the ability of LAEO-Net++ to successfully determine if two people are LAEO and the temporal window where it happens. Our model achieves state-of-the-art results on the existing TVHID-LAEO video dataset, significantly outperforming previous approaches. Finally, we apply LAEO-Net++ to a social network, where we automatically infer the social relationship between pairs of people based on the frequency and duration that they LAEO, and show that LAEO can be a useful tool for guided search of human interactions in videos.
Paper: https://ieeexplore.ieee.org/document/9311655
Short bio: MANUEL J. MARÍN-JIMÉNEZ (google scholar profile) received the BSc, MSc and PhD degrees from the University of Granada, Spain. He has worked, as a visiting student, at the Computer Vision Center of Barcelona (Spain), Vislab-ISR/IST of Lisboa (Portugal) and the Visual Geometry Group of Oxford (UK); and, as visiting researcher, at INRIA-Grenoble (Perception team) and the Human Sensing lab (CMU). He is the coauthor of more than 70 technical papers at international venues and serves as a reviewer of top computer vision and pattern recognition journals. Currently, he works as Associate Professor (tenure position) at the University of Cordoba (Spain). In addition, he is member of the Governing Board of AERFAI (Spanish Association of Pattern Recognition and Image Analysis). His research interests include object detection, human-centric video understanding (including biometrics), visual SLAM and machine learning.
