M.Sc. project on “Reinforcement and Deep Learning applied to Human-Robot Dialog”
Duration: 6 months (and it may continue with a PhD)
Short description: The main goal of this project is to design and develop an automatic system to be exploited by a humanoid robot in multiparty Human-robot interaction (i.e. involving several participants). The system has to infer robot actions from the perception of both external audio and video information captured by the robot sensors (e.g. a microphone array and a pair of stereo cameras), and the robot internal state representation. For example, the robot placed into a multiparty conversation should turn towards the relevant person and/or take a speech turn to announce some relevant information. One direction for this work is to start from Partially Observable Markov Decision Processes (POMDP), and associated reinforcement learning algorithms, which are state-of-the-art probabilistic models and strategies for the modeling of task planning and decision taking processes, and study how we can combine these techniques with deep neural networks in charge of discriminative feature exctraction (e.g. CNNs applied to video sequences of the scene, to extract basic visual descriptors of the multiparty interaction). This work will involve the use of a large audiovisual dataset of multiparty interactions, the implementation of supervised training / deep learning / reinforcement learning algorithms, and extensive evaluation of the developed system.
Keywords: human-robot interaction, situated dialog, reinforcement learning, deep neural networks, audio-visual scene understanding
Information for applicants: Please send your complete CV to Laurent Girin (laurent.girin@gipsa-lab.grenoble-inp.fr)
