Perception and interpretation of human activity

Auteur : Francis Colas

Informations générales

Encadrants	Francis Colas	Vincent Thomas	Serena Ivaldi
Adresse
Téléphone	03 54 95 86 30	03 54 95 85 08
Email	francis.colas@inria.fr	vincent.thomas@loria.fr	serena.ivaldi@inria.fr
Bureau	C125	C125	C104

Context

For a long time, industrial robots were secluded in safety cages and autonomous robots were mainly designed and programmed to work without humans or while avoiding them (as obstacles). There is a growing trend towards environment sharing and collaboration between humans and robots with the objective of having robots assist humans (for manufacturing but also in our daily lives). One of the main issues for human-robot interaction is, for the robot, to be able to adequately perceive and interprete what the human is doing.

Objectives

This internship is in the context of the “Flying co-worker” ANR project aiming at building a collaborative flying robot to help human workers. The objective of this subject is to be able to provide information about the current and future status of the human worker. This can mean the precise motion of the arms, hands, head of the human but also her posture and even the higher-level task she is performing.

There are several aspects to be tackled in the long term, each of which open for the present internship:

• human perception by a flying robot,

• human activity recognition,

• gesture and intention estimation,

• human activity prediction.

A possible workplan would be to start by study and use state-of-the-art techniques for activity recognition based on probabilistic approaches such as Hidden Markov Models . It would then be possible to develop new activity models which would maximize longer-term prediction rates, for instance, based on more accurate modeling of each task duration. A particular challenge is to be able to learn the parameters of such a new model. An additional step could be to leverage short-term motion prediction , coupled with the new activity model to further improve accuracy.

It is expected to balance theoretical contributions with experimental validation. To this end, various mobile robots (Tiago, Pepper, and several other ground robots) with different sensors are available as test platforms.

Cadre du travail

This internship takes place at Inria in Nancy in the Larsen team and is founded by the “Flying co-worker” ANR project. PhD funding will be available for the continuation of this project.

References

Malaisé, A. and Maurice, P. and Colas, F. and Charpillet, F. and Ivaldi, S. (2018) Activity Recognition With Multiple Wearable Sensors for Industrial Applications. International Conference on Advances in Computer-Human Interactions (ACHI).

Dermy, O. and Chaveroche, M. and Colas, F. and Charpillet, F. and Ivaldi, S. (2018) Prediction of Human Whole-Body Movements with AE-ProMPs. IEEE/RAS Int. Conference on Humanoid Robotics (HUMANOIDS).