Date of start: October 1, 2024
Advisors: Alexandre Krupa (Inria, Rainbow), Eric Marchand (Université de Rennes, Rainbow)
Place of work: The PhD student will be hosted in the IRISA/Inria Rainbow group (https://team.inria.fr/rainbow/) at Rennes.
Context
A major challenge in robotics is the interaction with deformable objects. Indeed, nowadays most of the robot control frameworks are built for rigid objects observed in the scene. Therefore, in order to extend the manipulation capabilities of robots, the main objective of this thesis is to investigate a new robotic control approach for manipulating soft objects to apply a desired deformation. The general idea is to use visual feedback to estimate and track in real time the deformations of the object of interest, and to develop a visual servo-control approach to control one or more robotic manipulators in order to apply a desired deformation to the object.
Automatically controlling the deformation of soft object would enable numerous robotic applications such as gripping a deformable object with a robotic hand, assembling flexible elements, accurately manipulating elastic materials or food products.
Recently, a number of studies have proposed solutions for applying controlled deformations to soft objects by robots, leading to the emergence of a new field of robotics research that is called « Shape servoing ».
Controlling the deformation of soft materials requires the knowledge of how displacements of the robotic manipulator are translated into material deformation. This relationship can be either estimated from past visual observation with data-driven approaches [1] or expressed by a physics-model of the object such as finite element [2] or mass-spring models [3-4].
Objectives
The various existing approaches need to extract visual geometric features to represent the object current deformations. For example they use 3D points of the objects surface provided by an RGB-D camera. Therefore, one of their limitations is the requirement of a real-time process that extracts and tracks these geometric features. This process usually involves a segmentation process which is only optimized and efficient for a specific type of object.
To avoid the need for extracting geometric features, we will propose in this thesis to consider directly the photometric information as well as the depth information provided by the RGB-D camera as the visual features to be regulated by the control law. This was already done for positioning a camera with respect to a rigid object by the “direct visual servoing” concept introduced in [5-6]. However, at our best knowledge photometric features were never considered for shape visual servoing.
This thesis will therefore focus on the elaboration of a new shape servoing framework that we call “photometric shape servoing”. One of the scientific contributions will concern the modelling of the interaction that provides the variation of the observed photometric information with respect to the motion of the robot acting on the soft object. This interaction model will then be considered in the design of a robot controller that will allow the autonomous shaping of a soft object by several robotic arms. A reduction of the dimensionality of the photometric shape features will be also considered thanks to projection approaches [7-8].
The methods studied will be developed, tested and validated on an experimental bench consisting of deformable objects, depth cameras, 2 robotic arms with 6 degrees of freedom, each equipped with an force sensor and a robotic griper/hand.
Qualification requirements:
The candidate must have an excellent track of records and a Master Degree (or equivalent) in robotics and computer vision.
The candidate must have the following qualifications:
- Strong background in robotics
- Experience with computer vision, physical robots, or 3D simulation
- Excellent programming skills in C++
- Strong proficiency in both written and spoken English
- Ability to perform experimental validations
- Ability to work independently as well as collaboratively
Application
Please send your CV, motivation letter, recommandation letters and list of marks (even preliminary) of your Master 2 or engineer formation to Eric.Marchand@irisa.fr and alexandre.krupa@inria.fr
Application deadline: May 15, 2024
References
[1] R. Lagneau, A. Krupa, M. Marchal. Automatic Shape Control of Deformable Wires based on Model-Free Visual Servoing. IEEE Robotics and Automation, 5(4):5252-5259, October 2020.
[2] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit and C. Duriez, « FEM-Based Deformation Control for Dexterous Manipulation of 3D Soft Objects, » In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’18, Pages 4007-4013, Madrid, Spain, October 2018.
[3] F. Makiyeh, F. Chaumette, M. Marchal, A. Krupa. Shape Servoing of a Soft Object Using Fourier Series and a Physics-based Model. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’23, Pages 6356-6363, Detroit, USA, October 2023.
[4] F. Makiyeh, M. Marchal, F. Chaumette, A. Krupa. Indirect Positioning of a 3D Point on a Soft Object Using RGB-D Visual Servoing and a Mass-Spring Model. In Int. Conf. on Control, Automation, Robotics and Vision, ICARCV’22, Singapore, December 2022.
[5] C. Collewet, E. Marchand, F. Chaumette. Visual servoing set free from image processing. In IEEE Int. Conf. on Robotics and Automation, ICRA’08, Pages 81-86, Pasadena, Californie, Mai 2008.
[6] C. Collewet, E. Marchand. Photometric visual servoing. IEEE Trans. on Robotics, 27(4):828-834, August 2011.
[7] E. Marchand. Direct visual servoing in the frequency domain. IEEE Robotics and Automation Letters, 5(2):620-627, April 2020.
[8] E. Marchand. Subspace-based Visual Servoing. IEEE Robotics and Automation Letters, 4(3):2699-2706, July 2019.
