PhD Position F/M Shape Visual Servoing of Deformable Objects Robust to Model Uncertainties
Contract type : Fixed-term contract
Level of qualifications required : Graduate degree or equivalent
Fonction : PhD Position
Context
Conditions:
- The PhD student will be hosted in the IRISA/Inria Rainbow team (https://team.inria.fr/rainbow/) at Rennes.
- The Ph.D. position is full-time for 3 years (standard duration in France). The position will be paid according to the French salary regulations for PhD students.
Supervisors: Alexandre Krupa, Paolo Robuffo Giordano, Tommaso Belvedere (Rainbow team)
How to apply? VERY IMPORTANT
Interested candidates are requested to apply via this form (https://team.inria.fr/rainbow/fr/appl-form-phd-shap-vis-serv/)
The position will remain open until a satisfactory candidate is found.
Assignment
Context:
A significant challenge in robotics lies in interacting with deformable objects. Most existing robotic control approaches assume that objects in the scene are rigid, rendering them ineffective in handling deformations. On the other hand, many applications of interest (e.g., in the medical domain or food industry) require the capability of manipulating deformable objects. In order to address these challenges, the Rainbow team has proposed over the recent years several novel visual servoing methods that enable robots to autonomously and dexterously manipulate soft objects using real-time visual feedback from an RGB-D depth camera. From an applied perspective, the robotic task involves autonomously and accurately deforming soft objects into desired shapes in an efficient, closed-loop manner using robotic arms.
Main activities
Envisaged Activities:
In order to autonomously manipulate and control the deformation of a soft object, it is crucial to establish the relationship between the robot’s motion and changes in the object’s shape. This relationship can be estimated from previous visual observations using data-driven approaches [1-2], expressed through a physical model of the object such as the finite element model (FEM) [3] or the mass-spring model (MSM) [4-5], or approximated by a geometrical model of deformation [6].
Most state-of-the-art approaches assume that the deformation of the soft object reaches a quasi-static state before the next control motion is applied, since this greatly simplifies the analysis and control design. Consequently, these approaches are unable to account for the inertial properties of the soft material during rapid motion. In this respect, we have recently proposed a control approach based on a coarse physical model of the soft object to be manipulated. This model employs a 3D mesh, representing the object’s mechanical behavior through a mass-spring model (MSM) [4] or finite element model (FEM) [7] to derive the controller analytical expression. Experimental results have demonstrated that our approach can deform a soft object into a desired shape very rapidly (in less than 5 seconds), thus being very far from a (simplified) quasi-static regime. See a video of our results at this link https://www.youtube.com/watch?v=-6V4tlMInqo
However, in these previous works, we assumed that the object was already gripped by the robot at predefined contact points whose 3D positions were known in advance. Furthermore, the physical parameters of the soft object, such as Young’s modulus and Poisson’s coefficient, were also assumed to be known and constant (isotropic) across all parts of the object.
To extend the applicability of shape visual servoing to a wide range of objects, including those lacking a precise model in advance, and to determine optimal grasping points for shaping, this PhD proposes the development of a new shape visual servoing approach that can explicitly account for various model uncertainties. These uncertainties could include the initial positions of the grasping points, which may shift during the shaping task, inaccuracies in the physical model’s parameters, and measurement errors from the visual sensor—all of which can impact task performance in terms of robustness and accuracy.
Two main objectives will be addressed in this thesis to manage uncertainties:
- Minimally Sensitive Trajectory Planning: Develop offline planning methods for time-varying trajectories of grasping points or shape trajectories. These trajectories will minimize the effects of uncertainties, aiming to achieve the closest possible match to the desired shape by the end of the visual servoing task. This research will build upon our recent works on Closed-loop State Sensitivity[8,9,10] in which we proposed a metric that quantifies how variations in model parameters affect the state/inputs of a robot in closed-loop (i.e., by also considering the strengths/weaknesses of the control action). This metric can then be used for trajectory optimization purposes (offline or online as MPC).
- Online Visual Servoing Controller: Create a new shape visual servoing controller that transforms the object’s initial shape into the desired shape while minimizing the system’s sensitivity to inaccuracies in real time, thereby optimizing performance in a similar fashion than what has been done in [9,10] for the case of a torque-controlled manipulator arm and an hexarotor, where the effects of model uncertainties could be significantly mitigated by a proper trajectory planning.
The proposed methods will be developed, tested, and validated on the TIRREX manipulation robotic platform (see pictures), which consists of three robotic arms (each with 7 degrees of freedom) mounted on a mobile base. Two arms are equipped with Allegro hands featuring tactile and force sensors, while the third arm integrates a depth camera. Deformable everyday objects will be used during the manipulation experiments. Two Franka Emika Panda robots equipped with grippers will also be available to test the methods developed.
References:
[1] D. Navarro-Alarcon, Y. Liu, J.G. Romero, and P. Li. On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments. The International Journal of Robotics Research, 33(11):1462– 1480, September 2014.
[2] R. Lageau, A. Krupa, M. Marchal. Automatic Shape Control of Deformable Wires based on Model-Free Visual Servoing. IEEE Robotics and Automation Letters, 5(4):5252-5259, October 2020.
[3] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit and C. Duriez, “FEM-Based Deformation Control for Dexterous Manipulation of 3D Soft Objects,” In IEEE/RSJ Int. Conf.on Intelligent Robots and Systems, IROS’18, Pages 4007-4013, Madrid, Spain.
[4] F. Makiyeh, F. Chaumette, M. Marchal, A. Krupa. Shape Servoing of a Soft Object Using Fourier Series and a Physics-based Model. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’23, Pages 6356-6363, Detroit, USA, October 2023.
[5] F. Makiyeh, M. Marchal, F. Chaumette, A. Krupa. Indirect Positioning of a 3D Point on a Soft Object Using RGB-D Visual Servoing and a Mass-Spring Model. In Int.Conf. on Control, Automation, Robotics and Vision, ICARCV’22, Singapore, December 2022.
[6] M. Shetab-Bushehri, M. Aranda, Y. Mezouar, E. Özgür. Lattice-Based Shape Tracking and Servoing of Elastic Objects. IEEE Transactions on Robotics, 40:364-381, November 2023.
[7] M. Fonkoua Ouafo, F. Chaumette, A. Krupa. Deformation Control of a 3D Soft Object using RGB-D Visual Servoing and FEM-based Dynamic Model. IEEE Robotics and Automation Letters, 9(8):6943-6950, August 2024.
[8] P. Robuffo Giordano, Q. Delamare, A. Franchi. Trajectory Generation for Minimum Closed-Loop State Sensitivity. In IEEE Int. Conf. on Robotics and Automation, ICRA'18, Pages 286-293, Brisbane, Australia, May 2018
[9] A. Srour, A. Franchi, P. Robuffo Giordano, M. Cognetti. Experimental Validation of Sensitivity-Aware Trajectory Planning for a Redundant Robotic Manipulator Under Payload Uncertainty. IEEE Robotics and Automation Letters, December 2024
[10] A. Afifi, T. Belvedere, A. Pupa, P. Robuffo Giordano, A. Franchi. Safe and Robust Planning for Uncertain Robots: A Closed-Loop State Sensitivity Approach. IEEE Robotics and Automation Letters, 9(11):9962-9969, November 2024
Skills
Skills/Requirements:
- M.Sc. degree in computer science, robotics, engineering, applied mathematics (or related fields)
- Good experience in C/C++ , ROS, Matlab/Simulink
- Experience with computer vision, physical robots or 3D simulation
- Scientific curiosity, large autonomy and ability to work independently
How to apply? VERY IMPORTANT
Interested candidates are requested to apply via this form (https://team.inria.fr/rainbow/fr/appl-form-phd-shap-vis-serv/)
The position will remain open until a satisfactory candidate is found.
Benefits package
- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
- Possibility of teleworking (after 6 months of employment) and flexible organization of working hours
- Professional equipment available (videoconferencing, loan of computer equipment, etc.)
- Social, cultural and sports events and activities
- Access to vocational training
- Social security coverage
Remuneration
2,200 brut per month
General Information
- Theme/Domain : Robotics and Smart environments
- Town/city : Rennes
- Inria Center : Centre Inria de l'Université de Rennes
- Starting date : 2025-10-01
- Duration of contract : 3 years
- Deadline to apply : 2025-02-28
Warning : you must enter your e-mail address in order to save your application to Inria. Applications must be submitted online on the Inria website. Processing of applications sent from other channels is not guaranteed.
Instruction to apply
Defence Security :
This position is likely to be situated in a restricted area (ZRR), as defined in Decree No. 2011-1425 relating to the protection of national scientific and technical potential (PPST).Authorisation to enter an area is granted by the director of the unit, following a favourable Ministerial decision, as defined in the decree of 3 July 2012 relating to the PPST. An unfavourable Ministerial decision in respect of a position situated in a ZRR would result in the cancellation of the appointment.
Recruitment Policy :
As part of its diversity policy, all Inria positions are accessible to people with disabilities.
Contacts
- Inria Team : RAINBOW
-
PhD Supervisor :
Krupa Alexandre / alexandre.krupa@inria.fr
About Inria
Inria is the French national research institute dedicated to digital science and technology. It employs 2,600 people. Its 200 agile project teams, generally run jointly with academic partners, include more than 3,500 scientists and engineers working to meet the challenges of digital technology, often at the interface with other disciplines. The Institute also employs numerous talents in over forty different professions. 900 research support staff contribute to the preparation and development of scientific and entrepreneurial projects that have a worldwide impact.