Phd Position F/m Shape Visual Servoing of Deformable Objects Robust to Model Uncertainties

Le descriptif de l’offre ci-dessous est en Anglais_

**Type de contrat**: CDD

**Niveau de diplôme exigé**: Bac + 5 ou équivalent

**Fonction**: Doctorant

**Contexte et atouts du poste**:
**Conditions**:

- The Ph.D. position is full-time for 3 years (standard duration in France). The position will be paid according to the French salary regulations for PhD students.

**Supervisors**: Alexandre Krupa, Paolo Robuffo Giordano, Tommaso Belvedere (Rainbow team)

**How to apply? VERY IMPORTANT**

**Mission confiée**:
**Context**:
**Principales activités**:
**Envisaged Activities**:
In order to autonomously manipulate and control the deformation of a soft object, it is crucial to establish the relationship between the robot’s motion and changes in the object’s shape. This relationship can be estimated from previous visual observations using data-driven approaches [1-2], expressed through a physical model of the object such as the finite element model (FEM) [3] or the mass-spring model (MSM) [4-5], or approximated by a geometrical model of deformation [6].

However, in these previous works, we assumed that the object was already gripped by the robot at predefined contact points whose 3D positions were known in advance. Furthermore, the physical parameters of the soft object, such as Young’s modulus and Poisson’s coefficient, were also assumed to be known and constant (isotropic) across all parts of the object.

To extend the applicability of shape visual servoing to a wide range of objects, including those lacking a precise model in advance, and to determine optimal grasping points for shaping, this PhD proposes the development of a **new shape visual servoing approach** that can explicitly account for various **model uncertainties**. These uncertainties could include the **initial positions of the grasping points**, which may shift during the shaping task, **inaccuracies in the physical model’s parameters**, and **measurement errors from the visual sensor**—all of which can impact task performance in terms of robustness and accuracy.

Two main objectives will be addressed in this thesis to manage uncertainties:

- **Minimally Sensitive Trajectory Planning**: Develop offline planning methods for time-varying trajectories of grasping points or shape trajectories. These trajectories will minimize the effects of uncertainties, aiming to achieve the closest possible match to the desired shape by the end of the visual servoing task. This research will build upon our recent works on **Closed-loop State Sensitivity**[8,9,10] in which we proposed a **metric that quantifies how variations in model parameters affect the state/inputs of a robot in closed-loop** (i.e., by also considering the strengths/weaknesses of the control action). This metric can then be used for trajectory optimization purposes (offline or online as MPC).
- **Online Visual Servoing Controller**: Create a new shape visual servoing controller that transforms the object’s initial shape into the desired shape while **minimizing the system’s sensitivity to inaccuracies in real time**, thereby optimizing performance in a similar fashion than what has been done in [9,10] for the case of a torque-controlled manipulator arm and an hexarotor, where the effects of model uncertainties could be significantly mitigated by a proper trajectory planning.

The proposed methods will be developed, tested, and validated on the **TIRREX manipulation robotic platform (see pictures)**, which consists of **three robotic arms** (each with 7 degrees of freedom) mounted on a **mobile base**. Two arms are equipped with Allegro hands featuring tactile and force sensors, while the third arm integrates a depth camera. Deformable everyday objects will be used during the manipulation experiments. Two Franka Emika Panda robots equipped with grippers will also be available to test the methods developed.

**References**:
[1] D. Navarro-Alarcon, Y. Liu, J.G. Romero, and P. Li. On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments. The International Journal of Robotics Research, 33(11):1462- 1480, September 2014.
[2] R. Lageau, A. Krupa, M. Marchal. Automatic Shape Control of Deformable Wires based on Model-Free Visual Servoing. IEEE Robotics and Automation Letters, 5(4):5252-5259, October 2020.
[3] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit and C. Duriez, “FEM-Based Deformation Control for Dexterous Manipulation of 3D Soft Objects,” In IEEE/RSJ Int. Conf.on Intelligent Robots and Systems, IROS’18, Pages 4007-4013, Madrid, Spain.
[4] F. Makiyeh, F. Chaumette, M. Marchal, A. Krupa. Shape Servoing of a Soft Object Using Fourier Series and a Physics-based Model. In IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, IROS’23, Pages 6356-6363, Detroit, USA, October 2023.
[5] F. Makiyeh, M. Marchal, F. Chaumette, A. Krupa. Indirect Positioning of a 3D Point on a Soft Object Using RGB-D Visual Servoing and a Mass-Spring Model. In Int.Conf.