Optimized Transmission of Spherical Video Streams for Immersive Teleoperation of Autonomous Robots

**Optimized Transmission of Spherical Video Streams for Immersive Teleoperation of Autonomous Robots**:

- Réf **ABG-131047**
- Sujet de Thèse
- 15/04/2025
- Contrat doctoral
- Université Bourgogne Europe, DRIVE UR1859, Nevers
- Lieu de travail- Nevers - Bourgogne-Franche-Comté - France
- Intitulé du sujet- Optimized Transmission of Spherical Video Streams for Immersive Teleoperation of Autonomous Robots
- Champs scientifiques- Informatique
- Sciences de l’ingénieur
- Mots clés- Spherical Video, Teleoperation, Adaptive Streaming, Distributed SLAM, AI, Robotics

**Description du sujet**:
**Context and Motivation**-
Robot teleoperation is a key challenge in many fields, from exploration in hazardous environments (disaster zones, dangerous industrial sites, underwater or space missions) to the maintenance of critical infrastructure and human assistance. The use of **360° cameras** enhances the operator’s perception and immersion, thereby facilitating remote control and interaction with the environment.However, real-time transmission of such high-definition and non-conventional geometry video streams presents several major challenges:
- **Bandwidth optimization**: How can we efficiently transmit spherical video while ensuring low latency and high quality where it matters most?
- **Remote 3D reconstruction and SLAM**: How can visual analysis and simultaneous mapping be combined for better perception and navigation?
- **Context-aware and network-constrained adaptation**: How can transmission be dynamically adjusted based on operator needs and available resources?
This PhD position aims to explore advanced strategies such as adaptive tiling, intelligent multicast, and spherical mesh optimization to enhance video streaming in a shared-control robot teleoperation context.**Scientific and Technical Objectives**
The main goal is to design an efficient transmission framework enabling **smooth interaction** between a remote operator and a robot equipped with **360° cameras**, while minimizing network and computational resource usage.- Optimized Transmission of Spherical Video Streams
- Development of an **adaptive spherical tiling strategy**, transmitting only the most relevant regions in high definition. Leveraging **contextual information**:

- Operator's gaze to prioritize visible tiles.
- Scene dynamics to detect the most informative regions.
- Integration of an **intelligent multicast mechanism**, enabling differentiated transmission depending on corresponding recipients (operator, vision algorithms, archiving, etc.).
- Distributed 3D Reconstruction and SLAM for Enhanced Teleoperation
- Development of a method combining **distributed SLAM and selective tile transmission** based on visual information density.
- Optimizing the tradeoff between **visual quality** (for the operator) and **3D reconstruction accuracy** (for robot navigation).
- Integration of **dynamic network and computing resource management** to adapt to latency and bandwidth constraints.
- Validation and Experiments on Various Robot Platforms
- Experiments will be conducted on humanoid robots at the CNRSAIST JRL lab in Japan, and on drone fleets at the DRIVE lab in Nevers, to evaluate the effectiveness of the proposed approach in diverse, real-world scenarios.
- Performance analysis will include **latency**, **robustness**, and **perceived teleoperation quality**.

**Scientific and Technical Challenges**
- How to ensure **realtime, low-latency transmission** of immersive video streams under constrained networks?
- What is the **optimal spherical tiling and prioritization strategy** to balance immersion and 3D reconstruction performance?
- How to design an **adaptive and intelligent multicast system** optimized for heterogeneous recipients with varying requirements?
- What is the impact of video stream **latency and jitter** on teleoperated driving of humanoids or drones?

**Applications and Impact**
The outcomes of this thesis can be applied in multiple domains:
- **Exploration and intervention** in extreme environments (e.g., rescue robots in disaster zones, space missions).
- **Inspection and maintenance** of critical infrastructure (e.g., industrial or underwater inspection robots).
- **Surveillance and security** (e.g., autonomous patrols with humanin-the-loop).
- **Humanrobot assistance and interaction** (e.g., teleoperated humanoid assistance robots).

**Expected profile**:
**Funding**:**

This position is supported by dual funding:

- The first phase will take place in Nevers, France (funded by a European project).
- The second phase will be conducted in Tsukuba, Japan (funded by AIST Japan).

**Important dates**:
Application deadline**:Early May 2025**

Expected start date**:September / October 2025**

**Prise de fonction**:

- 01/10/2025

**Nature du financement**:

- Contrat doctoral

**Précisions sur le financement**:

- Bourse France - Japon

**Présentation établissement et labo d'accueil**:

- Université Bourgogne Europe, DRIVE UR1859, N