Auto-regulated Traffic Signal Control in Multi-modal Urban Networks Using Graph-based Deep Reinforcement Learning

**Auto-Regulated Traffic Signal Control in Multi-Modal Urban Networks Using Graph-Based Deep Reinforcement Learning**:

- Réf **ABG-131317**
- Sujet de Thèse
- 21/04/2025
- Contrat doctoral
- LICIT laboratory (ENTPE/UGE), Lyon
- Lieu de travail- Lyon - Auvergne-Rhône-Alpes - France
- Intitulé du sujet- Auto-Regulated Traffic Signal Control in Multi-Modal Urban Networks Using Graph-Based Deep Reinforcement Learning
- Champs scientifiques- Informatique

**Description du sujet**:
Traffic Signal Control (TSC) is a cornerstone of urban traffic management, directly impacting traffic efficiency, network stability, and environmental performance [1]. Over the past decade, adaptive and intelligent TSC approaches have become essential tools for mitigating congestion. These methods adjust signal timings based on real-time traffic conditions, helping to reduce delays and improve throughput. Among these approaches, Reinforcement Learning (RL), particularly Deep Reinforcement Learning (DRL), has emerged as a promising paradigm capable of capturing complex traffic dynamics through interaction with the environment [2].

In real-world traffic networks, intersections are inherently interdependent: the conditions at one intersection are influenced by upstream inflows and downstream congestion, forming tightly coupled spatial dependencies. This complexity becomes more pronounced when multiple intersections share major traffic flows or transit routes. As such, isolated signal optimization is often insufficient. Recent work has explored Multi-Agent Reinforcement Learning (MARL) to coordinate control across multiple intersections via distributed agents. These decentralized approaches offer scalability and robustness but require careful coordination strategies to avoid myopic or conflicting decisions [3].

Challenges in Coordination and Perception

A critical open issue remains: (i) how intersections can effectively exchange and process relevant information, and (ii) to what extent an intersection is interlinked with others [4]. In most practical deployments, controllers use data only from signalized intersections, without considering the impact of non-signalized nodes (e.g., roundabouts or priority-to-the-right junctions), which are common in urban networks. These elements can significantly affect the dynamics of nearby controlled intersections.

This issue can be interpreted as a Partial Observability problem, similar to those encountered when deploying agents in real-world scenarios. There is thus a need to develop models capable of capturing heterogeneous neighborhood effects—i.e., identifying which nearby nodes influence a given intersection and integrating only the relevant information into the decision-making process [4,5].

When such neighborhood information is incorporated into the agent controlling an intersection [4], the process is typically mono-directional: the surrounding context is used to enhance the agent's perception, but without introducing a truly mutual relationship aimed at cooperation. As a result, agents tend to maintain selfish decisions, with little consideration for the impact on their surroundings, even if their perception is augmented by local context.

Need for Dynamic Protection and Proactive Coordination

Furthermore, even with sophisticated multi-agent control, oversaturated conditions (e.g., during rush hours or major public events) can lead to gridlock and systemic collapse due to spillback effects. To address this, the concept of Perimeter Control has been proposed [6], which involves restricting vehicle inflows into high-demand areas to preserve internal flow conditions. However, most existing approaches rely on static boundaries and centralized coordination, limiting scalability, transferability, and adaptability to real-time changes.

There is a pressing need for adaptive, agent-driven perimeter protection, capable of dynamically identifying and regulating protected zones based on local observations and decentralized operations [7]. Achieving this requires developing agents with local perception and control, capable of exchanging information with neighbors to foster cooperative behaviors. This is a key step toward the emergence of self-organized, proactive traffic management strategies, particularly in the context of spatially dynamic protected networks.

Embracing Multi-Modality and Multi-Objective Optimization

Managing the multi-objectivity and multi-modality of urban traffic is also becoming increasingly essential. Urban intersections accommodate a wide variety of users, including private vehicles, freight, bicycles, pedestrians, and public transit. Buses, in particular, are sensitive to signal timing and congestion, requiring headway regularity to avoid bunching and ensure reliable service.

Despite some recent progress [8], most RL-based TSC approaches still fail to model real-world bus dynamics, such as open-loop operations or heterogeneous passenger demand. Beyond multi-modality, mult