Research Internship

**Company Description** About Ubisoft**

Ubisoft’s 19,000 team members, working across more than 30 countries around the world, are bound by a common mission to enrich players’ lives with original and memorable gaming experiences. Their commitment and talent have brought to life many acclaimed franchises such as Assassin’s Creed, Far Cry, Watch Dogs, Just Dance, Rainbow Six, and many more to come. Ubisoft is an equal opportunity employer that believes diverse backgrounds and perspectives are key to creating worlds where both players and teams can thrive and express themselves. If you are excited about solving game-changing challenges, cutting edge technologies and pushing the boundaries of entertainment, we invite you to join our journey and help us create the unknown.

**Ubisoft Bordeaux**

Founded in 2017, Ubisoft Bordeaux works with passion on the biggest AAA titles to deliver the best gaming experiences. Today, the studio is composed of 400 talents from 20 different nationalities, working on licenses such as Assassin's Creed, Beyond Good & Evil 2 and a free to play game, BattleCore Arena. At the same time, the studio has set up a Tech branch which works on all Ubisoft's online services (named Online Services) as well as on the Anvil game engine. Ubisoft Bordeaux is also home to a R&D team, La Forge, which brings together engineers and researchers to work together on prototypes for game production, particularly around AI topics.

Recent advancements in Large Language Models (LLM) and Visual Language Models (VLM) are revolutionizing many fields, including video games. Multiple video-game companies are already studying how to leverage LLMs for game-changing innovations, such as enabling players to cooperate in natural language with NPCs (non-player characters). Within academia, multiple research labs are studying how to adapt LLMs/VLMs to new use cases, such as interacting in complex environments through reinforcement learning or evolutionary strategies. These breakthroughs open up vast opportunities for enhancing player immersion through more intelligent and responsive NPCs.

This internship aims to explore the integration of LLMs and VLMs with reinforcement learning (RL) or evolutionary strategies to enhance the intelligence and behavior of NPCs and bots in video games. By combining the creative potential of LLMs with the interactive feedback loops of RL, we aim to build better NPCs and reduce iteration time for game developers.

Relevant literature:

- Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
- Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

**Objectives**:

- Investigate the current uses of LLMs/VLMs in gaming, focusing on how they enhance NPC decision-making.
- Study recent research on using reinforcement learning to improve the performance of LLM/VLMs, with a particular focus on grounding these models in interactive environments such as video games.
- Throughout discussion with production teams, identify a potential real-world use case on which to calibrate prototyping and research.
- Experiment with methods to optimize NPC learning in simulated or live gaming environments, providing a balance between realism and computational efficiency.

**Internship Structure**:
The internship will take place within Ubisoft’s La Forge, where cutting-edge tools and resources are available for research, including a set of realistic 3D games specifically designed for research purposes, large datasets, and state-of-the-art learning algorithms deployed on GPU clusters. This internship will benefit from close collaboration with the Inria Flowers team, led by Pierre-Yves Oudeyer.
**Qualifications**
- You are in the final year of your engineering degree or pursuing a research master’s in computer science or a related field.
- You possess a solid foundation in computer science and mathematics, particularly in areas relevant to machine learning, deep learning, LLMs/VLMs, or reinforcement learning.
- You have experience using Python and machine learning libraries to build and experiment with models.
- Your English proficiency enables you to collaborate effectively in an international environment and communicate clearly with colleagues from diverse backgrounds.
- An interest in game design and game development.

**Additional Information**
This job is open for an **internship (6-month contract).**

**Remote**: hybrid model

**Process**:

- Interview with our recruiter
- One or more technical and project interviews with the manager and his team
- We are working to enrich players’ lives through unique and memorable gaming experiences and by improving the positive impacts of our games. To get there, we are creating a safer, more inclusive work environment, we are giving back to the communities where Ubisoft operates by working with local non-profit partners and by working to reduce the environmental impact of our busin