Internship On Multimodal Large Language Models
il y a 5 jours
Multimodality, in particular the incorporation of speech representations into LLMs, enables systems to comprehend and respond to spoken language with unparalleled accuracy and depth. Different methods for fusion have been explored recently, ranging from shallow fusion through discrete speech units modeling [1,2] to more complex approaches for integrating speech representations into the LLM decoding procedure [3,4,5,6].
Beyond conventional text-based interfaces, multimodal LLMs enriched with speech capabilities have the potential to transform various domains, from virtual assistants to accessibility aids and interactive storytelling platforms.
In this internship, the selected student will delve into approaches for integrating speech into LLMs for speech-related tasks.
Required skills
- Last year MSc or PhD student in speech processing and/or NLP-related domains
- Solid deep learning background
- Experience with pytorch toolkit
- This is not a remote position, the student is expected to spend the entire internship in Grenoble, France.
- This is not a summer internship, we require a mínimal availability of 5 months.
References
[1] Kim et al. Unified Speech-Text Pretraining for Spoken Dialog Modeling, NeurIPS, 2024
[2] Chang et al. Exploration of Efficient End-to-End ASR Using Discretized Input from Self-Supervised Learning. Interspeech, 2023
**[3] Tan et al. SSR**: Alignment-Aware Modality Connector for Speech Language Models. arXiv, 2024
**[4] Tang et al. SALMONN**: Towards Generic Hearing Abilities for Large Language Models. ICLR, 2024
**[5] Deng et al. Wav2Prompt**: End-to-End Speech Prompt Generation and Tuning for LLM in Zero and Few-Shot Learning. arXiv, 2024
**[6] Hu et al. WavLLM**: Towards Robust and Adaptive Speech Large Language Model. EMNLP Findings, 2024
Application instructions
About NAVER LABS
NAVER is the #1 Internet portal in Korea with activities that span a wide range of businesses including search, commerce, content, financial and cloud platforms.
NAVER LABS, co-located in Korea and France, is the organization dedicated to preparing NAVER’s future. NAVER LABS Europe is located in a spectacular setting in Grenoble, in the heart of the French Alps. Scientists at NAVER LABS Europe are empowered to pursue long-term research problems that, if successful, can have significant impact and transform NAVER. We take our ideas as far as research can to create the best technology of its kind. Active participation in the academic community and collaborations with world-class public research groups are, among others, important tools to achieve these goals. Teamwork, focus and persistence are important values for us.
NAVER LABS Europe is an equal opportunity employer.
-
Meylan, France NAVER LABS Europe Temps pleinWe’re looking for a Computer Science/Deep Learning PhD student or an outstanding Masters student, for a 5-6 month research internship starting early 2025. The internship will be hosted at NAVER LABS Europe, near Grenoble, France, where you will be integrated in the NLP team. For examples of papers first-authored by former interns around the general topic...
-
Meylan, France Orange Temps pleinVotre rôle est d'effectuer un travail de Post doc sur : " Architecture Description Language et modèle runtime pour le Continuum Cloud2IoT " qui contribuera au projet collaboratif " 5GMetaverse4Industry " financé par la BPI. Contexte global et problématique Orange et INRIA collaborent au travers d'un laboratoire commun et d'une équipe commune sur la...
-
Postdoc architecture Description Language and Model
il y a 2 jours
Meylan, France Orange Temps plein**About the role**: - Postdoc Context and problem: - Scientific objectives - results and issues: - The scientific objective is to contribute to the management of geo-distributed infrastructures built on the Cloud2IoT continuum. This includes characterizing and describing Cloud2IoT infrastructures via an ADL. - The main obstacles to overcome are: - Define...
-
Meylan, Auvergne-Rhône-Alpes, France NAVER LABS Europe Temps pleinAbout NAVER LABS Europe NAVER LABS Europe is part of the R&D division of NAVER, Korea's leading Internet portal and a global tech company with a range of services that include search, commerce, content, fintech, robotics and cloud. The positionWe're looking for a highly experienced and exceptionally talented senior scientist in machine learning and...
-
Developpeur Fullstack
il y a 1 semaine
Meylan, Auvergne-Rhône-Alpes, France lehibou Temps pleinNous recherchons un Développeur Full Stack pour participer au développement d'un système de chatbot. Il/elle sera responsable de la construction des composants front-end et back-end du système, garantissant une expérience utilisateur fluide et des capacités back-end robustes.Ce rôle implique la mise en ?uvre d'interfaces conversationnelles et...
-
Research Scientist in Visual Representation Learning F/M
il y a 7 jours
Meylan, Auvergne-Rhône-Alpes, France NAVER LABS Europe Temps pleinAbout NAVER LABS Europe NAVER LABS Europe is part of the R&D division of NAVER, Korea's leading Internet portal and a global tech company with a range of services that include search, commerce, content, fintech, robotics and cloud. The positionWe are looking for a research scientist to join the Visual Representation Learning (VRL) team at NAVER LABS...
-
Cad Engineer
il y a 1 semaine
Meylan, France Dolphin Design Temps pleinIn a context where connected objects require more and more performance, more and more autonomy, and where data exchanges are exploding, the need for frugality in power consumption becomes crucial. Our technologies reconcile the need for more performance, more frugality but also more sustainability. Dolphin Design ambition is to reconcile technological...
-
Acheteur commodités f/h
il y a 1 semaine
Meylan, Auvergne-Rhône-Alpes, France The businesses of Merck KGaA, Darmstadt, Germany Temps pleinExprimez votre talent avec nous Vous voulez explorer, franchir des obstacles, faire des découvertes ? Nous savons que vos projets sont ambitieux. Les nôtres aussi Dans le monde entier, nos collègues ont la passion de l'innovation scientifique et technologique qui enrichit les vies humaines grâce à nos solutions dans les domaines Healthcare, Life...
-
Senior Digital Designer
il y a 5 jours
Meylan, France Dolphin Design Temps plein**Paul's Digital Design team is looking for a Senior Digital Designer Audio !** To support the growth of its **Audio & Metering** team and to meet the ever-increasing challenges of its customers regarding the density, power consumption and operating frequency of their integrated circuits, Dolphin Design is looking for a **Senior Digital Designer (F/M) based...
