Internship On Multimodal Large Language Models
il y a 4 jours
Multimodality, in particular the incorporation of speech representations into LLMs, enables systems to comprehend and respond to spoken language with unparalleled accuracy and depth. Different methods for fusion have been explored recently, ranging from shallow fusion through discrete speech units modeling [1,2] to more complex approaches for integrating speech representations into the LLM decoding procedure [3,4,5,6].
Beyond conventional text-based interfaces, multimodal LLMs enriched with speech capabilities have the potential to transform various domains, from virtual assistants to accessibility aids and interactive storytelling platforms.
In this internship, the selected student will delve into approaches for integrating speech into LLMs for speech-related tasks.
Required skills
- Last year MSc or PhD student in speech processing and/or NLP-related domains
- Solid deep learning background
- Experience with pytorch toolkit
- This is not a remote position, the student is expected to spend the entire internship in Grenoble, France.
- This is not a summer internship, we require a mínimal availability of 5 months.
References
[1] Kim et al. Unified Speech-Text Pretraining for Spoken Dialog Modeling, NeurIPS, 2024
[2] Chang et al. Exploration of Efficient End-to-End ASR Using Discretized Input from Self-Supervised Learning. Interspeech, 2023
**[3] Tan et al. SSR**: Alignment-Aware Modality Connector for Speech Language Models. arXiv, 2024
**[4] Tang et al. SALMONN**: Towards Generic Hearing Abilities for Large Language Models. ICLR, 2024
**[5] Deng et al. Wav2Prompt**: End-to-End Speech Prompt Generation and Tuning for LLM in Zero and Few-Shot Learning. arXiv, 2024
**[6] Hu et al. WavLLM**: Towards Robust and Adaptive Speech Large Language Model. EMNLP Findings, 2024
Application instructions
About NAVER LABS
NAVER is the #1 Internet portal in Korea with activities that span a wide range of businesses including search, commerce, content, financial and cloud platforms.
NAVER LABS, co-located in Korea and France, is the organization dedicated to preparing NAVER’s future. NAVER LABS Europe is located in a spectacular setting in Grenoble, in the heart of the French Alps. Scientists at NAVER LABS Europe are empowered to pursue long-term research problems that, if successful, can have significant impact and transform NAVER. We take our ideas as far as research can to create the best technology of its kind. Active participation in the academic community and collaborations with world-class public research groups are, among others, important tools to achieve these goals. Teamwork, focus and persistence are important values for us.
NAVER LABS Europe is an equal opportunity employer.
-
Meylan, France NAVER LABS Europe Temps pleinWe’re looking for a Computer Science/Deep Learning PhD student or an outstanding Masters student, for a 5-6 month research internship starting early 2025. The internship will be hosted at NAVER LABS Europe, near Grenoble, France, where you will be integrated in the NLP team. For examples of papers first-authored by former interns around the general topic...
-
Meylan, France Orange Temps pleinPostdoc Archi Description Language, modèle runtime pour Continuum Cloud2IoT Orange Meylan, Auvergne-Rhône-Alpes, France L'ambition de la Division Innovation est de porter plus loin l'innovation d'Orange et de renforcer son leadership technologique, en mobilisant nos capacités de recherche pour nourrir une innovation responsable au service de l'humain,...
-
Senior Research Scientist – Machine Learning for Robotics F/M
il y a 2 semaines
Meylan, Auvergne-Rhône-Alpes, France NAVER LABS Europe Temps pleinAbout NAVER LABS Europe NAVER LABS Europe is part of the R&D division of NAVER, Korea's leading Internet portal and a global tech company with a range of services that include search, commerce, content, fintech, robotics and cloud. The positionWe're looking for a highly experienced and exceptionally talented senior scientist in machine learning and...
-
Postdoc: ADL et Runtime pour Cloud2IoT Continuum
il y a 2 semaines
Meylan, France Orange Temps pleinUne entreprise leader en technologie à Meylan cherche un Postdoctorant pour travailler sur l'Architecture Description Language et modéliser le continuum Cloud2IoT. Le candidat idéal doit avoir un doctorat en informatique, avec une solide expertise en modélisation et programmation, ainsi qu'un intérêt pour la recherche appliquée. Ce rôle clé implique...
-
Research Scientist in Visual Representation Learning F/M
il y a 2 semaines
Meylan, Auvergne-Rhône-Alpes, France NAVER LABS Europe Temps pleinAbout NAVER LABS Europe NAVER LABS Europe is part of the R&D division of NAVER, Korea's leading Internet portal and a global tech company with a range of services that include search, commerce, content, fintech, robotics and cloud. The positionWe are looking for a research scientist to join the Visual Representation Learning (VRL) team at NAVER LABS...
-
Senior Digital Designer
il y a 4 jours
Meylan, France Dolphin Design Temps plein**Paul's Digital Design team is looking for a Senior Digital Designer Audio !** To support the growth of its **Audio & Metering** team and to meet the ever-increasing challenges of its customers regarding the density, power consumption and operating frequency of their integrated circuits, Dolphin Design is looking for a **Senior Digital Designer (F/M) based...
-
Technical Program Manager 6
il y a 1 semaine
Meylan, France Lam Research Temps plein**The Group You’ll Be A Part Of** The Global Customer Operations Group brings industry insights and experience to help our customers deliver on their technology roadmaps. We work tirelessly to earn the trust of our customers, because we know that working closely together leads to the highest quality outcomes. **The Impact You’ll Make** You are key...
-
Communication Specialist
il y a 1 semaine
Meylan, France Dolphin Design Temps plein**Are you ready to embark on an exciting adventure in the high-tech world?**: **Do you aspire to drive innovation and growth for an ambitious company?**: **Look no further, as this opportunity is tailor-made for you!**: We are actively seeking a creative and visionary Senior Communication Specialist (F/M) to lead the global communication efforts of our...
