Master internship

Master 2 Research Internship – Acoustic Aware Speech Enhancement in Distributed Microphone Arrays

Lab:
Loria / Inria Nancy – Grand Est, Nancy )

Supervisors:
Romain Serizel (LORIA), François Effa (LORIA)

Start:
Spring 2026

Duration:
6 Months

Motivations and context

This internship takes place within the ANR-DFG project AWESOME. The project involves researchers from Université de Lorraine/LORIA in Nancy (France), INRIA in Strasbourg (France) and Dept. of Medical Physics and Acoustics at the University of Oldenburg in Oldenburg (Germany). The internship could potentially lead to a PhD thesis on a related topic.

In many speech communication applications, such as conferencing systems and hearing aids, the microphones capture not only the desired talker but also ambient noise, competing talkers and reverberation, i.e. acoustic reflections from walls and other objects. To improve the quality and intelligibility of recorded speech, various model-based and deep-learning-based speech enhancement algorithms have been proposed, aiming to suppress undesired noise and reverberation without distorting the desired talker [1], [2]. In general, multi-microphone algorithms outperform single-microphone algorithms as they can exploit the spatial information of the sound field in addition to spectro-temporal information.

By considering multiple spatially distributed devices, more detailed information about the sound field can be acquired and the likelihood that some microphones are closer to the (desired and undesired) sound sources is increased. Nevertheless, unlike traditional microphone arrays, acoustic sensor networks can be highly dynamic in the sense that the number and position of the devices is not known and may even vary over time. To support multi-microphone speech enhancement, methods to estimate acoustic parameters of the environment such as reverberation time, room geometry and acoustic reflections can provide valuable information [3].

Goals and Objectives

During this master project we will explore how acoustic scene parameters can be integrated explicitly into the loss function of the speech enhancement algorithms. This approach aims to improve the model's performance by providing additional guidance during the training process. One method is multi-task training, where the model is trained to perform multiple related tasks simultaneously. For example, the model can be trained to extract speech while also estimating acoustic parameters such as reverberation time or the positions of the microphone arrays [4]. The motivation here is to incorporate acoustic knowledge within the model and improve its robustness to different acoustic environments. An alternative to multi-task learning is adversarial training [5], where the model is deliberately trained using adversarially altered inputs to force prediction errors. The primary motivation behind these approaches is to enable the model to disentangle the target speech from other scene parameters. By doing so, the model becomes more robust to changes in acoustic conditions such as changes in the positions of the microphone arrays.

Profile

• Excellent level in Python programming. PyTorch knowledge is an added value.
• Training in Deep Learning and Signal Processing. Additional knowledge or interest for audio, acoustics, numerical methods or optimization are an added value.
• 2nd year master level (in computer science, signal processing, machine learning, acoustics or applied mathematics) with a strong interest for academic research

Bibliography

[1]​ S. Doclo, W. Kellermann, S. Makino, and S. E. Nordholm, 'Multichannel Signal Enhancement Algorithms for Assisted Listening Devices: Exploiting spatial diversity using multiple microphones', IEEE Signal Processing Magazine, vol. 32, no. 2, pp. 18–30, Mar. 2015.

[2]​ R. Haeb-Umbach, T. Nakatani, M. Delcroix, C. Boeddeker, and T. Ochiai, 'Microphone Array Signal Processing and Deep Learning for Speech Enhancement: Combining model-based and data-driven approaches to parameter estimation and filtering', IEEE Signal Processing Magazine, vol. 41, no. 6, pp. 12–23, Nov. 2024.

[3]​ D. De Carlo, P. Tandeitnik, C. Foy, N. Bertin, A. Deleforge, and S. Gannot, 'dEchorate: a calibrated room impulse response dataset for echo-aware signal processing', EURASIP Journal on Audio, Speech, and Music Processing, vol. 2021, no. 1, p. 39, Nov. 2021.

[4]​ R. Giri, M. L. Seltzer, J. Droppo, and D. Yu, 'Improving speech recognition in reverberation using a room-aware deep neural network and multi-task learning', in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr. 2015, pp. 5014–5018.

[5]​ G. Lample, N. Zeghidour, N. Usunier, A. Bordes, L. DENOYER, and M. A. Ranzato, 'Fader Networks: Manipulating Images by Sliding Attributes', in Advances in Neural Information Processing Systems, 2017.