Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection - Le Mans Université
Article Dans Une Revue IEEE/ACM Transactions on Audio, Speech and Language Processing Année : 2024

Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection

Résumé

Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD) are key pre-processing tasks for speaker diarization. In the meeting context, it is often easier to capture speech with a distant device. This consideration however leads to severe performance degradation. We study a unified supervised learning framework to solve distant multi-microphone joint VAD and OSD (VAD+OSD). This paper investigates various multi-channel VAD+OSD front-ends that weight and combine incoming channels. We propose three algorithms based on the Self-Attention Channel Combinator (SACC), previously proposed in the literature. Experiments conducted on the AMI meeting corpus exhibit that channel combination approaches bring significant VAD+OSD improvements in the distant speech scenario. Specifically, we explore the use of learned complex combination weights and demonstrate the benefits of such an approach in terms of explainability. Channel combination-based VAD+OSD systems are evaluated on the final back-end task, i.e. speaker diarization, and show significant improvements. Finally, since multi-channel systems are trained given a fixed array configuration, they may fail in generalizing to other array setups, e.g. mismatched number of microphones. A channel-number invariant loss is proposed to learn a unique feature representation regardless of the number of available microphones. The evaluation conducted on mismatched array configurations highlights the robustness of this training strategy.
Fichier principal
Vignette du fichier
2205_paper_shorter-2.pdf (829.25 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04454362 , version 1 (13-02-2024)

Identifiants

  • HAL Id : hal-04454362 , version 1

Citer

Théo Mariotte, Anthony Larcher, Silvio Montrésor, Jean-Hugh Thomas. Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection. IEEE/ACM Transactions on Audio, Speech and Language Processing, In press. ⟨hal-04454362⟩
49 Consultations
27 Téléchargements

Partager

More