Cross-Lingual Transfer Learning for Low-Resource Speech Translation - Le Mans Université Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

Cross-Lingual Transfer Learning for Low-Resource Speech Translation

Résumé

The paper presents a novel three-step transfer learning framework for enhancing cross-lingual transfer from high- to low-resource languages in the downstream application of Automatic Speech Translation. The approach integrates a semantic knowledge-distillation step into the existing two-step cross-lingual transfer learning framework XLS-R. This extra step aims to encode semantic knowledge in the multilingual speech encoder pre-trained via Self-Supervised Learning using unlabeled speech. Our proposed three-step cross-lingual transfer learning framework addresses the large cross-lingual transfer gap (TRFGap) observed in the XLS-R framework between high-resource and low-resource languages. We validate our proposal through extensive experiments and comparisons on the CoVoST-2 benchmark, showing significant improvements in translation performance, especially for low-resource languages, and a notable reduction in the TRFGap.
Fichier principal
Vignette du fichier
Template.pdf (171.79 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04432308 , version 1 (01-02-2024)
hal-04432308 , version 2 (07-02-2024)

Identifiants

  • HAL Id : hal-04432308 , version 2

Citer

Sameer Khurana, Nauman Dawalatabad, Antoine Laurent, Luis Vicente, Pablo Gimeno, et al.. Cross-Lingual Transfer Learning for Low-Resource Speech Translation. 2024. ⟨hal-04432308v2⟩
156 Consultations
193 Téléchargements

Partager

Gmail Facebook X LinkedIn More