Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition - Laboratoire Informatique de l'Université du Maine Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition

Résumé

Evaluating automatic speech recognition (ASR) systems is a classical but difficult and still open problem, which often boils down to focusing only on the word error rate (WER). However, this metric suffers from many limitations and does not allow an in-depth analysis of automatic transcription errors. In this paper, we propose to study and understand the impact of rescoring using language models in ASR systems by means of several metrics often used in other natural language processing (NLP) tasks in addition to the WER. In particular, we introduce two measures related to morpho-syntactic and semantic aspects of transcribed words: 1) the POSER (Part-of-speech Error Rate), which should highlight the grammatical aspects, and 2) the Em-bER (Embedding Error Rate), a measurement that modifies the WER by providing a weighting according to the semantic distance of the wrongly transcribed words. These metrics illustrate the linguistic contributions of the language models that are applied during a posterior rescoring step on transcription hypotheses.
Fichier principal
Vignette du fichier
Thibault_Roux___InterSpeech_2022_v2.pdf (130.32 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03712735 , version 1 (04-07-2022)
hal-03712735 , version 2 (17-11-2022)

Identifiants

  • HAL Id : hal-03712735 , version 1

Citer

Thibault Bañeras Roux, Mickael Rouvier, Jane Wottawa, Richard Dufour. Qualitative Evaluation of Language Model Rescoring in Automatic Speech Recognition. Interspeech, Sep 2022, Incheon, South Korea. ⟨hal-03712735v1⟩
373 Consultations
429 Téléchargements

Partager

Gmail Facebook X LinkedIn More