Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus - LIUM - Equipe Language and Speech Technology Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus

Résumé

Word embedding methods allow to represent words as vectors in a space that is structured using word co-occurrences so that words with close meanings are close in this space. These vectors are then provided as input to automatic systems to solve natural language processing problems. Because interpretability is a necessary condition to trusting such systems, interpretability of embedding spaces, the first link in the chain is an important issue. In this paper, we thus evaluate the interpretability of vectors extracted with two approaches: SPINE, a k-sparse auto-encoder, and SINr, a graph-based method. This evaluation is based on a Word Intrusion Task with human annotators. It is operated using a large French corpus, and is thus, as far as we know, the first large-scale experiment regarding word embedding interpretability on this language. Furthermore, contrary to the approaches adopted in the literature where the evaluation is performed on a small sample of frequent words, we consider a more realistic use-case where most of the vocabulary is kept for the evaluation. This allows to show how difficult this task is, even though SPINE and SINr show some promising results. In particular, SINr results are obtained with a very low amount of computation compared to SPINE, while being similarly interpretable.
Fichier principal
Vignette du fichier
2022.lrec-1.469.pdf (268.48 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03770444 , version 1 (06-09-2022)

Identifiants

  • HAL Id : hal-03770444 , version 1

Citer

Thibault Prouteau, Nicolas Dugué, Nathalie Camelin, Sylvain Meignier. Are Embedding Spaces Interpretable? Results of an Intrusion Detection Evaluation on a Large French Corpus. LREC 2022, Jun 2022, Marseille, France. pp.4414-4419. ⟨hal-03770444⟩
162 Consultations
58 Téléchargements

Partager

Gmail Facebook X LinkedIn More