Visualizing Embeddings to Study Gender-Related Differences in Word Meaning

Abstract

Development of the models of distributional semantics is one of the most important directions of research in modern NLP. This field is developing rapidly. New transformed-based models allow one to obtain good results in a lot of practical tasks, although the problem of their interpretability remains largely unsolved despite research efforts made in this direction. It should also be noted that, despite the obvious progress in the field, very little attention has been given to the problem of estimating and assessing the differences in word meaning (in the sense of distributional semantics) related to the characteristics of text authors (gender, age, psychological traits, etc.). This problem has not only a theoretical but also a practical value. Currently, no attention is being paid to the characteristics of authors whose texts are used to construct pretrained models widely used in NLP, and knowing individual differences in word meaning is crucial to understanding the biases existing in these models. We use the existing methods of word embedding visualization to show the differences in the structure of word meaning related to the gender of authors and propose clustering methods to study this structure. We conclude that the development of the methods aimed at visualizing and interpreting the individual differences in word meaning is crucial both for the efficient solution of various NLP tasks and for the theory of word meaning.