Application of multimodal approach for identifying similarities in multi-dimensional datasets with usage example

Abstract

Increasingly, one can find research from various fields devoted to working with different formats of data, tasks, research perspectives or views on the object of research. Often these studies use the term "multimodal", which at the same time varies greatly from region to region. Thus, researchers from different subject areas are faced with the same task: choosing an approach and processing multimodal data. The problem with this task lies in the narrow area of applicability of the proposed solutions. Then the very concept of multimodality needs formalization. At the same time, it becomes necessary to allocate data structures to work with this concept. It is also important to describe the approaches for analyzing data in the selected data structures. This article proposes a definition of the concept of multimodal data, describes 4 structures for working with it, and also proposes the method for identifying the most similar multimodal objects. In addition, the given structures are illustrated by examples. The method for finding similar multimodal objects is supplemented with modification descriptions so that it can be applied to all 4 multimodal data structures. The article also demonstrates the application of the method on a general example with a description of data structures - the study of cities according to the characteristics of the population, climate and the number of universities. The example of the study is for illustrative purposes only, however, it can be used for further research after verification by appropriate specialists. The article provides methods for configuring the method and recommendations for working with them. Calculated object similarity (coherence power) is a way to define a multidimensional metric over a complex data structure. At the end of the article, directions for further research are given, which are already being carried out by the authors at the present time.