Recognizing real estate agent ads using machine learning methods

Abstract

A solution of housing stock state open data validation problem for further using the verified data in the urban environment monitoring process is described. The validation problem in this study is the automatic recognition of potentially unreliable ads published by real estate agents. Automation is complicated by the lack of formal criteria for ads affiliation, the existing methods dependence on additional resources and information sources. Existing methods for solving this and similar problems of recognizing unreliable information are considered using fake news identifying as an example. Two groups of methods are distinguished: social-based and content-based. The first are poorly applicable to this work’s task, while the latter marks data being reliable or unreliable based on natural language analysis and classification methods, which is synonymous with the problem posed in this work. The result is the problem formulation as the binary classification of ads based on machine learning methods task. A pre-labeled dataset of Moscow apartments ads from open sources was used to train and test the classification models. Six classification models were compared by target metrics: accuracy, precision, recall, F1 and ROC-AUC metrics. The best model by metrics was selected and its hyperparameters were optimized.