Methods to improve the accuracy of machine learning algorithms while reducing the dimensionality of the data set

Abstract

The limited availability of information collection is a factor hindering the application of high-performance machine learning algorithms. The development of methods to improve the accuracy of models while reducing the observation periods, can be an effective tool for prediction in understudied areas. The paper considers the relationship between the dimensionality of the data set and the predictive capabilities of machine learning models, and determines the impact of the number of observations on the accuracy and robustness of models built on ensemble algorithms and regularized regression algorithms. In the course of the experiments, the change in the weighted average absolute error with decreasing the dimensionality of the set was considered, and the algorithms most resistant to this factor were identified. The lower limit of use of ensemble algorithms for detection of regularities and construction of stable model, in regression tasks, in cases of non-linear dependence of target variable with predictors and under condition of absence of high impact of anomalies and noises in data was revealed. The effect of automated Bayesian hyperparameter optimization on model accuracy when the data set is reduced is considered. The models for which pre-optimization of hyperparameters, by means of wood-structured Parzen estimation, is the most preferable are determined.