Anomaly detection with autoencoders

Abstract

The purpose of this work is to reduce the cost of troubleshooting digital equipment by improving anomaly recognition methods based on the use of autoencoders. To do this, the authors propose to use two types of autoencoders: a deep feed-forward autoencoder and a deep convolutional autoencoder, and as a comparison, it is proposed to use two supervised machine learning methods: the logistic regression method and the support vector machine. The comparison confirmed the effectiveness of the proposed autoencoders. The NSL-KDD dataset was chosen for experiments with the algorithms. It includes more than 10,000 measurements and 41 parameters characterizing the network flow. This dataset contains both normal and abnormal network stream data. Before training machine learning algorithms and autoencoders, the current data set was pre-processed: correlation elimination, categorical data binarization, data grouping into attack categories. The results of using autoencoders developed by the authors to search for anomalies have shown the effectiveness.