Introduction to Data Poison Attacks on Machine Learning Models

Abstract

This article discusses one of the possible classes of attacks on machine learning systems - poisoning attacks. Classically, poisoning attacks are special modifications of the training data, which are designed to influence the model obtained after training in a necessary way for the attacker. Attacks can be aimed at lowering the overall accuracy or fairness of the model, or at, for example, providing the necessary classification results under certain conditions. The technique for implementing such attacks includes algorithms for determining the elements of training data that are most responsible for learning outcomes (for generated generalizations), minimizing the amount of poisoned data, and also for ensuring maximum invisibility of the changes being made. Among poisoning attacks, the most dangerous are the so-called trojans (backdoors), when, by means of specially prepared training data, they achieve a change in the logic of the model for a certain labeled input data. In addition to modifying training data, poisoning attacks also include direct attacks on ready-made machine learning models or their executable code.