Abstract
Machine learning is vulnerable to a wide variety of attacks. It is now well understood that by changing the underlying data distribution, an adversary can poison the model trained with it or introduce backdoors. In this paper we present a novel class of training-time attacks that require no changes to the underlying dataset or model architecture, but instead only change the order in which data are supplied to the model. In particular, we find that the attacker can either prevent the model from learning, or poison it to learn behaviours specified by the attacker. Furthermore, we find that even a single adversarially-ordered epoch can be enough to slow down model learning, or even to reset all of the learning progress. Indeed, the attacks presented here are not specific to the model or dataset, but rather target the stochastic nature of modern learning procedures. We extensively evaluate our attacks on computer vision and natural language benchmarks to find that the adversary can disrupt model training and even introduce backdoors.
Original language | English |
---|---|
Title of host publication | Advances in Neural Information Processing Systems 34 proceedings (NeurIPS 2021) |
Editors | M. Ranzato, A. Beygelzimer , K. Nguyen, P. S. Liang, J.W. Vaughan, Y. Dauphin |
Publisher | Neural Information Processing Systems |
Number of pages | 12 |
Publication status | Published - 6 Dec 2021 |
Event | Thirty-fifth Conference on Neural Information Processing Systems - Virtual Duration: 6 Dec 2021 → 14 Dec 2021 https://nips.cc/Conferences/2021 |
Publication series
Name | Advances in Neural Information Processing Systems |
---|---|
ISSN (Print) | 1049-5258 |
Conference
Conference | Thirty-fifth Conference on Neural Information Processing Systems |
---|---|
Abbreviated title | NeurIPS 2021 |
Period | 6/12/21 → 14/12/21 |
Internet address |