Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection

Madeleine Schneider; David Aspinall; Nathaniel D. Bastian

doi:10.1109/BigData52589.2021.9671580

Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection

Madeleine Schneider, David Aspinall, Nathaniel D. Bastian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Adversarial machine learning, a technique which seeks to deceive machine learning (ML) models, threatens the utility and reliability of ML systems. This is particularly relevant in critical ML implementations such as those found in Network Intrusion Detection Systems (NIDS). This paper considers the impact of adversarial influence on NIDS and proposes ways to improve ML based systems. Specifically, we consider five feature robustness metrics to determine which features in a model are most vulnerable, and four defense methods. These methods are tested on six ML models with four adversarial sample generation techniques. Our results show that across different models and adversarial generation techniques, there is limited consistency in vulnerable features or in effectiveness of defense method.

Original language	English
Title of host publication	2021 IEEE International Conference on Big Data (Big Data)
Editors	Yixin Chen, Heiko Ludwig, Yicheng Tu, Usama Fayyad, Xingquan Zhu, Xiaohua Hu, Suren Byna, Xiong Liu, Jianping Zhang, Shirui Pan, Vagelis Papalexakis, Jianwu Wang, Alfredo Cuzzocrea, Carlos Ordonez
Publisher	Institute of Electrical and Electronics Engineers
Pages	3343-3352
Number of pages	10
ISBN (Electronic)	978-1-6654-3902-2
ISBN (Print)	978-1-6654-4599-3
DOIs	https://doi.org/10.1109/BigData52589.2021.9671580
Publication status	Published - 13 Jan 2022
Event	2021 IEEE International Conference on Big Data - Online Conference Duration: 15 Dec 2021 → 18 Dec 2021 https://bigdataieee.org/BigData2021/index.html

Conference

Conference	2021 IEEE International Conference on Big Data
Abbreviated title	BigData 2021
Period	15/12/21 → 18/12/21
Internet address	https://bigdataieee.org/BigData2021/index.html

Keywords / Materials (for Non-textual outputs)

machine learning
network security
adversarial artificial intelligence
model robustness evaluation

Access to Document

10.1109/BigData52589.2021.9671580Licence: All Rights Reserved

https://ieeexplore.ieee.org/abstract/document/9671580Licence: All Rights Reserved

Cite this

Schneider, M., Aspinall, D., & Bastian, N. D. (2022). Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection. In Y. Chen, H. Ludwig, Y. Tu, U. Fayyad, X. Zhu, X. Hu, S. Byna, X. Liu, J. Zhang, S. Pan, V. Papalexakis, J. Wang, A. Cuzzocrea, & C. Ordonez (Eds.), 2021 IEEE International Conference on Big Data (Big Data) (pp. 3343-3352). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/BigData52589.2021.9671580

Schneider, Madeleine ; Aspinall, David ; Bastian, Nathaniel D. / Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection. 2021 IEEE International Conference on Big Data (Big Data). editor / Yixin Chen ; Heiko Ludwig ; Yicheng Tu ; Usama Fayyad ; Xingquan Zhu ; Xiaohua Hu ; Suren Byna ; Xiong Liu ; Jianping Zhang ; Shirui Pan ; Vagelis Papalexakis ; Jianwu Wang ; Alfredo Cuzzocrea ; Carlos Ordonez. Institute of Electrical and Electronics Engineers, 2022. pp. 3343-3352

@inproceedings{767cb3c611cf46fea41bc1193ff7411d,

title = "Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection",

abstract = "Adversarial machine learning, a technique which seeks to deceive machine learning (ML) models, threatens the utility and reliability of ML systems. This is particularly relevant in critical ML implementations such as those found in Network Intrusion Detection Systems (NIDS). This paper considers the impact of adversarial influence on NIDS and proposes ways to improve ML based systems. Specifically, we consider five feature robustness metrics to determine which features in a model are most vulnerable, and four defense methods. These methods are tested on six ML models with four adversarial sample generation techniques. Our results show that across different models and adversarial generation techniques, there is limited consistency in vulnerable features or in effectiveness of defense method.",

keywords = "machine learning, network security, adversarial artificial intelligence, model robustness evaluation",

author = "Madeleine Schneider and David Aspinall and Bastian, {Nathaniel D.}",

note = "Funding Information: ACKNOWLEDGMENTS This work was supported in part by the U.S. Army Combat Capabilities Development Command (DEVCOM) C5ISR Center under Support Agreement No. USMA21056 and the U.S. Army DEVCOM Army Research Laboratory under Support Agreement No. USMA21050. This work was conducted as part of an MSc dissertation at the University of Edinburgh. The views expressed in this paper are those of the authors and do not reflect the official policy or position of the United States Military Academy, the United States Army, the United States Department of Defense, or the United States Government. Publisher Copyright: {\textcopyright} 2021 IEEE.; 2021 IEEE International Conference on Big Data, BigData 2021 ; Conference date: 15-12-2021 Through 18-12-2021",

year = "2022",

month = jan,

day = "13",

doi = "10.1109/BigData52589.2021.9671580",

language = "English",

isbn = "978-1-6654-4599-3",

pages = "3343--3352",

editor = "Yixin Chen and Heiko Ludwig and Yicheng Tu and Usama Fayyad and Xingquan Zhu and Xiaohua Hu and Suren Byna and Xiong Liu and Jianping Zhang and Shirui Pan and Vagelis Papalexakis and Jianwu Wang and Alfredo Cuzzocrea and Carlos Ordonez",

booktitle = "2021 IEEE International Conference on Big Data (Big Data)",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

url = "https://bigdataieee.org/BigData2021/index.html",

}

Schneider, M, Aspinall, D & Bastian, ND 2022, Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection. in Y Chen, H Ludwig, Y Tu, U Fayyad, X Zhu, X Hu, S Byna, X Liu, J Zhang, S Pan, V Papalexakis, J Wang, A Cuzzocrea & C Ordonez (eds), 2021 IEEE International Conference on Big Data (Big Data). Institute of Electrical and Electronics Engineers, pp. 3343-3352, 2021 IEEE International Conference on Big Data, 15/12/21. https://doi.org/10.1109/BigData52589.2021.9671580

Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection. / Schneider, Madeleine; Aspinall, David; Bastian, Nathaniel D.
2021 IEEE International Conference on Big Data (Big Data). ed. / Yixin Chen; Heiko Ludwig; Yicheng Tu; Usama Fayyad; Xingquan Zhu; Xiaohua Hu; Suren Byna; Xiong Liu; Jianping Zhang; Shirui Pan; Vagelis Papalexakis; Jianwu Wang; Alfredo Cuzzocrea; Carlos Ordonez. Institute of Electrical and Electronics Engineers, 2022. p. 3343-3352.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection

AU - Schneider, Madeleine

AU - Aspinall, David

AU - Bastian, Nathaniel D.

N1 - Funding Information: ACKNOWLEDGMENTS This work was supported in part by the U.S. Army Combat Capabilities Development Command (DEVCOM) C5ISR Center under Support Agreement No. USMA21056 and the U.S. Army DEVCOM Army Research Laboratory under Support Agreement No. USMA21050. This work was conducted as part of an MSc dissertation at the University of Edinburgh. The views expressed in this paper are those of the authors and do not reflect the official policy or position of the United States Military Academy, the United States Army, the United States Department of Defense, or the United States Government. Publisher Copyright: © 2021 IEEE.

PY - 2022/1/13

Y1 - 2022/1/13

N2 - Adversarial machine learning, a technique which seeks to deceive machine learning (ML) models, threatens the utility and reliability of ML systems. This is particularly relevant in critical ML implementations such as those found in Network Intrusion Detection Systems (NIDS). This paper considers the impact of adversarial influence on NIDS and proposes ways to improve ML based systems. Specifically, we consider five feature robustness metrics to determine which features in a model are most vulnerable, and four defense methods. These methods are tested on six ML models with four adversarial sample generation techniques. Our results show that across different models and adversarial generation techniques, there is limited consistency in vulnerable features or in effectiveness of defense method.

AB - Adversarial machine learning, a technique which seeks to deceive machine learning (ML) models, threatens the utility and reliability of ML systems. This is particularly relevant in critical ML implementations such as those found in Network Intrusion Detection Systems (NIDS). This paper considers the impact of adversarial influence on NIDS and proposes ways to improve ML based systems. Specifically, we consider five feature robustness metrics to determine which features in a model are most vulnerable, and four defense methods. These methods are tested on six ML models with four adversarial sample generation techniques. Our results show that across different models and adversarial generation techniques, there is limited consistency in vulnerable features or in effectiveness of defense method.

KW - machine learning

KW - network security

KW - adversarial artificial intelligence

KW - model robustness evaluation

U2 - 10.1109/BigData52589.2021.9671580

DO - 10.1109/BigData52589.2021.9671580

M3 - Conference contribution

SN - 978-1-6654-4599-3

SP - 3343

EP - 3352

BT - 2021 IEEE International Conference on Big Data (Big Data)

A2 - Chen, Yixin

A2 - Ludwig, Heiko

A2 - Tu, Yicheng

A2 - Fayyad, Usama

A2 - Zhu, Xingquan

A2 - Hu, Xiaohua

A2 - Byna, Suren

A2 - Liu, Xiong

A2 - Zhang, Jianping

A2 - Pan, Shirui

A2 - Papalexakis, Vagelis

A2 - Wang, Jianwu

A2 - Cuzzocrea, Alfredo

A2 - Ordonez, Carlos

PB - Institute of Electrical and Electronics Engineers

T2 - 2021 IEEE International Conference on Big Data

Y2 - 15 December 2021 through 18 December 2021

ER -

Schneider M, Aspinall D, Bastian ND. Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection. In Chen Y, Ludwig H, Tu Y, Fayyad U, Zhu X, Hu X, Byna S, Liu X, Zhang J, Pan S, Papalexakis V, Wang J, Cuzzocrea A, Ordonez C, editors, 2021 IEEE International Conference on Big Data (Big Data). Institute of Electrical and Electronics Engineers. 2022. p. 3343-3352 doi: 10.1109/BigData52589.2021.9671580

Evaluating Model Robustness to Adversarial Samples in Network Intrusion Detection

Abstract

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this