Research and classification of the main types of attacks on artificial intelligence systems in cybersecurity
Abstract
The modern development of artificial intelligence (AI) and machine learning (ML) opens up new opportunities in the field of cybersecurity, but at the same time creates serious challenges in the form of intelligent cyberattacks. The study is devoted to the analysis and classification of ways to use AI for malicious purposes and the study of effective methods to counter such threats. In particular, the article covers the main types of attacks using ML technologies, which demonstrate how attackers can manipulate machine learning algorithms, undermine trust in data, and bypass protection systems. Special attention is paid to the mechanisms of data poisoning attacks, as they are considered the most influential in machine learning, which involve introducing malicious data into the process of training models, which leads to distortion of results and undermines the effectiveness of security algorithms. Privacy attacks are analyzed as a way to obtain confidential information from ML models, which can be used to steal user data. Abuse attacks demonstrate how attackers can use AI tools to automate attacks, scale phishing campaigns, and analyze vulnerabilities in defense systems. The relevance of the study is due to the fact that traditional approaches to cyber defense are no longer able to effectively counter threats that adapt and evolve due to machine learning. The article emphasizes the critical importance of researching defense methods, in particular, building reliable machine learning systems that have built-in mechanisms for detecting anomalies and adapting to new threats. One of the key approaches is federated learning, which allows training models without centralized data storage, reducing the risk of information leakage. The development of deep learning in the field of cyber defense is also considered, which allows analyzing behavioral patterns of threats in real time. The combination of technological measures with human control remains an important aspect, since, despite the power of AI tools, the human factor remains key in the process of ensuring cybersecurity. Thus, the article demonstrates the balance between the opportunities and threats of AI in the field of cybersecurity, emphasizing the need for further research in the direction of resilient ML models that can effectively resist attacks. Without proper regulation and control, AI can become not only a defender, but also a tool for attackers, which requires the development of new security strategies and international regulation in the field of cybersecurity.
Downloads
References
Vassilev A., Oprea A., Fordyce A., Anderson H. (2024) Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. (National Institute of Standards and Technology, Gaithersburg, MD) NIST Artificial Intelligence (AI) Report, NIST Trustworthy and Responsible AI NIST AI 100-2e2023. – Access mode: https://doi.org/10.6028/NIST.AI.100-2e2023
Booth H., Souppaya M., Vassilev A., Ogata M., Stanley M., Scarfone K. (2024) Secure Development Practices for Generative AI and Dual-Use Foundation AI Models: An SSDF Community Profile. (National Institute of Standards and Technology, Gaithersburg, MD), NIST Special Publication (SP) NIST SP 800-218A. – Access mode: https://doi.org/10.6028/NIST.SP.800-218A
Oprea A., Singhal A. and Vassilev A.. (2022) Poisoning Attacks Against Machine Learning: Can Machine Learning Be Trustworthy?, in Computer, vol. 55, no. 11, pp. 94-99, Nov. URL: https://doi.org/10.1109/MC.2022.3190787
Hui Wei, Hao Tang, Xuemei Jia, Zhixiang Wang, Hanxun Yu, Zhubo Li, Shin’ichi Satoh, Luc Van Gool, Zheng Wang. (2024) Physical Adversarial Attack Meets Computer Vision: A Decade Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.46, no.12, pp.9797-9817, URL: https://doi.org/10.48550/arXiv.2209.15179
Koundinya A., Patil S., Chandu B. (2024) Data Poisoning Attacks in Cognitive Computing, IEEE 9th International Conference for Convergence in Technology (I2CT), pp.1-4, https://doi.org/10.1109/I2CT61223.2024.10544345
National Institute of Standards and Technology. (2023) Artificial Intelligence Risk Management Framework. (AI RMF 1.0). – Access mode: https://doi.org/10.6028/NIST.AI.100-1
Biggio B., Roli F. (2018) Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84:317–331 https://doi.org/10.48550/arXiv.1712.03141
Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras. (2018) When does machine learning FAIL? generalized transferability for evasion and poisoning attacks. In 27th USENIX Security Symposium (USENIX Security 18), pp. 1299–1316, https://www.usenix.org/conference/usenixsecurity18/presentation/suciu
Biggio B., Nelson B., Laskov P. (2012) Poisoning attacks against support vector machines. In Proceedings of the 29th International Conference on International Conference on Machine Learning, ICML, URL: https://doi.org/10.48550/arXiv.1206.6389
Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. (2018) Trojaning attack on neural networks. In NDSS. The Internet Society, URL: https://dx.doi.org/10.14722/ndss.2018.23291
Kairouz, Peter; McMahan, H. Brendan; Avent, Brendan; Bellet, Aurélien; Bennis, Mehdi; Bhagoji, Arjun Nitin; Bonawitz, Kallista; Charles, Zachary; Cormode, Graham (2021). Advances and Open Problems in Federated Learning. Foundations and Trends in Machine Learning 14 (1–2): https://doi.org/10.1561/2200000083. ISSN 1935-8237
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus. (2014) Intriguing properties of neural networks. In International Conference on Learning Representations, URL: https://doi.org/10.48550/arXiv.1312.6199
Ian Goodfellow, Jonathon Shlens, Christian Szegedy. (2015) Explaining and harnessing adversarial examples. In International Conference on Learning Representations, URL: https://doi.org/10.48550/arXiv.1412.6572
Nicholas Carlini, Chang Liu, Ulfar Erlingsson, Jernej Kos, Dawn Song. (2019) The Secret Sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Security Symposium, USENIX 19, pages 267–284. – URL: https://arxiv.org/abs/1802.08232
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert - Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, Colin Raffel. (2021) Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633–2650. USENIX Association, URL: https://doi.org/10.48550/arXiv.2012.07805
Karan Ganju, Qi Wang, Wei Yang, Carl A. Gunter, and Nikita Borisov. (2018) Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS ’18, page 619–633, New York, NY, USA. Association for Computing Machinery. URL: https://doi.org/10.1145/3243734.3243834
Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, Tudor Dumitras.(2018) When does machine learning FAIL? generalized transferability for evasion and poisoning attacks. In 27th USENIX Security Symposium (USENIX Security 18), pages 1299–1316. URL: https://www.usenix.org/conference/usenixsecurity18/presentation/suciu
Battista Biggio, Blaine Nelson, Pavel Laskov. (2012) Poisoning attacks against support vector machines. In Proceedings of the 29th International Conference on International Conference on Machine Learning, ICML, URL: https://doi.org/10.48550/arXiv.1206.6389
Nihad Hassan. (2024) What is data poisoning (AI poisoning) and how does it work? Search Enterprise AI, TechTarget. – URL: https://www.techtarget.com/searchenterpriseai/definition/data-poisoning-AI- poisoning
Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Jacob Steinhardt, and Alistair Stewart. (2019) Sever: A robust meta-algorithm for stochastic optimization. In International Conference on Machine Learning, pages 1596–1606. PMLR, URL: https://doi.org/10.48550/arXiv.1803.02815
Elan Rosenfeld, Ezra Winston, Pradeep Ravikumar, and Zico Kolter. (2020) Certifed robustness to label-flipping attacks via randomized smoothing. In International Conference on Machine Learning, pages 8230–8241. PMLR, URL: https://doi.org/10.48550/arXiv.2002.03018
The Tactics & Techniques of Adversarial Machine Learning. HiddenLayer/ (2022). – URL: https://hiddenlayer.com/innovation-hub/the-tactics-and-techniques-of-adversarial-ml
Chi Zhang, Zifan Wang, Ravi Mangal, Matt Fredrikson, Limin Jia, Corina Pasareanu. (2023) Transfer Attacks and Defenses for Large Language Models on Coding Tasks. – URL: https://doi.org/10.48550/arXiv.2311.13445
D. Li and Q. Li,(2023) Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection, in IEEE Transactions on Information Forensics and Security. – URL: https://doi.org/10.48550/arXiv.2006.16545
Vassilev A, Oprea A, Fordyce A, Anderson H (2025) Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations. (National Institute of Standards and Technology, Gaithersburg, MD) NIST Artificial Intelligence (AI) Report, NIST Trustworthy and Responsible AI NIST AI 100-2e2025. – URL: https://doi.org/10.6028/NIST.AI.100-2e2025
This work is licensed under a Creative Commons Attribution 4.0 International License.
