1,721,132 research outputs found
Machine learning in computer security is difficult to fix
Full text linkThis study is timely and exciting as it clearly points out some common issues hindering the design of ML models for computer security and how to overcome them. This witnesses once again that, despite the impressive performance reported in many of the published papers in this area, the reality is quite different, and applying ML in computer security is much more challenging than it may seem
Wild patterns: ten years after the rise of adversarial machine learning
Full text linkLearning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms
Adversarial Machine Learning: Attacks From Laboratories to the Real World
No full textAdversarial machine learning (AML) is a recent research field that investigates potential security issues related to the use of machine learning (ML) algorithms in modern artificial intelligence (AI)-based systems, along with defensive techniques to protect ML algorithms against such threats. The main threats against ML encompass a set of techniques that aim to mislead ML models through adversarial input perturbations. Unlike ML-enabled crimes, in which ML is used for malicious and offensive purposes, and ML-enabled security mechanisms, in which ML is used for securing existing systems, AML techniques exploit and specifically address the security vulnerabilities of ML algorithms
Explainability-based Debugging of Machine Learning for Vulnerability Discovery
No full textMachine learning has been successfully used for increasingly complex and critical tasks, achieving high performance and efficiency that would not be possible for human operators. Unfortunately, recent studies have shown that, despite its power, this technology tends to learn spurious correlations from data, making it weak and susceptible to manipulation. Explainability techniques are often used to identify the most relevant features contributing to the decision. However, this is often done by taking examples one by one and trying to show the problem locally. To mitigate this issue, we propose in this paper a systematic method to leverage explainability techniques and build on their results to highlight problems in the model design and training. With an empirical analysis on the Devign dataset, we validate the proposed methodology with a CodeBERT model trained for vulnerability discovery, showing that, despite its impressive performances, spurious correlations consistently steer its decision
Pattern recognition systems under attack: design issues and research challenges
No full textWe analyze the problem of designing pattern recognition systems in adversarial settings, under an engineering viewpoint, motivated by their increasing exploitation in security-sensitive applications like spam and malware detection, despite their vulnerability to potential attacks has not yet been deeply understood. We ̄rst review previous work and report examples of how a complex system may be evaded either by leveraging on trivial vulnerabilities of its untrained components, e.g. parsing errors in the pre-processing steps, or by exploiting more subtle vul- nerabilities of learning algorithms. We then discuss the need of exploiting both reactive and proactive security paradigms complementarily to improve the security by design. Our ultimate goal is to provide some useful guidelines for improving the security of pattern recognition in adversarial settings, and to suggest related open issues to foster research in this area
- …
