1,720,997 research outputs found
Explainable AI methods and their interplay with privacy protection
Full text linkRecent years have seen the emergence of Machine Learning models, which are accurate
but lack transparency in their decision-making processes. The field of Explainable Artificial
Intelligence has emerged to address this issue, but many questions remain unanswered.
This Ph.D. Thesis presents two key contributions: (i) a novel variant of a local rule-based
explanation method that provides stable and actionable explanations, and (ii) an investigation
into the relationship between Data Privacy and Explainable Artificial Intelligence,
examining their synergies and tensions.
For (i), an improvement of a local explanation method is designed, using factual logic
rules to explain black-box decisions and providing actionable counterfactual logic rules for
suggesting changes in instances to achieve different outcomes. Explanations are generated
from a decision tree that mimics the local behavior of the black-box model. The decision
tree is obtained through a stability and fidelity-driven ensemble learning approach, where
neighbor instances are synthetically generated using a genetic algorithm guided by the
black-box behavior.
Regarding (ii), two perspectives on privacy are addressed: (a) how Explainable Artificial
Intelligence can enhance individuals’ privacy awareness and (b) how Explainable Artificial
Intelligence can compromise privacy. A framework called Expert is developed to predict
users’ privacy risk and provide explanations, focusing on human mobility data. Additionally,
a visualization module is incorporated to display mobility data explanations on a map.
To assess privacy exposure, instead, a new membership attack for Machine Learning models
is proposed, and a methodology called reveal is introduced to evaluate the privacy
risks associated with local explainers based on surrogate models. The experimental analysis
demonstrates that global explainers pose a more significant threat to individual privacy
compared to local explainers.
These findings highlight the delicate balance between explainability and privacy in developing Artificial Intelligence systems
Evaluating the privacy exposure of interpretable global and local explainers
Full text linkDuring the last few years, the abundance of data has significantly boosted the performance of Machine Learning models, integrating them into several aspects of daily life. However, the rise of powerful Artificial Intelligence tools has introduced ethical and legal complexities. This paper proposes a computational framework to analyze the ethical and legal dimensions of Machine Learning models, focusing specifically on privacy concerns and interpretability. In fact, recently, the research community proposed privacy attacks able to reveal whether a record was part of the black-box training set or inferring variable values by accessing and querying a Machine Learning model. These attacks highlight privacy vulnerabilities and prove that GDPR regulation might be violated by making data or Machine Learning models accessible. At the same time, the complexity of these models, often labelled as “black-boxes”, has made the development of explanation methods indispensable to enhance trust and facilitate their acceptance and adoption in high-stake scenarios. Our study highlights the trade-off between interpretability and privacy protection. By introducing REVEAL, this paper proposes a framework to evaluate the privacy exposure of black-box models and their surrogate-based explainers, whether local or global. Our methodology is adaptable and applicable across diverse black-box models and various privacy attack scenarios. Through an in-depth analysis, we show that the interpretability layer introduced by explanation models might jeopardize the privacy of individuals in the training data of the black-box, particularly with powerful privacy attacks requiring minimal knowledge but causing significant privacy breaches
Pairwise Difference Learning for Classification
Full text linkPairwise difference learning (PDL) has recently been introduced as a new meta-learning technique for regression. Instead of learning a mapping from instances to outcomes in the standard way, the key idea is to learn a function that takes two instances as input and predicts the difference between the respective outcomes. Given a function of this kind, predictions for a query instance are derived from every training example and then averaged. This paper extends PDL toward the task of classification and proposes a meta-learning technique for inducing a PDL classifier by solving a suitably defined (binary) classification problem on a paired version of the original training data. We analyze the performance of the PDL classifier in a large-scale empirical study and find that it outperforms state-of-the-art methods in terms of prediction performance. Last but not least, we provide an easy-to-use and publicly available implementation of PDL in a Python package
A new approach for cross-silo federated learning and its privacy risks
No full textFederated Learning has witnessed an increasing popularity in the past few years for its ability to train Machine Learning models in critical contexts, using private data without moving them. Most of the approaches in the literature are focused on mobile environments, where mobile devices contain the data of single users, and typically deal with images or text data. In this paper, we define hcsfedavg, a novel federated learning approach tailored for training machine learning models on data distributed over federated organizations hierarchically organized. Our method focuses on the generalization capabilities of the neural network models, providing a new mechanism for selecting their best weights. In addition, it is tailored for tabular data.
We empirically test the performance of our approach on two different tabular datasets, showing excellent results in terms of performance and generalization capabilities.
Then, we also tackle the problem of assessing the privacy risk of users represented in the training data. In particular, we empirically show, by attacking the hcsfedavg models with the Membership Inference Attack, that the privacy of the users in the training data may have high risk
Explainable for Trustworthy AI
No full textBlack-box Artificial Intelligence (AI) systems for automated decision making are often based on over (big) human data, map a user’s features into a class or a score without exposing why. This is problematic for the lack of transparency and possible biases inherited by the algorithms from human prejudices and collection artefacts hidden in the training data, leading to unfair or wrong decisions. The future of AI lies in enabling people to collaborate with machines to solve complex problems. This requires good communication, trust, clarity, and understanding, like any efficient collaboration. Explainable AI (XAI) addresses such challenges, and for years different AI communities have studied such topics, leading to different definitions, evaluation protocols, motivations, and results. This chapter provides a reasoned introduction to the work of Explainable AI to date and surveys the literature focusing on symbolic AI-related approaches. We motivate the needs of XAI in real-world and large-scale applications while presenting state-of-the-art techniques and best practices and discussing the many open challenges
Benchmarking and survey of explanation methods for black box models
Full text linkThe rise of sophisticated black-box machine learning models in Artificial Intelligence systems has prompted the need for explanation methods that reveal how these models work in an understandable way to users and decision makers. Unsurprisingly, the state-of-the-art exhibits currently a plethora of explainers providing many different types of explanations. With the aim of providing a compass for researchers and practitioners, this paper proposes a categorization of explanation methods from the perspective of the type of explanation they return, also considering the different input data formats. The paper accounts for the most representative explainers to date, also discussing similarities and discrepancies of returned explanations through their visual appearance. A companion website to the paper is provided as a continuous update to new explainers as they appear. Moreover, a subset of the most robust and widely adopted explainers, are benchmarked with respect to a repertoire of quantitative metrics
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
- …
