1,720,973 research outputs found
Load Classification: A Case Study for Applying Neural Networks in Hyper-Constrained Embedded Devices
Full text linkThe application of Artificial Intelligence to the industrial world and its appliances has recently grown in popularity. Indeed, AI techniques are now becoming the de-facto technology for the resolution of complex tasks concerning computer vision, natural language processing and many other areas. In the last years, most of the the research community efforts have focused on increasing the performance of most common AI techniques—e.g., Neural Networks, etc.—at the expenses of their complexity. Indeed, many works in the AI field identify and propose hyper-efficient techniques, targeting high-end devices. However, the application of such AI techniques to devices and appliances which are characterised by limited computational capabilities, remains an open research issue. In the industrial world, this problem heavily targets low-end appliances, which are developed focusing on saving costs and relying on—computationally—constrained components. While some efforts have been made in this area through the proposal of AI-simplification and AI-compression techniques, it is still relevant to study which available AI techniques can be used in modern constrained devices. Therefore, in this paper we propose a load classification task as a case study to analyse which state-of-the-art NN solutions can be embedded successfully into constrained industrial devices. The presented case study is tested on a simple microcontroller, characterised by very poor computational performances—i.e., FLOPS –, to mirror faithfully the design process of low-end appliances. A handful of NN models are tested, showing positive outcomes and possible limitations, and highlighting the complexity of AI embedding
GNN2GNN: Graph Neural Networks to Generate Neural Networks
No full textThe success of neural networks (NNs) is tightly linked with their architectural design—a complex problem by itself. We here introduce a novel framework leveraging Graph Neural Networks to Generate Neural Networks (GNN2GNN) where powerful NN architectures can be learned out of a set of available architecture-performance pairs. GNN2GNN relies on a three-way adversarial training of GNN, to optimise a generator model capable of producing predictions about powerful NN architectures. Unlike Neural Architecture Search (NAS) techniques proposing efficient searching algorithms over a set of NN architectures, GNN2GNN relies on learning NN architectural design criteria. GNN2GNN learns to propose NN architectures in a single step – i.e., training of the generator –, overcoming the recursive approach characterising NAS. Therefore, GNN2GNN avoids the expensive and inflexible search of efficient structures typical of NAS approaches. Extensive experiments over two state-of-the-art datasets prove the strength of our framework, showing that it can generate powerful architectures with high probability. Moreover, GNN2GNN outperforms possible counterparts for generating NN architectures, and shows flexibility against dataset quality degradation. Finally, GNN2GNN paves the way towards generalisation between datasets
Measuring Trustworthiness in Neuro-Symbolic Integration
Full text linkNeuro-symbolic integration of symbolic and subsymbolic techniques represents a fast-growing AI trend aimed at mitigating the issues of neural networks in terms of decision processes, reasoning, and interpretability. Several state-of-the-art neuro-symbolic approaches aim at improving performance, most of them focusing on proving their effectiveness in terms of raw predictive performance and/or reasoning capabilities. Meanwhile, few efforts have been devoted to increasing model trustworthiness, interpretability, and efficiency - mostly due to the complexity of measuring effectively improvements in terms of trustworthiness and interpretability. This is why here we analyse and discuss the need for ad-hoc trustworthiness metrics for neurosymbolic techniques. We focus on two popular paradigms mixing subsymbolic computation and symbolic knowledge, namely: (i) symbolic knowledge extraction (SKE), aimed at mapping subsymbolic models into human-interpretable knowledge bases; and (ii) symbolic knowledge injection (SKI), aimed at forcing subsymbolic models to adhere to a given symbolic knowledge. We first emphasise the need for assessing neuro-symbolic approaches from a trustworthiness perspective, highlighting the research challenges linked with this evaluation and the need for ad-hoc trust definitions. Then we summarise recent developments in SKE and SKI metrics focusing specifically on several trustworthiness pillars such as interpretability, efficiency, and robustness of neuro-symbolic methods. Finally, we highlight open research opportunities towards reliable and flexible trustworthiness metrics for neuro-symbolic integration
Towards Quality-of-Service Metrics for Symbolic Knowledge Injection
Full text linkThe integration of symbolic knowledge and sub-symbolic predictors represents a recent popular trend in AI. Among the set of integration approaches, Symbolic Knowledge Injection (SKI) proposes the exploitation of human-intelligible knowledge to steer sub-symbolic models towards some desired behaviour. The vast majority of works in the field of SKI aim at increasing the predictive performance of the sub-symbolic model at hand and, therefore, measure SKI strength solely based on performance improvements. However, a variety of artefacts exist that affect this measure, mostly linked to the quality of the injected knowledge and the underlying predictor. Moreover, the use of injection techniques introduces the possibility of producing more efficient sub-symbolic models in terms of computations, energy, and data required. Therefore, novel and reliable Quality-of-Service (QoS) measures for SKI are clearly needed, aiming at robustly identifying the overall quality of an injection mechanism. Accordingly, in this work, we propose and mathematically model the first – up to our knowledge – set of QoS metrics for SKI, focusing on measuring injection robustness and efficiency gain
EneA-FL: Energy-aware orchestration for serverless federated learning
Full text linkFederated Learning (FL) represents the de-facto standard paradigm for enabling distributed learning over multiple clients in real-world scenarios. Despite the great strides reached in terms of accuracy and privacy awareness, the real adoption of FL in real-world scenarios, in particular in industrial deployment environments, is still an open thread. This is mainly due to privacy constraints and to the additional complexity stemming from the set of hyperparameters to tune when employing AI techniques on bandwidth-, computing-, and energy-constrained nodes. Motivated by these issues, we focus on scenarios where participating clients are characterised by highly heterogeneous computing capabilities and energy budgets proposing EneA-FL, an innovative scheme for serverless smart energy management. This novel approach dynamically adapts to optimise the training process while fostering seamless interaction between Internet of Things (IoT) devices and edge nodes. In particular, the proposed middleware provides a containerised software module that efficiently manages the interaction of each worker node with the central aggregator. By monitoring local energy budget, computational capabilities, and target accuracy, EneA-FL intelligently takes informed decisions about the inclusion of specific nodes in the subsequent training rounds, effectively balancing the tripartite trade-off between energy consumption, training time, and final accuracy. Finally, in a series of extensive experiments across diverse scenarios, our solution demonstrates impressive results, achieving between 30% and 60% lower energy consumption against popular client selection approaches available in the literature while being up to 3.5 times more efficient than standard FL solutions
DETONAR: Detection of Routing Attacks in RPL-based IoT
Full text linkThe Internet of Things (IoT) is a reality that changes several aspects of our daily life, from smart home monitoring to the management of critical infrastructure. The “Routing Protocol for low power and Lossy networks” (RPL) is the only de-facto standardized routing protocol in IoT networks and is thus deployed in environmental monitoring, healthcare, smart building, and many other IoT applications. In literature, we can find several attacks aiming to affect and disrupt RPL-based networks. Therefore, it is fundamental to develop security mechanisms that detect and mitigate any potential attack in RPL-based networks. Current state-of-the-art security solutions deal with very few attacks while introducing heavy mechanisms at the expense of IoT devices and the overall network performance. In this work, we aim to develop an Intrusion Detection System (IDS) capable of dealing with multiple attacks while avoiding any RPL overhead. The proposed system is called DETONAR - DETector of rOutiNg Attacks in Rpl - and it relies on a packet sniffing approach. DETONAR uses a combination of signature and anomaly-based rules to identify any malicious behavior in the traffic (e.g., application and DIO packets). To the best of our knowledge, there are no exhaustive datasets containing RPL traffic for a vast range of attacks. To overcome this issue and evaluate our IDS, we propose RADAR - Routing Attacks DAtaset for Rpl: the dataset contains five simulations for each of the 14 considered attacks in 16 static-nodes networks. DETONAR’s attack detection exceeds 80% for 10 attacks out of 14, while maintaining false positives close to zero
Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review
Full text linkIn this article, we focus on the opacity issue of sub-symbolic machine learning predictors by promoting two complementary activities—symbolic knowledge extraction (SKE) and symbolic knowledge injection (SKI)—from and into sub-symbolic predictors. We consider as symbolic any language being intelligible and interpretable for both humans and computers. Accordingly, we propose general meta-models for both SKE and SKI, along with two taxonomies for the classification of SKE and SKI methods. By adopting an explainable artificial intelligence (XAI) perspective, we highlight how such methods can be exploited to mitigate the aforementioned opacity issue. Our taxonomies are attained by surveying and classifying existing methods from the literature, following a systematic approach, and by generalising the results of previous surveys targeting specific sub-topics of either SKE or SKI alone. More precisely, we analyse 132 methods for SKE and 117 methods for SKI, and we categorise them according to their purpose, operation, expected input/output data and predictor types. For each method, we also indicate the presence/lack of runnable software implementations. Our work may be of interest for data scientists aiming at selecting the most adequate SKE/SKI method for their needs, and may also work as suggestions for researchers interested in filling the gaps of the current state-of-the-art as well as for developers willing to implement SKE/SKI-based technologies
Shallow2Deep: Restraining Neural Networks Opacity through Neural Architecture Search
Full text linkRecently, the Deep Learning (DL) research community has focused on developing efficient and highly performing Neural Networks (NN). Meanwhile, the eXplainable AI (XAI) research community has focused on making Machine Learning (ML) and Deep Learning methods interpretable and transparent, seeking explainability. This work is a preliminary study on the applicability of Neural Architecture Search (NAS) (a sub-field of DL looking for automatic design of NN structures) in XAI. We propose Shallow2Deep, an evolutionary NAS algorithm that exploits local variability to restrain opacity of DL-systems through NN architectures simplification. Shallow2Deep effectively reduces NN complexity – therefore their opacity – while reaching state-of-the-art performances. Unlike its competitors, Shallow2Deep promotes variability of localised structures in NN, helping to reduce NN opacity. The proposed work analyses the role of local variability in NN architectures design, presenting experimental results that show how this feature is actually desirable
Peer-Reviewed Federated Learning
Full text linkWhile representing the de-facto framework for enabling distributed training of Machine Learning models, Federated Learning (FL) still suffers convergence issues when non-Independent and Identically Distributed (non-IID) data are considered. In this context, the local model optimisation on different data distributions generate dissimilar updates, which are difficult to aggregate and translate into sub-optimal convergence. To tackle this issues, we propose Peer-Reviewed Federated Learning (PRFL), an extension of the traditional FL training process inspired by the peer-review procedure common in the academic field, where model updates are reviewed by several other clients in the federation before being aggregated at the server-side. PRFL aims at enabling the identification of relevant updates, while disregarding the ineffective ones. We implement PRFL on top of the Flower FL library, and make Peer-Reviewed Flower a publicly-available library for the modular implementation of any review-based FL algorithm. A preliminary case study on both regression and classification tasks highlights the potential of PRFL, showcasing how the distributed solution can achieve performance similar to that obtained by the corresponding centralised algorithm, even when non-IID data are considered
Graph Neural Networks as the Copula Mundi between Logic and Machine Learning: A Roadmap
Full text linkCombining machine learning (ML) and computational logic (CL) is hard, mostly because of the inherently- different ways they use to represent knowledge. In fact, while ML relies on fixed-size numeric repre- sentations leveraging on vectors, matrices, or tensors of real numbers, CL relies on logic terms and clauses—which are unlimited in size and structure.
Graph neural networks (GNN) are a novelty in the ML world introduced for dealing with graph- structured data in a sub-symbolic way. In other words, GNN pave the way towards the application of ML to logic clauses and knowledge bases. However, there are several ways to encode logic knowledge into graphs: which is the best one heavily depends on the specific task at hand.
Accordingly, in this paper, we (i) elicit a number of problems from the field of CL that may benefit from many graph-related problems where GNN has been proved effective; (ii) exemplify the application of GNN to logic theories via an end-to-end toy example, to demonstrate the many intricacies hidden behind the technique; (iii) discuss the possible future directions of the application of GNN to CL in general, pointing out opportunities and open issues
- …
