1,721,009 research outputs found
A survey on modern trainable activation functions
No full textIn neural networks literature, there is a strong interest in identifying and defining activation functions which can improve neural network performance. In recent years there has been a renovated interest in the scientific community in investigating activation functions which can be trained during the learning process, usually referred to as trainable, learnable or adaptable activation functions. They appear to lead to better network performance. Diverse and heterogeneous models of trainable activation function have been proposed in the literature. In this paper, we present a survey of these models. Starting from a discussion on the use of the term “activation function” in literature, we propose a taxonomy of trainable activation functions, highlight common and distinctive proprieties of recent and past models, and discuss main advantages and limitations of this type of approach. We show that many of the proposed approaches are equivalent to adding neuron layers which use fixed (non-trainable) activation functions and some simple local rule that constrains the corresponding weight layers
Improving face recognition in low quality video sequences: Single frame vs multi-frame super-resolution
No full textRe-Identification aims to detect the presence of a subject spotted in one video in other videos. Traditional methods use information extracted from single frames like color, clothes, etc. A sequence in time domain of consecutive subject images could contain a greater amount of information compared with a single image of the same subject. Typically, these sequences are taken from surveillance cameras at very poor resolution. Even with modern cameras the resolution can be a problem when dealing with a subject who is far from the camera. A possible way of handling low resolution images is by using a multi-frame super-resolution algorithm. Multi-frame super-resolution image reconstruction aims at obtaining a high-resolution image by fusing a set of low-resolution images. Low-resolution images are usually subject to some degradation which causes substantial information loss. Therefore, contiguous images in a sequence could be viewed as a degraded version (SR image) of an image at higher resolution (HR image). Using a multi-frame SR algorithm could achieve a restoration of the HR image. This work aims to investigate the possibility of using a multi-frame super-resolution algorithm to enhance the performance of a classic re-identification system by exploiting information provided by video sequences made available by a video surveillance system. In the case that the SR technique employed results in an effective performance enhancement, we intend to show empirically how many match frames are required to have an effective improvement
XAI approach for addressing the dataset shift problem: BCI as a case study
No full textIn the Machine Learning (ML) literature, a well-known problem is the Dataset Shift problem where, differently from the ML standard hypothesis, the data in the training and test sets can follow different probability distributions leading ML systems toward poor generalisation performances. Therefore, such systems can be unreliable and risky, particularly when used in safety-critical domains. This problem is intensely felt in the Brain-Computer Interface (BCI) context, where bio-signals as Electroencephalographic (EEG) are used. In fact, EEG signals are highly non-stationary signals both over time and between different subjects. Despite several efforts in developing BCI systems to deal with different acquisition times or subjects, performance in many BCI applications remains low. Exploiting the knowledge from eXplainable Artificial Intelligence (XAI) methods can help develop EEG-based AI approaches, overcoming the performance returned by the current ones. The proposed framework will give greater robustness and reliability to BCI systems with respect to the current state of the art, alleviating the dataset shift problem and allowing a BCI system to be used by different subjects at different times without the need for further calibration/training stages
A simple and efficient architecture for trainable activation functions
No full textAutomatically learning the best activation function for the task is an active topic in neural network research. At the moment, despite promising results, it is still challenging to determine a method for learning an activation function that is, at the same time, theoretically simple and easy to implement. Moreover, most of the methods proposed so far introduce new parameters or adopt different learning techniques. In this work, we propose a simple method to obtain a trained activation function which adds to the neural network local sub-networks with a small number of neurons. Experiments show that this approach could lead to better results than using a pre-defined activation function, without introducing the need to learn a large number of additional parameters
XAI approach for addressing the dataset shift problem: BCI as a case study
Full text linkIn the Machine Learning (ML) literature, a well-known problem is the Dataset Shift problem where, differently from the ML standard hypothesis, the data in the training and test sets can follow different probability distributions leading ML systems toward poor generalisation performances. Therefore, such systems can be unreliable and risky, particularly when used in safety-critical domains. This problem is intensely felt in the Brain-Computer Interface (BCI) context, where bio-signals as Electroencephalographic (EEG) are used. In fact, EEG signals are highly non-stationary signals both over time and between different subjects. Despite several efforts in developing BCI systems to deal with different acquisition times or subjects, performance in many BCI applications remains low. Exploiting the knowledge from eXplainable Artificial Intelligence (XAI) methods can help develop EEG-based AI approaches, overcoming the performance returned by the current ones. The proposed framework will give greater robustness and reliability to BCI systems with respect to the current state of the art, alleviating the dataset shift problem and allowing a BCI system to be used by different subjects at different times without the need for further calibration/training stages
An XAI-based masking approach to improve classification systems
Full text linkExplainable Artificial Intelligence (XAI) seeks to elucidate the decision-making mechanisms of AI models, enabling users to glean insights beyond the results they produce. While a key objective of XAI is to enhance the performance of AI models through explanatory processes, a notable portion of XAI literature predominantly addresses the explanation of AI systems, with limited focus on leveraging XAI methods for performance improvement. This study introduces a novel approach utilizing Integrated Gradients explanations to enhance a classification system, which is subsequently evaluated on three datasets: Fashion-MNIST, CIFAR10, and STL10. Empirical findings indicate that Integrated Gradients explanations effectively contribute to enhancing classification performance
On the effects of data normalization for domain adaptation on EEG data
No full textIn Machine Learning (ML), a well-known problem is the Dataset Shift problem where the data in the training and test sets can follow different probability distributions, leading ML systems toward poor generalization performances. This problem is intensely felt in Brain-Computer Interfaces (BCIs), where bio-signals as Electroencephalographic (EEG) are often used. Indeed, EEG signals are highly non-stationary both over time and between different subjects. To overcome this problem, several solutions are based on transfer learning approaches such as Domain Adaption (DA). In several cases, however, the actual causes of the improvements remain ambiguous. This paper focuses on the impact of data normalization strategies applied together with DA methods. In particular, using SEED, DEAP, and BCI Competition IV 2a EEG datasets, we experimentally evaluated the impact of different normalization strategies applied with and without several well-known DA methods. It results that the choice of the normalization strategy plays a key role on the classifier performances in DA scenarios, and, often, the use of only an appropriate normalization schema outperforms the DA technique. For SEED and BCI Competition IV 2a, a proper normalization strategy alone in a cross-subject context allows to reach accuracy of 81.52±7.26% and 68.52±11.35%, respectively. In a cross-session context, the accuracy of 86.56±8.15% and 67.82±12.48% for SEED and BCI Competition can be reached, respectively. For DEAP, the best cross-subject performance achieved using only normalization was 39.33±14.08%. All these results are comparable with the performance obtained by several well-known DA strategies
Dynamic Local Filters in Graph Convolutional Neural Networks
No full textOver the last few years, we have seen increasing data generated from non-Euclidean domains, usually represented as graphs with complex relationships. Graph Neural Networks (GNN) have gained a high interest because of their potential in processing graph-structured data. In particular, there is a strong interest in performing convolution on graphs using specific GNN architectures, generally called Graph Convolutional Neural Networks (GCNN). This paper presents a novel method to adapt the behaviour of a GCNN using an input-based dynamically generated filter. Notice that the idea of adapting the network behaviour to the inputs they process to maximize the total performances has aroused much interest in the neural networks literature over the years. The experimental assessment confirms the capabilities of the proposed approach, achieving promising results using simple architectures with a low number of filters
Toward the application of XAI methods in EEG-based systems
Full text linkAn interesting case of the well-known Dataset Shift Problem is the classification of Electroencephalogram (EEG) signals in the context of Brain-Computer Interface (BCI). The non-stationarity of EEG signals can lead to poor generalisation performance in BCI classification systems used in different sessions, also from the same subject. In this paper, we start from the hypothesis that the Dataset Shift problem can be alleviated by exploiting suitable eXplainable Artificial Intelligence (XAI) methods to locate and transform the relevant characteristics of the input for the goal of classification. In particular, we focus on an experimental analysis of explanations produced by several XAI methods on an ML system trained on a typical EEG dataset for emotion recognition. Results show that many relevant components found by XAI methods are shared across the sessions and can be used to build a system able to generalise better. However, relevant components of the input signal also appear to be highly dependent on the input itself
- …
