1,721,223 research outputs found
Neural Network for Graphs: A Contextual Constructive Approach
No full textThis paper presents a new approach for learning in structured domains (SDs) using a constructive neural network for graphs (NN4G). The new model allows the extension of the input domain for supervised neural networks to a general class of graphs including both acyclic/cyclic, directed/undirected labeled graphs. In particular, the model can realize adaptive contextual transductions, learning the mapping from graphs for both classification and regression tasks. In contrast to previous neural networks for structures that had a recursive dynamics, NN4G is based on a constructive feedforward architecture with state variables that uses neurons with no feedback connections. The neurons are applied to the input graphs by a general traversal process that relaxes the constraints of previous approaches derived by the causality assumption over hierarchical input data. Moreover, the incremental approach eliminates the need to introduce cyclic dependencies in the definition of the system state variables. In the traversal process, the NN4G units exploit (local) contextual information of the graphs vertices. In spite of the simplicity of the approach, we show that, through the compositionality of the contextual information developed by the learning, the model can deal with contextual information that is incrementally extended according to the graphs topology. The effectiveness and the generality of the new approach are investigated by analyzing its theoretical properties and providing experimental results
Recursive Processing of Structured Domains in Machine Learning -PhD Thesis, TD-13/03
No full textPhD Thesis, TD-13/0
On the Predictive Effects of Markovian and Architectural Factors of Echo State Networks
No full textEcho State Networks (ESNs) represent an emerging paradigm for modeling Recurrent Neural Networks (RNNs).In this report we try to identify and investigate some of the main aspects that can be accounted for the success and limitations of this class of models.Independently of the architectural design, we first show the effect on ESNs behavior due to the contractivity of the state transition function and the related Markovian bias.The purpose of our study is also to give an insight on how and why a larger reservoir may improve the predictive performance. We identify four key factors which can influence the performance of ESNs: input variability, multiple time-scales dynamics, non-linear interactions among units and regression in a high dimensional state space. Several variants of the basic ESN model are introduced in order to study these main factors. The proposed variants are tested on four datasets: the Mackey-Glass chaotic time series, the 10th order NARMA system, and two predictive tasks on a symbolic sequence domain with Markovian/anti-Markovian flavor. Experimental evidence shows that all the key identified factors have a major role in determining ESN performances
Richness of Node Embeddings in Graph Echo State Networks
No full textGraph Echo State Networks (GESN) have recently proved effective in node classification tasks, showing particularly able to address the issue of heterophily. While previous literature has analyzed the design of reservoirs for sequence ESN and GESN for graph-level tasks, the factors that contribute to rich node embeddings are so far unexplored. In this paper we analyze the impact of different reservoir designs on node classification accuracy and on the quality of node embeddings computed by GESN using tools from the areas of information theory and numerical analysis. In particular, we propose an entropy measure for quantifying information in node embeddings
A reservoir computing approach for human gesture recognition from kinect data
No full textThis paper describes a novel approach for human gesture recognition from motion data captured by a Kinect camera. The proposed method is based on encoding the temporal history of input data using bidirectional Echo State Networks, whereas the output is computed by means of a multi-layer perceptron with softmax. Results achieved at the time-series classification challenge organized within the 2016 ECML PKDD Workshop on Advanced Analytics and Learning on Temporal Data show the potentiality of the approach
Deep Reservoir Neural Networks for Trees
No full textTree structured data are a flexible tool to properly express many forms of hierarchical information. However, learning of such data through deep recursive models is particularly demanding. We will show through the introduction of the Deep Tree Echo State Network model (DeepTESN) that the randomized Neural Networks framework offers a formidable approach to allow an efficient treatment of learning in tree structured domains by deep architectures. Theoretical properties, for the Reservoir Computing setup constraints, and empirical behavior of the proposed approach are analyzed, showing its feasibility and accuracy
Tree Echo State Networks
Full text linkIn this paper we present the Tree Echo State Network (TreeESN) model, generalizing the paradigm of Reservoir Computing to tree structured data. TreeESNs exploit an untrained generalized recursive reservoir, exhibiting extreme efficiency for learning in structured domains. In addition, we highlight through the paper other characteristics of the approach: First, we discuss the Markovian characterization of reservoir dynamics, extended to the case of tree domains, that is implied by the contractive setting of the TreeESN state transition function. Second, we study two types of state mapping functions to map the tree structured state of TreeESN into a fixed-size feature representation for classification or regression tasks. The critical role of the relation between the choice of the state mapping function and the Markovian characterization of the task is analyzed and experimentally investigated on both artificial and real-world tasks. Finally, experimental results on benchmark and real-world tasks show that the TreeESN approach, in spite of its efficiency, can achieve comparable results with state-of-the-art, although more complex, neural and kernel based models for tree structured data
- …
