1,720,974 research outputs found
Residual Echo State Networks: Residual recurrent neural networks with stable dynamics and fast learning
No full textResidual connections have been established as a staple for modern deep learning architectures. Most of their applications are cast towards feedforward computing. In this paper, we study the architectural bias of residual connections in the context of recurrent neural networks (RNNs), specifically in the temporal dimension. We frame our discussion from the perspective of Reservoir Computing and dynamical system theory, focusing on important aspects of neural computation like memory capacity, long-term information processing, stability, and nonlinear computation capability. Experiments corroborate the striking advantage brought by temporal residual connections for a plethora of different time series processing tasks, comprehending memory-based, forecasting, and classification problems
Edge of Stability Echo State Network
No full textEcho state networks (ESNs) are time series processing models working under the echo state property (ESP) principle. The ESP is a notion of stability that imposes an asymptotic fading of the memory of the input. On the other hand, the resulting inherent architectural bias of ESNs may lead to an excessive loss of information, which in turn harms the performance in certain tasks with long short-term memory requirements. To bring together the fading memory property and the ability to retain as much memory as possible, in this article, we introduce a new ESN architecture called the Edge of Stability ESN (). The introduced model is based on defining the reservoir layer as a convex combination of a nonlinear reservoir (as in the standard ESN), and a linear reservoir that implements an orthogonal transformation. In virtue of a thorough mathematical analysis, we prove that the whole eigenspectrum of the Jacobian of the map can be contained in an annular neighborhood of a complex circle of controllable radius. This property is exploited to tune the ’s dynamics close to the edge-of-chaos regime by design. Remarkably, our experimental analysis shows that model can reach the theoretical maximum short-term memory capacity (MC). At the same time, in comparison to conventional reservoir approaches, is shown to offer an excellent trade-off between memory and nonlinearity, as well as a significant improvement of performance in autoregressive nonlinear modeling and real-world time series modeling
Non-dissipative Reservoir Computing Approaches for Time-Series Classification
No full textReservoir Computing (RC) is a consolidated framework for designing fastly trainable recurrent neural systems, where the dynamical component is fixed and initialized to implement a fading memory over the input signal. In this paper, we study the behavior of a recently introduced class of alternative RC approaches in which the fixed dynamical component implements a stable but non-dissipative system, so that the driving temporal signal can be propagated through multiple time steps effectively. We analyze the behavior of two classes of non-dissipative RC in terms of dynamical stability and show the resulting advantages in time-series classification tasks in comparison to conventional RC
Sparse Reservoir Topologies for Physical Implementations of Random Oscillators Networks
No full textPhysical implementation of recurrent neural networks is hindered by the fact that hidden units need to be trained and are often fully-connected. We propose to relieve both these constraints by adopting and improving on an oscillators-based reservoir computing model called Random Oscillators Network (RON). RON is a recurrent neural network composed by damped oscillatory units that showed excellent performance in many sequence processing tasks. RON does not require training of its hidden parameters, since it leverages on a random heterogeneous reservoir. However, the reservoir of RON depends on a fully-connected set of oscillators. In this paper, we propose 6 sparse topologies for RON and we study the performance of the model across different levels of sparsity and different numbers of hidden units. Our experiments highlight that RON can tolerate large levels of sparsity without harming its expressive power, and in most cases even outperforming its fully-connected counterpart. Also RON clearly surpasses Leaky ESN (sparse and fully-connected) and LSTM in all benchmarks. We believe RON to be an ideal candidate for the realization and the study of physical neural networks in the real world
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Dispelling the Myths Behind First-author Citation Counts
Full text linkWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
- …
