186,217 research outputs found
Dynamical systems learning by a circuit theoretic approach
No full textIn this paper, we derive a new general method for both on-line and off-line backward gradient computation of a system output, or cost function, with respect to system parameters, using a circuit theoretic approach. The system can be any causal, in general nonlinear and time-variant, dynamic system represented by a Signal Flow Graph (SFG), in particular any feedforward, time delay or recurrent neural network. The gradient is obtained in a straightforward way by the analysis of two numerical circuits, the original one and its adjoint (obtained from the first by simple transformations) without the complex chain rule expansions of derivatives usually employed
A new IIR-MLP learning algorithm for on-line signal processing
No full textWe propose a new learning algorithm for locally recurrent neural networks, called truncated recursive backpropagation which can be easily implemented on-line with good performance. Moreover it generalises the algorithm proposed by Waibel et al. (1989) for TDNN, and includes the Back and Tsoi (1991) algorithm as well as BPS and standard on-line backpropagation as particular cases. The proposed algorithm has a memory and computational complexity that can be adjusted by a careful choice of two parameters h and h' and so it is more flexible than a previous algorithm proposed by us. Although for the sake of brevity we present the new algorithm only for IIR-MLP networks, it can be applied also to any locally recurrent neural network. Some computer simulations of dynamical system identification tests, reported in literature, are also presented to assess the performance of the proposed algorithm applied to the IIR-MLP
A new unsupervised neural learning rule for orthonormal signal processing
No full textWe derive a new class of neural unsupervised learning rules which arises from the analysis of the dynamics of an abstract mechanical system. The corresponding algorithms can be used to solve several problems in the area of digital signal processing, where orthonormal matrices are involved. We present an application which deals with blind separation of sources, i.e. a new method to perform efficient independent component analysis (ICA) of random signal
New second-order algorithms for recurrent neural networks based on conjugate gradient
No full textWe derive two second-order algorithms, based on the conjugate gradient method, for online training of recurrent neural networks. These algorithms use two different techniques to extract second-order information on the Hessian matrix without calculating or storing it and without making numerical approximations. Several simulation results for nonlinear system identification tests by locally recurrent neural networks are reported for both the off-line and online case
Real time system modelling using locally recurrent neural networks
No full textIn this paper dynamic neural networks for system modelling are considered: architectural issues are presented but the paper focuses on learning algorithms that work real-time. A recent architecture called locally recurrent neural network is presented in its different versions and compared to traditional networks internally static but provided with external buffer and MLP with finite memory synapses. Simulations results show better modelling performance for locally recurrent networks and so an improved training algorithm is developed for them: causal backpropagation through time. Validation tests shows that the networks are modelling the underlying system and not just overfitting the dat
On-line learning algorithms for neural networks with IIR synapses
No full textThis paper is focused on the learning algorithms for dynamic multilayer perceptron neural networks where each neuron synapsis is modelled by an infinite impulse response (IIR) filter (IIR MLP). In particular, the backpropagation through time (BPTT) algorithm and its less demanding approximated on-line versions are considered. In fact it is known that the BPTT algorithm is not causal and therefore can be implemented only in batch mode, while many real problems require on-line adaptation. In this paper the authors give the complete BPTT formulation for the IIR MLP, derive an already known on-line learning algorithm as a particular approximation of the BPTT, and propose a new approximated algorithm. Several computer simulations of identification of dynamical systems are also presented to assess the performance of the approximated algorithms and to compare the IIR MLP with more traditional dynamic networks
A Unifying View of Gradient Calculations and Learning for Locally Recurrent Neural Networks
No full text- …
