1,081 research outputs found
SparseProp: Efficient Event-Based Simulation and Training of Sparse Recurrent Spiking Neural Networks
No full textSpiking Neural Networks (SNNs) are biologically-inspired models that are capable of processing information in streams of action potentials. However, simulating and training SNNs is computationally expensive due to the need to solve large systems of coupled differential equations. In this paper, we introduce SparseProp, a novel event-based algorithm for simulating and training sparse SNNs. Our algorithm reduces the computational cost of both the forward and backward pass operations from O(N) to O(log(N)) per network spike, thereby enabling numerically exact simulations of large spiking networks and their efficient training using backpropagation through time. By leveraging the sparsity of the network, SparseProp eliminates the need to iterate through all neurons at each spike, employing efficient state updates instead. We demonstrate the efficacy of SparseProp across several classical integrate-and-fire neuron models, including a simulation of a sparse SNN with one million LIF neurons. This results in a speed-up exceeding four orders of magnitude relative to previous event-based implementations. Our work provides an efficient and exact solution for training large-scale spiking neural networks and opens up new possibilities for building more sophisticated brain-inspired models
Sparse chaos in cortical circuits
No full textNerve impulses, the currency of information flow in the brain, are generated by an instability of the neuronal membrane potential dynamics. Neuronal circuits exhibit collective chaos that appears essential for learning, memory, sensory processing, and motor control. However, the factors controlling the nature and intensity of collective chaos in neuronal circuits are not well understood. Here we use computational ergodic theory to demonstrate that basic features of nerve impulse generation profoundly affect collective chaos in neuronal circuits. Numerically exact calculations of Lyapunov spectra, Kolmogorov-Sinai-entropy, and upper and lower bounds on attractor dimension show that changes in nerve impulse generation in individual neurons moderately impact information encoding rates but qualitatively transform phase space structure. Specifically, we find a drastic reduction in the number of unstable manifolds, Kolmogorov-Sinai entropy, and attractor dimension. Beyond a critical point, marked by the simultaneous breakdown of the diffusion approximation, a peak in the largest Lyapunov exponent, and a localization transition of the leading covariant Lyapunov vector, networks exhibit sparse chaos: prolonged periods of near stable dynamics interrupted by short bursts of intense chaos. Analysis of large, more realistically structured networks supports the generality of these findings. In cortical circuits, biophysical properties appear tuned to this regime of sparse chaos. Our results reveal a close link between fundamental aspects of single-neuron biophysics and the collective dynamics of cortical circuits, suggesting that nerve impulse generation mechanisms are adapted to enhance circuit controllability and information flow
Boosting of neural circuit chaos at the onset of collective oscillations
No full textNeuronal spiking activity in cortical circuits is often temporally structured by collective rhythms. Rhythmic activity has been hypothesized to regulate temporal coding and to mediate the flexible routing of information flow across the cortex. Spiking neuronal circuits, however, are non-linear systems that, through chaotic dynamics, can amplify insignificant microscopic fluctuations into network-scale response variability. In nonlinear systems in general, rhythmic oscillatory drive can induce chaotic behavior or boost the intensity of chaos. Thus, neuronal oscillations could rather disrupt than facilitate cortical coding functions by flooding the finite population bandwidth with chaotically-boosted noise. Here we tackle a fundamental mathematical challenge to characterize the dynamics on the attractor of effectively delayed network models. We find that delays introduce a transition to collective oscillations, below which ergodic theory measures have a stereotypical dependence on the delay so far only described in scalar systems and low-dimensional maps. We demonstrate that the emergence of internally generated oscillations induces a complete dynamical reconfiguration, by increasing the dimensionality of the chaotic attractor, the speed at which nearby trajectories separate from one another, and the rate at which the network produces entropy. We find that periodic input drive leads to a dramatic increase of chaotic measures at a the resonance frequency of the recurrent network. However, transient oscillatory input only has a moderate role on the collective dynamics. Our results suggest that simple temporal dynamics of the mean activity can have a profound effect on the structure of the spiking patterns and therefore on the information processing capability of neuronal networks
Lyapunov spectra of chaotic recurrent neural networks
No full textRecurrent networks are widely used as models of biological neural circuits and in artificial intelligence applications. Mean-field theory has been used to uncover key properties of recurrent network models such as the onset of chaos and their largest Lyapunov exponents, but quantities such as attractor dimension and Kolmogorov-Sinai entropy have thus far remained elusive. We calculate the complete Lyapunov spectrum of recurrent neural networks and show that chaos in these networks is extensive with a size-invariant Lyapunov spectrum and attractor dimensions much smaller than the number of phase space dimensions. The attractor dimension and entropy rate increase with coupling strength near the onset of chaos but decrease far from the onset, reflecting a reduction in the number of unstable directions. We analytically approximate the full Lyapunov spectrum using random matrix theory near the onset of chaos for strong coupling and discrete-time dynamics. We show that a generalized time-reversal symmetry of the network dynamics induces a point symmetry of the Lyapunov spectrum reminiscent of the symplectic structure of chaotic Hamiltonian systems. Temporally fluctuating input can drastically reduce both the entropy rate and the attractor dimension. We lay out a comprehensive set of controls for the accuracy and convergence of Lyapunov exponents. For trained recurrent networks, we find that Lyapunov spectrum analysis quantifies error propagation and stability achieved by different learning algorithms. Our methods apply to systems of arbitrary connectivity and highlight the potential of Lyapunov spectrum analysis as a diagnostic for machine learning applications of recurrent networks
Boosting of neural circuit chaos at the onset of collective oscillations
No full textNeuronal spiking activity in cortical circuits is often temporally structured by collective rhythms. Rhythmic activity has been hypothesized to regulate temporal coding and to mediate the flexible routing of information flow across the cortex. Spiking neuronal circuits, however, are non-linear systems that, through chaotic dynamics, can amplify insignificant microscopic fluctuations into network-scale response variability. In nonlinear systems in general, rhythmic oscillatory drive can induce chaotic behavior or boost the intensity of chaos. Thus, neuronal oscillations could rather disrupt than facilitate cortical coding functions by flooding the finite population bandwidth with chaotically-boosted noise. Here we tackle a fundamental mathematical challenge to characterize the dynamics on the attractor of effectively delayed network models. We find that delays introduce a transition to collective oscillations, below which ergodic theory measures have a stereotypical dependence on the delay so far only described in scalar systems and low-dimensional maps. We demonstrate that the emergence of internally generated oscillations induces a complete dynamical reconfiguration, by increasing the dimensionality of the chaotic attractor, the speed at which nearby trajectories separate from one another, and the rate at which the network produces entropy. We find that periodic input drive leads to a dramatic increase of chaotic measures at a the resonance frequency of the recurrent network. However, transient oscillatory input only has a moderate role on the collective dynamics. Our results suggest that simple temporal dynamics of the mean activity can have a profound effect on the structure of the spiking patterns and therefore on the information processing capability of neuronal networks
Dataset: A reanalysis of “Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons”
No full textDataset for Engelken et al., 2016
Dynamical models of cortical circuits
No full textCortical neurons operate within recurrent neuronal circuits. Dissecting their operation is key to understanding information processing in the cortex and requires transparent and adequate dynamical models of circuit function. Convergent evidence from experimental and theoretical studies indicates that strong feedback inhibition shapes the operating regime of cortical circuits. For circuits operating in inhibition-dominated regimes, mathematical and computational studies over the past several years achieved substantial advances in understanding response modulation and heterogeneity, emergent stimulus selectivity, inter-neuron correlations, and microstate dynamics. The latter indicate a surprisingly strong dependence of the collective circuit dynamics on the features of single neuron action potential generation. New approaches are needed to definitely characterize the cortical operating regime
Coat Cooke & Joe Poole | Coat Cooke & Rainer Wiens: Reviews
No full textCoat Cooke album reviews by Randy Raine-Reusch. Coat Cooke (sax); Joe Poole (drums); Rainer Wiens (guitar)
Robert Rainer and Claud Garner
No full textAuthor Claud Garner, right, autographed copies of his second novel while discussing a tour of other Southwest cities with Robert Rainer, representing his publisher, Creative Age Press. Published in the Fort Worth Star - Telegram morning edition, September 29, 1950.https://mavmatrix.uta.edu/specialcollections_startelegram1950s/6596/thumbnail.jp
- …
