32 research outputs found
Scrittore stoico anonimo, Opera incerta (PHerc. 1020), coll. 104-112. Edizione, introduzione e commento
No full textPHerc. 1020 (SVF 2. 131 = FDS 88) è uno dei sette papiri di sicura o probabile paternità stoica conservati nella collezione ercolanese. Esso è privo di subscriptio, per cui dell’opera in esso conservata si ignorano autore e titolo. Svariati elementi sembrano corroborare la tesi, risalente a Hans von Arnim, che PHerc. 1020 contenga parte di un’opera risalente a Crisippo o a uno dei suoi immediati successori. A favore della paternità crisippea vengono qui forniti nuovi argomenti, che si aggiungono a quelli già addotti da von Arnim, Pohlenz e Keil. Per quanto riguarda il contenuto del libro, non siamo autorizzati a concludere né che esso equivalesse a uno scritto di tipo esclusivamente morale, piuttosto che logico o epistemologico, né che trattasse unicamente del sapiente stoico. Al contrario, dall’esame puntuale del testo, volto in particolare a comprenderlo in relazione alle altre numerose testimonianze sullo Stoicismo antico in nostro possesso, è emerso che esso presentava una singolare compenetrazione di logica, etica ed epistemologia. Facendo uso di nuove metodologie in campo papirologico, i due editori hanno ricostruito per la prima volta l’anatomia del rotolo e la sequenza dei frammenti e hanno ristabilito il testo con nuovi criteri editoriali basandosi sull’autopsia del manoscritto originale. Il presente lavoro consiste in una nuova edizione critica delle ultime otto colonne del papiro (coll. 104-112 Alessandrelli-Ranocchia), le meglio conservate e le uniche sinora edite dagli studiosi, e si inquadra nell’edizione complessiva di PHerc. 1020 programmata nell’ambito del progetto ERC Starting Grant 241184-PHerc finanziato dalla Commissione Europea (FP7, Ideas, www.pherc.eu)PHerc. 1020 (SVF 2. 131 = FDS 88) is one of the seven certain or probable Stoic papyri of the Herculaneum collection. Since the papyrus has no subscriptio, the author and the title of the work contained in it are unknown. Several elements seem to corroborate Hans von Arnim’s thesis that PHerc. 1020 hands down a work by either Chrysippus or one of his immediate successors. New arguments are advanced here in favour of this authorship beside those formerly adduced by von Arnim, Pohlenz and Keil. As far as the book’s content is concerned, we are not allowed to conclude that it was merely ethical, rather than purely logical or epistemological, nor that it only focused on the Stoic sage. On the contrary, from a detailed exegetical analysis and a comparison with the other evidence on Early Stoicism available to us it emerges that the work displayed a unique combination of ethics, logic and epistemology. By using new methods for the reading and editing of Herculaneum papyri, the editors have reconstructed for the first time the anatomy of the roll and the sequence of the fragments, while also establishing the Greek text on the basis of personal inspection of the original manuscript. This study is a new critical edition of the last eight columns of the papyrus (coll. 104-112 Alessandrelli-Ranocchia) – the best preserved columns and the only ones to have been studied by scholars so far – and constitutes the first part of the comprehensive edition of PHerc. 1020 included in the Project ERC Starting Grant 241184-PHerc funded by the European Commission (FP7, Ideas, www.pherc.eu)
About the de Almeida–Thouless line in neural networks
No full textIn this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington-Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line
Parallel Learning by Multitasking Neural Networks
No full textA modern challenge of Artificial Intelligence is learning multiple patterns
at once (i.e.parallel learning). While this can not be accomplished by standard
Hebbian associative neural networks, in this paper we show how the Multitasking
Hebbian Network (a variation on theme of the Hopfield model working on sparse
data-sets) is naturally able to perform this complex task. We focus on systems
processing in parallel a finite (up to logarithmic growth in the size of the
network) amount of patterns, mirroring the low-storage level of standard
associative neural networks at work with pattern recognition. For mild dilution
in the patterns, the network handles them hierarchically, distributing the
amplitudes of their signals as power-laws w.r.t. their information content
(hierarchical regime), while, for strong dilution, all the signals pertaining
to all the patterns are raised with the same strength (parallel regime).
Further, confined to the low-storage setting (i.e., far from the spin glass
limit), the presence of a teacher neither alters the multitasking performances
nor changes the thresholds for learning: the latter are the same whatever the
training protocol is supervised or unsupervised. Results obtained through
statistical mechanics, signal-to-noise technique and Monte Carlo simulations
are overall in perfect agreement and carry interesting insights on multiple
learning at once: for instance, whenever the cost-function of the model is
minimized in parallel on several patterns (in its description via Statistical
Mechanics), the same happens to the standard sum-squared error Loss function
(typically used in Machine Learning)
Replica symmetry breaking in supervised and unsupervised Hebbian networks
No full textHebbian neural networks with multi-node interactions, often called Dense Associative Memories, have recently attracted considerable interest in the statistical mechanics community, as they have been shown to outperform their pairwise counterparts in a number of features, including resilience against adversarial attacks, pattern retrieval with extremely weak signals and supra-linear storage capacities. However, their analysis has so far been carried out within a replica-symmetric theory. In this manuscript, we relax the assumption of replica symmetry and analyse these systems at one step of replica-symmetry breaking, focusing on two different prescriptions for the interactions that we will refer to as supervised and unsupervised learning. We derive the phase diagram of the model using two different approaches, namely Parisi’s hierarchical ansatz for the relationship between different replicas within the replica approach, and the so-called telescope ansatz within Guerra’s interpolation method: our results show that replica-symmetry breaking does not alter the threshold for learning and slightly increases the maximal storage capacity. Further, we also derive analytically the instability line of the replica-symmetric theory, using a generalization of the De Almeida and Thouless approach
Yet another exponential Hopfield model
No full texthttp://dx.doi.org/10.13039/501100004271 University of Rome La Sapienz
Replica symmetry breaking in dense Hebbian neural networks
No full textUnderstanding the glassy nature of neural networks is pivotal both for theoretical and computational
advances in Machine Learning and Theoretical Artificial Intelligence. Keeping the focus on dense associative Hebbian neural networks (i.e. Hopfield networks with polynomial interactions of even degree P > 2), the purpose of this paper is twofold: at first we develop rigorous mathematical approaches to address properly a statistical mechanical
picture of the phenomenon of replica symmetry breaking (RSB) in these networks, then -deepening results stemmed via these routes- we aim to inspect the glassiness that they hide.
In particular, regarding the methodology, we provide two techniques: the former (closer to mathematical physics in spirit) is an adaptation of the transport PDE to this case, while the latter (more probabilistic in its nature) is an extension of Guerra’s interpolation breakthrough.
Beyond coherence among the results, either in replica symmetric and in the one-step replica symmetry breaking level of description, we prove the Gardner’s picture (heuristically achieved through the replica trick) and we identify the maximal storage capacity by a groundstate analysis in the Baldi-Venkatesh high-storage regime. In the second part of the paper we investigate the glassy structure of these networks: at difference with the replica symmetric scenario (RS), RSB actually stabilizes the spin-glass phase.We report huge differences w.r.t. the standard pairwise Hopfield limit: in particular, it is known that it is possible to express the free energy of the Hopfield neural network (and, in a cascade fashion, all its properties) as a linear combination of the free energies of a hard spin glass (i.e. the Sherrington–Kirkpatrick
model) and a soft spin glass (the Gaussian or ”spherical”model). While this continues to hold also in the first step of RSB for the Hopfieldmodel, this is no longer truewhen interactions are more than pairwise (whatever the level of description, RS or RSB). For dense networks solely the free energy of the hard spin glass survives. As the Sherrington–Kirkpatrick spin glass is full-RSB (i.e. Parisi theory holds for that model), while the Gaussian spin-glass is replica symmetric, these different representation theorems prove a huge diversity in the underlying glassiness of associative neural networks
Generalized hetero-associative neural networks
No full textAuto-associative neural networks (e.g. the Hopfield model implementing the standard Hebbian prescription) serve as a foundational framework for pattern recognition and associative memory in statistical mechanics. However, their hetero-associative counterparts, though less explored, exhibit even richer computational capabilities. In this work, we examine a straightforward extension of Kosko's bidirectional associative memory, namely a three-directional associative memory, that is a tripartite neural network equipped with generalized Hebbian weights. Through both analytical approaches (using replica-symmetric statistical mechanics) and computational methods (via Monte Carlo simulations), we derive phase diagrams within the space of control parameters, revealing a region where the network can successfully perform pattern recognition as well as other tasks. In particular, it can achieve pattern disentanglement, namely, when presented with a mixture of patterns, the network can recover the original patterns. Furthermore, the system is capable of retrieving Markovian sequences of patterns and performing generalized frequency modulation
Dense Hebbian neural networks: a replica symmetric picture of supervised learning
No full textWe consider dense, associative neural-networks trained by a teacher (i.e.,
with supervision) and we investigate their computational capabilities
analytically, via statistical-mechanics of spin glasses, and numerically, via
Monte Carlo simulations. In particular, we obtain a phase diagram summarizing
their performance as a function of the control parameters such as quality and
quantity of the training dataset, network storage and noise, that is valid in
the limit of large network size and structureless datasets: these networks may
work in a ultra-storage regime (where they can handle a huge amount of
patterns, if compared with shallow neural networks) or in a ultra-detection
regime (where they can perform pattern recognition at prohibitive
signal-to-noise ratios, if compared with shallow neural networks). Guided by
the random theory as a reference framework, we also test numerically learning,
storing and retrieval capabilities shown by these networks on structured
datasets as MNist and Fashion MNist. As technical remarks, from the analytic
side, we implement large deviations and stability analysis within Guerra's
interpolation to tackle the not-Gaussian distributions involved in the
post-synaptic potentials while, from the computational counterpart, we insert
Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of
the synaptic tensors, overall obtaining a novel and broad approach to
investigate supervised learning in neural networks, beyond the shallow limit,
in general.Comment: arXiv admin note: text overlap with arXiv:2211.1406
Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning
No full textWe consider dense, associative neural-networks trained with no supervision
and we investigate their computational capabilities analytically, via a
statistical-mechanics approach, and numerically, via Monte Carlo simulations.
In particular, we obtain a phase diagram summarizing their performance as a
function of the control parameters such as the quality and quantity of the
training dataset and the network storage, valid in the limit of large network
size and structureless datasets. Moreover, we establish a bridge between
macroscopic observables standardly used in statistical mechanics and loss
functions typically used in the machine learning. As technical remarks, from
the analytic side, we implement large deviations and stability analysis within
Guerra's interpolation to tackle the not-Gaussian distributions involved in the
post-synaptic potentials while, from the computational counterpart, we insert
Plefka approximation in the Monte Carlo scheme, to speed up the evaluation of
the synaptic tensors, overall obtaining a novel and broad approach to
investigate neural networks in general
