55 research outputs found
Generalization Learning in a Perceptron with Binary Synapses
No full textWe consider the generalization problem for a perceptron with binary synapses, implementing the Stochastic Belief-Propagation-Inspired (SBPI) learning algorithm which we proposed earlier, and perform a mean-field calculation to obtain a differential equation which describes the behaviour of the device in the limit of a large number of synapses N. We show that the solving time of SBPI is of order N√(logN) , while the similar, well-known clipped perceptron (CP) algorithm does not converge to a solution at all in the time frame we considered. The analysis gives some insight into the ongoing process and shows that, in this context, the SBPI algorithm is equivalent to a new, simpler algorithm, which only differs from the CP algorithm by the addition of a stochastic, unsupervised meta-plastic reinforcement process, whose rate of application must be less than 2/√(πN) for the learning to be achieved effectively. The analytical results are confirmed by simulations
Systematically and efficiently improving -means initialization by pairwise-nearest-neighbor smoothing
No full textWe present a meta-method for initializing (seeding) the -means clustering
algorithm called PNN-smoothing. It consists in splitting a given dataset into
random subsets, clustering each of them individually, and merging the
resulting clusterings with the pairwise-nearest-neighbor (PNN) method. It is a
meta-method in the sense that when clustering the individual subsets any
seeding algorithm can be used. If the computational complexity of that seeding
algorithm is linear in the size of the data and the number of clusters ,
PNN-smoothing is also almost linear with an appropriate choice of , and
quite competitive in practice. We show empirically, using several existing
seeding methods and testing on several synthetic and real datasets, that this
procedure results in systematically better costs. In particular, our method of
enhancing -means++ seeding proves superior in both effectiveness and speed
compared to the popular "greedy" -means++ variant. Our implementation is
publicly available at https://github.com/carlobaldassi/KMeansPNNSmoothing.jl.Comment: https://openreview.net/forum?id=FTtFAg3pek 16 pages (+8 appendix), 2
figures, 4 tables (+14 appendix). Transactions on Machine Learning Research,
Dec 202
Non-linear integration of crowded orientation signals
No full textAbstractCrowding of oriented signals has been explained as linear, compulsory averaging of the signals from target and flankers [Parkes, L., Lund, J., Angelucci, A., Solomon, J. A., & Morgan, M. (2001). Compulsory averaging of crowded orientation signals in human vision. Nature Neuroscience, 4(7), 739–744]. On the other hand, a comparable search task with sparse stimuli is well modeled by a ‘Signed–Max’ rule that integrates non-linearly local tilt estimates [Baldassi, S., & Verghese, P. (2002). Comparing integration rules in visual search. Journal of Vision, 2(8), 559–570], as reflected by the bimodality of the distributions of reported tilts in a magnitude matching task [Baldassi, S., Megna, N., & Burr, D. C. (2006). Visual clutter causes high-magnitude errors. PLoS Biology, 4(3), e56]. This study compares the two models in the context of crowding by using a magnitude matching task, to measure distributions of perceived target angles and a localization task, to probe the degree of access to local information. Response distributions were bimodal, implying uncertainty, only in the presence of abutting flankers. Localization of the target is relatively preserved but it quantitatively falls in between the predictions of the two models, possibly suggesting local averaging followed by a max operation. This challenges the notion of global averaging and suggests some conscious access to local orientation estimates
Input-driven unsupervised learning in recurrent neural networks
No full textUnderstanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is an attractor neural network with Hebbian learning (e.g. the Hopfield model). The model simplicity and the locality of the synaptic update rules come at the cost of a limited storage capacity, compared with the capacity achieved with supervised learning algorithms, whose biological plausibility is questionable. Here, we present an on-line learning rule for a recurrent neural network that achieves near-optimal performance without an explicit supervisory error signal and using only locally accessible information, and which is therefore biologically plausible. The fully connected network consists of excitatory units with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the patterns to be memorized are presented on-line as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs ('local fields'). Synapses corresponding to active inputs are modified as a function of the position of the local field with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. An additional parameter of the model allows to trade storage capacity for robustness, i.e. increased size of the basins of attraction. We simulated a network of 1001 excitatory neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction: our results show that, for any given basin size, our network more than doubles the storage capacity, compared with a standard Hopfield network. Our learning rule is consistent with available experimental data documenting how plasticity depends on firing rate. It predicts that at high enough firing rates, no potentiation should occu
Reward sharpens orientation coding independently on attention
No full textRewarding improves performance. Is it due to modulations of the output modules of the neural systems or are there mechanisms favoring more 'generous' inputs? Some recent study included V1 in the the circuitry of reward-based modulations, but the effects of reward can easily be confused with effects of attention. Here we address this issue with a psychophysical dual task to control attention while orientation sensitivity on targets associated to different levels of reward is measured. We found that different reward rates improve orientation discrimination and sharpen the internal response distributions. Data are unaffected by changing attentional load nor by dissociating the feature of the reward cue from the feature relevant for the task. This suggests that reward may act independently on attention by modulating the activity of early sensory stages, perhaps V1, through a SNR improvement of task-relevant channels. Reward acts like attention, but using separate channels
Search superiority in autism within, but not outside the crowding regime
No full textAutosomal recessive spastic paraplegia with thinning of corpus callosum
(ARHSP-TCC) is a complex form of HSP initially described in Japan but
subsequently reported to have a worldwide distribution with a particular high
frequency in multiple families from the Mediterranean basin. We recently showed
that ARHSP-TCC is commonly associated with mutations in SPG11/KIAA1840 on
chromosome 15q. We have now screened a collection of new patients mainly
originating from Italy and Brazil, in order to further ascertain the spectrum of
mutations in SPG11, enlarge the ethnic origin of SPG11 patients, determine the
relative frequency at the level of single Countries (i.e., Italy), and establish
whether there is one or more common mutation. In 25 index cases we identified 32
mutations; 22 are novel, including 9 nonsense, 3 small deletions, 4 insertions, 1
in/del, 1 small duplication, 1 missense, 2 splice-site, and for the first time a
large genomic rearrangement. This brings the total number of SPG11 mutated
patients in the SPATAX collection to 111 cases in 44 families and in 17 isolated
cases, from 16 Countries, all assessed using homogeneous clinical criteria. While
expanding the spectrum of mutations in SPG11, this larger series also
corroborated the notion that even within apparently homogeneous population a
molecular diagnosis cannot be achieved without full gene sequencing
Effectiveness and safety of infliximab dose escalation in patients with refractory Takayasu arteritis: A real-life experience from a monocentric cohort
No full textObjectives: To evaluate effectiveness and safety of infliximab dose escalation in Takayasu arteritis (TAK) patients. To identify factors associated with refractoriness to standard-dose infliximab.Methods: Medical records of infliximab-treated TAK patients from a large single-centre observational cohort were reviewed. Infliximab therapy duration, concomitant therapies, and reasons for dose escalation and therapy suspension were evaluated. Occurrence of adverse events was recorded. A comparison between patients who maintained infliximab standard-dose and those who needed dose-escalation was performed. Factors associated with refractoriness to standard dose were analysed.Results: Forty-one patients were included. Starting infliximab dose was 5 mg/kg 6-weekly and 28 patients (68%) needed dose escalation. Persistence/recurrence of clinical symptoms was the most frequent reason for escalation. Median therapy duration was 39 (IQR, 26-61) months in the standard-dose group and 68 (38-87) months in the intensified-dose group. In the intensified-dose-group, infliximab was suspended in eight patients (29%) after a median of 38 (31-71) months, due to loss of response (n= 7) or patient's request (n= 1). Patients in the intensifieddose group had a higher number of relapses (3.4 vs 0.8 events/patient) and received a higher cumulative steroid dose (1.7 [1.6-2.3] vs 1.3 [1-1.6] g/month of prednisone). Three patients from the intensified-dose group had serious infections; one patient from the standard-dose group developed paradoxical psoriasis. At univariate analysis, age at diagnosis and age at infliximab start were associated with infliximab escalation.Conclusion: In TAK, dose escalation is safe and allows to optimise infliximab durability in refractory patients. Younger patients seem to be more refractory to standard dosages
Comparative judgements of facial emotions are affected by semantic congruity not by SNARC
No full textBehavioural evidence based on speeded classification of centrally presented emotions suggests that the mental representation of affects is similar to number, with faster left-sided responses to negative emotions (anger) and faster right-sided responses for positive emotions (happiness), SNARC-Like Effect (SLE). However, it is not clear whether a similar effect would hold true using a One Interval Speed-Comparison Task (OIS-CT) between pairs of simultaneously displayed facial expressions (horizontally aligned), either fully-emotional (happy/angry) or half-emotional (neutral/happy-or-angry). In this case a Semantic Congruity Effect (SCE) might rise. Emotion comparison indeed requires judging greater or lesser, depending on whether the task involves detecting the happiest or the angriest face. The speed of comparative judgements should thus be faster when the target image is emotional rather neutral irrespective from the response side relative to average valence, and/or the spatial congruency of image-pairs with the left-to-right mental format. This would produce standard vs reversed SLE for spatially congruent (angriest-left/happiest-right) vs incongruent displays. In particular, for incongruent displays, a positive right-to-left speed advantage for half-emotional displays with negative rather than positive average valence should be observed: vice-versa for congruent displays. We found a strong SCE, not a SLE, in two Experiments involving the same OIS-CT with (self-terminating stimulus, n=40) vs without foveation (stimulus presentation time=[190, 200] ms, n=40). Individual average speeds were fully accounted for by a model formalizing SCE: a nlme-regression including the sum between global display and absolute target valence as the only covariate of speeds
Efficient supervised learning in networks with binary synapses
No full textRecent experimental studies indicate that synaptic changes induced by neuronal activity are discrete jumps between a small number of stable states. Learning in systems with discrete synapses is known to be a computationally hard problem. Here, we study a neurobiologically plausible on-line learning algorithm that derives from belief propagation algorithms. We show that it performs remarkably well in a model neuron with binary synapses, and a finite number of "hidden" states per synapse, that has to learn a random classification task. Such a system is able to learn a number of associations close to the theoretical limit in time that is sublinear in system size. This is to our knowledge the first on-line algorithm that is able to achieve efficiently a finite number of patterns learned per binary synapse. Furthermore, we show that performance is optimal for a finite number of hidden states that becomes very small for sparse coding. The algorithm is similar to the standard "perceptron" learning algorithm, with an additional rule for synaptic transitions that occur only if a currently presented pattern is "barely correct." In this case, the synaptic changes are metaplastic only (change in hidden states and not in actual synaptic state), stabilizing the synapse in its current state. Finally, we show that a system with two visible states and K hidden states is much more robust to noise than a system with K visible states. We suggest that this rule is sufficiently simple to be easily implemented by neurobiological systems or in hardware
- …
