85 research outputs found
Properties of the geometry of solutions and capacity of multilayer neural networks with rectified linear unit activations
No full textRectified linear units (ReLUs) have become the main model for the neural units in current deep learning systems. This choice was originally suggested as a way to compensate for the so-called vanishing gradient problem which can undercut stochastic gradient descent learning in networks composed of multiple layers. Here we provide analytical results on the effects of ReLUs on the capacity and on the geometrical landscape of the solution space in two-layer neural networks with either binary or real-valued weights. We study the problem of storing an extensive number of random patterns and find that, quite unexpectedly, the capacity of the network remains finite as the number of neurons in the hidden layer increases, at odds with the case of threshold units in which the capacity diverges. Possibly more important, a large deviation approach allows us to find that the geometrical landscape of the solution space has a peculiar structure: While the majority of solutions are close in distance but still isolated, there exist rare regions of solutions which are much more dense than the similar ones in the case of threshold units. These solutions are robust to perturbations of the weights and can tolerate large perturbations of the inputs. The analytical results are corroborated by numerical findings
Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures
No full textWe analyze the connection between minimizers with good generalizing properties and high local entropy regions of a threshold-linear classifier in Gaussian mixtures with the mean squared error loss function. We show that there exist configurations that achieve the Bayes-optimal generalization error, even in the case of unbalanced clusters. We explore analytically the error-counting loss landscape in the vicinity of a Bayes-optimal solution, and show that the closer we get to such configurations, the higher the local entropy, implying that the Bayes-optimal solution lays inside a wide flat region. We also consider the algorithmically relevant case of targeting wide flat minima of the (differentiable) mean squared error loss. Our analytical and numerical results show not only that in the balanced case the dependence on the norm of the weights is mild, but also, in the unbalanced case, that the performances can be improved
Non-linear integration of crowded orientation signals
No full textAbstractCrowding of oriented signals has been explained as linear, compulsory averaging of the signals from target and flankers [Parkes, L., Lund, J., Angelucci, A., Solomon, J. A., & Morgan, M. (2001). Compulsory averaging of crowded orientation signals in human vision. Nature Neuroscience, 4(7), 739–744]. On the other hand, a comparable search task with sparse stimuli is well modeled by a ‘Signed–Max’ rule that integrates non-linearly local tilt estimates [Baldassi, S., & Verghese, P. (2002). Comparing integration rules in visual search. Journal of Vision, 2(8), 559–570], as reflected by the bimodality of the distributions of reported tilts in a magnitude matching task [Baldassi, S., Megna, N., & Burr, D. C. (2006). Visual clutter causes high-magnitude errors. PLoS Biology, 4(3), e56]. This study compares the two models in the context of crowding by using a magnitude matching task, to measure distributions of perceived target angles and a localization task, to probe the degree of access to local information. Response distributions were bimodal, implying uncertainty, only in the presence of abutting flankers. Localization of the target is relatively preserved but it quantitatively falls in between the predictions of the two models, possibly suggesting local averaging followed by a max operation. This challenges the notion of global averaging and suggests some conscious access to local orientation estimates
Methods to Reproduce In-Plane Deformability of Orthotropic Floors in the Finite Element Models of Buildings
No full textIn the modelling of reinforced concrete (RC) buildings, the rigid diaphragm hypothesis to represent the in-plane behavior of floors was and still is very commonly adopted because of its simplicity and computational cheapness. However, since excessive floor in-plane deformability can cause a very different redistribution of lateral forces on vertical resisting elements, it may be necessary to consider floor deformability. This paper investigates the classical yet intriguing question of modeling orthotropic RC floor systems endowed with lightening elements by means of a uniform orthotropic slab in order to describe accurately the building response under seismic loads. The simplified method, commonly adopted by engineers and based on the equivalence between the transverse stiffness of the RC elements of the real floor and those of the orthotropic slab, is presented. A case study in which this simplified method is used is also provided. Then, an advanced finite element (FE)-based method to determine the elastic properties of the equivalent homogenized orthotropic slab is proposed. The novel aspect of this method is that it takes into account the interaction of shell elements with frame elements in the 3D FE model of the building. Based on the results obtained from the application of this method to a case study, a discussion on the adequacy of the simplified method is also provided
Typical and atypical solutions in nonconvex neural networks with discrete and continuous weights
No full textWe study the binary and continuous negative-margin perceptrons as simple nonconvex neural network models learning random rules and associations. We analyze the geometry of the landscape of solutions in both models and find important similarities and differences. Both models exhibit subdominant minimizers which are extremely flat and wide. These minimizers coexist with a background of dominant solutions which are composed by an exponential number of algorithmically inaccessible small clusters for the binary case (the frozen 1-RSB phase) or a hierarchical structure of clusters of different sizes for the spherical case (the full RSB phase). In both cases, when a certain threshold in constraint density is crossed, the local entropy of the wide flat minima becomes nonmonotonic, indicating a breakup of the space of robust solutions into disconnected components. This has a strong impact on the behavior of algorithms in binary models, which cannot access the remaining isolated clusters. For the spherical case the behavior is different, since even beyond the disappearance of the wide flat minima the remaining solutions are shown to always be surrounded by a large number of other solutions at any distance, up to capacity. Indeed, we exhibit numerical evidence that algorithms seem to find solutions up to the SAT/UNSAT transition, that we compute here using an 1RSB approximation. For both models, the generalization performance as a learning device is shown to be greatly improved by the existence of wide flat minimizers even when trained in the highly underconstrained regime of very negative margins
Methods to Reproduce In-Plane Deformability of Orthotropic Floors in the Finite Element Models of Buildings
No full textIn the modelling of reinforced concrete (RC) buildings, the rigid diaphragm hypothesis to represent the in-plane behavior of floors was and still is very commonly adopted because of its simplicity and computational cheapness. However, since excessive floor in-plane deformability can cause a very different redistribution of lateral forces on vertical resisting elements, it may be necessary to consider floor deformability. This paper investigates the classical yet intriguing question of modeling orthotropic RC floor systems endowed with lightening elements by means of a uniform orthotropic slab in order to describe accurately the building response under seismic loads. The simplified method, commonly adopted by engineers and based on the equivalence between the transverse stiffness of the RC elements of the real floor and those of the orthotropic slab, is presented. A case study in which this simplified method is used is also provided. Then, an advanced finite element (FE)-based method to determine the elastic properties of the equivalent homogenized orthotropic slab is proposed. The novel aspect of this method is that it takes into account the interaction of shell elements with frame elements in the 3D FE model of the building. Based on the results obtained from the application of this method to a case study, a discussion on the adequacy of the simplified method is also provided
Spatiotemporal mechanisms of perisaccadic vision revealed by psychophysical reverse correlation
No full textProduction and metallurgic results obtained by the new automatic forging system with oleodinamic press 25 MN
No full textInfectious diseases seeker (Ids): An innovative tool for prompt identification of infectious diseases during outbreaks
No full textBackground: Several technologies for rapid molecular identification of pathogens are currently available; jointly with monitoring tools (i.e., web-based surveillance tools, infectious diseases modelers, and epidemic intelligence methods), they represent important components for timely outbreak detection and identification of the involved pathogen. The application of these approaches is usually feasible and effective when performed by healthcare professionals with specific expertise and skills and when data and resources are easily accessible. Contrariwise, in the field situation where healthcare workers or first responders from heterogeneous competences can be asked to investigate an outbreak of unknown origin, a simple and suitable tool for rapid agent identification and appropriate outbreak management is highly needed. Most especially when time is limited, available data are incomplete, and accessible infrastructure and resources are inadequate. The use of a prompt, user-friendly, and accessible tool able to rapidly recognize an infectious disease outbreak and with high sensitivity and precision may be a game-changer to support emergency response and public health investigations. Methods: This paper presents the work performed to implement and test an innovative tool for prompt identification of infectious diseases during outbreaks, called Infectious Diseases Seeker (IDS). IDS is a standalone software that runs on the most common operative systems. It has been built by integrating a database containing an interim set of 60 different disease causative agents and COVID-19 data and is able to work in an off-line mode without requiring a network connection. Results: IDS has been applied in a real and complex scenario in terms of concomitant infectious diseases (yellow fever, COVID-19, and Lassa fever), as can be in the second part of 2020 in Nigeria. The outcomes have allowed inferring that yellow fever (YF), and not Lassa fever, was affecting the area under investigation. Conclusions: Our result suggests that a tool like IDS could be valuable for the quick and easy identification and discrimination of infectious disease outbreaks even when concurrent outbreaks occur, like for the case study of YF and COVID-19 pandemic in Nigeria
Effect of delayed artificial insemination on conception rate using a commercially available semen sexing agent in bovine
No full text- …
