247 research outputs found
Active fixation as an efficient coding strategy for neuromorphic vision
No full textAbstract Contrary to a photographer, who puts a great effort in keeping the lens still, eyes insistently move even during fixation. This benefits signal decorrelation, which underlies an efficient encoding of visual information. Yet, camera motion is not sufficient alone; it must be coupled with a sensor specifically selective to temporal changes. Indeed, motion induced on standard imagers only results in burring effects. Neuromorphic sensors represent a valuable solution. Here we characterize the response of an event-based camera equipped with fixational eye movements (FEMs) on both synthetic and natural images. Our analyses prove that the system starts an early stage of redundancy suppression, as a precursor of subsequent whitening processes on the amplitude spectrum. This does not come at the price of corrupting structural information contained in local spatial phase across oriented axes. Isotropy of FEMs ensures proper representations of image features without introducing biases towards specific contrast orientations
Recurrent models of orientation selectivity enable robust early-vision processing in mixed-signal neuromorphic hardware
No full textISSN:2041-172
A Bio-Inspired Neuromorphic Active Vision System Based on Fixational Eye Movements
No full textSimilar to biological retinas, neuromorphic Dynamic Vision Sensor (DVS) devices only respond to changes in the visual scene. It has been observed that in biological systems there is a causal relationship between fixational eye movements and target visibility during fixation, which plays a central role in vision. Based on these findings we implemented an active vision system comprising of a DVS mounted on a pan-tilt unit to introduce microscopic and erratic camera movements as a pivot for artificial vision of static scenes. The key principle is that moving the sensor over an image shifts the low temporal frequency power of a static scene into a range that an event-based retina can properly signal and encode it as highly synchronous activity. By characterizing the signal provided by the active vision system we evidenced (1) an amplification of its response to high spatial frequencies; (2) a whitening effect when scaling stimulus contrast to match the structure of natural images; and (3) an equalized response to all possible orientations of static stimuli related to the isotropic statistics of the random-like motion. The design of a further proper anisotropic spatial summation of events with opponent contrast polarity in a biologically-realistic spiking neural network allowed the detection of information relative to the local orientation of stimuli in a fully bio-inspired fashion. We validate the system proposed with experimental results using synthetic control stimuli
Emergence of Gabor-like Receptive Fields in a Recurrent Network of Mixed-Signal Silicon Neurons
No full textMixed signal analog/digital neuromorphic circuits offer an ideal computational substrate for testing and validating hypotheses about models of sensory processing, as they are affected by low resolution, variability, and other limitations that affect in a similar way real neural circuits. In addition, their real-time response properties allow to test these models in closed-loop sensory-processing hardware setups and to get an immediate feedback on the effect of different parameter settings. Within this context we developed a recurrent neural network architecture based on a model of the retinocortical visual
pathway to obtain neurons highly tuned to oriented visual stimuli along a specific direction and with a specific spatial frequency, with Gabor-like receptive fields. The computation performed by the retina is emulated by a Dynamic Vision Sensor (DVS) while the following feed-forward and recurrent processing stages are implemented by a Dynamic Neuromorphic Asynchronous Processor (DYNAP) chip that comprises adaptive integrate-and fire neurons and dynamic synapses. We show how the network implemented on this device gives rise to neurons tuned to specific orientations and spatial frequencies, independent of the temporal frequency of the visual stimulus. Compared to alternative feed-forward schemes, the model proposed produces highly structured receptive fields with a limited number of synaptic connections, thus optimizing hardware resources. We validate the model and approach proposed with experimental results using both synthetic and natural images
Learning bio-inspired head-centric representations of 3D shapes in an active fixation setting
No full textWhen exploring the surrounding environment with the eyes, humans and primates need to interpret three-dimensional (3D) shapes in a fast and invariant way, exploiting a highly variant and gaze-dependent visual information. Since they have front-facing eyes, binocular disparity is a prominent cue for depth perception. Specifically, it serves as computational substrate for two ground mechanisms of binocular active vision: stereopsis and binocular coordination. To this aim, disparity information, which is expressed in a retinotopic reference frame, is combined along the visual cortical pathways with gaze information and transformed in a head-centric reference frame. Despite the importance of this mechanism, the underlying neural substrates still remain widely unknown. In this work, we investigate the capabilities of the human visual system to interpret the 3D scene exploiting disparity and gaze information. In a psychophysical experiment, human subjects were asked to judge the depth orientation of a planar surface either while fixating a target point or while freely exploring the surface. Moreover, we used the same stimuli to train a recurrent neural network to exploit the responses of a modelled population of cortical (V1) cells to interpret the 3D scene layout. The results for both human performance and from the model network show that integrating disparity information across gaze directions is crucial for a reliable and invariant interpretation of the 3D geometry of the scene
The Active Side of Stereopsis: Fixation Strategy and Adaptation to Natural Environments
No full textAbstractDepth perception in near viewing strongly relies on the interpretation of binocular retinal disparity to obtain stereopsis. Statistical regularities of retinal disparities have been claimed to greatly impact on the neural mechanisms that underlie binocular vision, both to facilitate perceptual decisions and to reduce computational load. In this paper, we designed a novel and unconventional approach in order to assess the role of fixation strategy in conditioning the statistics of retinal disparity. We integrated accurate realistic three-dimensional models of natural scenes with binocular eye movement recording, to obtain accurate ground-truth statistics of retinal disparity experienced by a subject in near viewing. Our results evidence how the organization of human binocular visual system is finely adapted to the disparity statistics characterizing actual fixations, thus revealing a novel role of the active fixation strategy over the binocular visual functionality. This suggests an ecological explanation for the intrinsic preference of stereopsis for a close central object surrounded by a far background, as an early binocular aspect of the figure-ground segregation process.</jats:p
Assessment of stereoscopic depth perception in augmented reality environments based on low-cost technologies
No full textIn the last decade, there has been a rapidly growing development of low-cost and commercial interactive systems, both for professional (e.g. scientific visualization, surgery, rehabilitation), and consumer applications (e.g. 3D cinema and videogames). Moreover, there is a recent interest towards the development of simple and affordable systems for motor and cognitive rehabilitation applications to allow patients to perform psychomotor rehabilitation exercises without having to leave their homes [Attygalle et al. 2008].</p
Centric-minded templates for self-motion perception
No full textAbstractWe propose a two-layer neuromorphic architecture by which motion field pattern, generated during locomotion, are processed by template detectors specialized for gaze-directed self-motion (expansion and rotation). The templates provide a gaze-centered computation for analyzing motion field in terms of how it is related to the fixation point (i.e., the fovea). The analysis is performed by relating the vectorial components of the act of motion to variations (i.e., asymmetries) of the local structure of the motion field. Notwithstanding their limited extension in space, such centric-minded templates extract, as a whole, global information from the input flow field, being sensitive to different local instances of the same global property of the vector field with respect to the fixation point; a quantitative analysis, in terms of vectorial operators, evidences this property as tuning curves for heading direction. Model performances, evaluated in several situations characterized by conditions of absence and presence of pursuit eye movements, validate the approach. We observe that the gaze-centered model provides an explicit testable hypothesis that can guide further explorations of visual motion processing in extrastriate cortical areas
Vergence control learning through real V1 disparity tuning curves
No full textA neural network architecture able to autonomously learn effective disparity-vergence responses and drive the vergence eye movements of a simulated binocular active vision system is proposed. The proposed approach, instead of exploiting purposely designed resources, relies on the direct use of a set of real disparity tuning curves, measured in the primary visual cortex of two macaque monkeys and courteously made available by (Prince et al., 2002), that provides a distributed representation of binocular disparity. The network evolves following a differential Hebbian rule that exploits the overall population activity to measure the state of the system, i.e. The deviation from the desired vergence position, so as its modification as a consequence of the action performed. Accordingly, the signal provides an effective intrinsic reward to develop a stable and accurate vergence behaviour. Emerging from a direct interaction of the sensing system with the environment, the resulting control provides a precise and accurate control for small disparities, as well as a raw control on a broader working range when large disparities are experienced. The developed control converges to a stable state that intrinsically and continuously adapts to the characteristics of the ongoing stimulation. The results proved how actually naturally distributed resources allows for robust and flexible learning capabilities in changeable situations
An architectural hypothesis for direction selectivity in the visual cortex: the role of spatially asymmetric intracortical inhibition
No full textWithin a linear field approach, an architectural model for simple cell direction selectivity in the visual cortex is proposed. The origin of direction selectivity is related to recurrent intracortical interactions with a spatially asymmetric character along the axis of stimulus motion. No explicit asymmetric temporal mechanisms are introduced or adopted. The analytical investigation of network behavior, carried out under the assumption of a linear superposition of geniculate and intracortical contributions, shows that motion sensitivity of the resulting receptive fields emerges as a dynamic property of the cortical network without any feed-forward direction selectivity bias. A detailed analysis of the effects of the architectural characteristics of the cortical network on directionality and velocity-response curves was conducted by systematically varying the model's parameters
- …
