1,720,972 research outputs found
Neural Mechanisms of Cortical Motion Computation Based on a Neuromorphic Sensory System.
No full textThe visual cortex analyzes motion information along hierarchically arranged visual areas that interact through bidirectional interconnections. This work suggests a bio-inspired visual model focusing on the interactions of the cortical areas in which a new mechanism of feedforward and feedback processing are introduced. The model uses a neuromorphic vision sensor (silicon retina) that simulates the spike-generation functionality of the biological retina. Our model takes into account two main model visual areas, namely V1 and MT, with different feature selectivities. The initial motion is estimated in model area V1 using spatiotemporal filters to locally detect the direction of motion. Here, we adapt the filtering scheme originally suggested by Adelson and Bergen to make it consistent with the spike representation of the DVS. The responses of area V1 are weighted and pooled by area MT cells which are selective to different velocities, i.e. direction and speed. Such feature selectivity is here derived from compositions of activities in the spatio-temporal domain and integrating over larger space-time regions (receptive fields). In order to account for the bidirectional coupling of cortical areas we match properties of the feature selectivity in both areas for feedback processing. For such linkage we integrate the responses over different speeds along a particular preferred direction. Normalization of activities is carried out over the spatial as well as the feature domains to balance the activities of individual neurons in model areas V1 and MT. Our model was tested using different stimuli that moved in different directions. The results reveal that the error margin between the estimated motion and synthetic ground truth is decreased in area MT comparing with the initial estimation of area V1. In addition, the modulated V1 cell activations shows an enhancement of the initial motion estimation that is steered by feedback signals from MT cells
Enhanced Human-Computer Interaction: A Unified Pipeline for Classification and Gesture Analysis
Full text linkThe purpose of this study is to develop a unified framework that combines object classification with vision-based gesture recognition. The proposed approach integrates YOLOv3 object detection enhanced by Z-Score Propensity Normalization to minimize false positives in Non-Maximum Suppression. Gesture recognition is performed using geometric contour detection and a Support Vector Machine classifier trained with Principal Component Analysis, which hierarchically refines detected bounding boxes and classifies hand gestures using spatial-temporal distance metrics. Experimental results show an average accuracy of 96.70%, a precision of 0.968, and an F1-score of 0.9671 for recognizing three gestures: hands down, one hand up, and hands up. This integrated method significantly improves computational efficiency and robustness, demonstrating strong potential for practical applications in augmented reality, assistive technologies, and immersive computing
Motion estimation for smooth-cross and plaid stimuli.
No full text<p>The stimuli are rotated in a counterclockwise direction. The first column of each stimulus contains the input image, accumulated events <i>e</i><sup><i>on</i></sup> and <i>e</i><sup><i>off</i></sup>, and the ground truth optical flow field. The first rows of the second, third and fourth columns represent the estimated motion in areas V1, MT and the modulated V1, respectively. The polar plot shows the direction selectivity of V1, MT and the modulated V1 of the bounded region (red square). The overall errors between the estimated motion and their respective ground truth are depicted in the histograms where the abscissa represents the binning in the range of the angular error Φ which are combined into one bar [<i>θ</i> − 7.5°, <i>θ</i> + 7.5°), and the ordinate represents the number of events.</p
speed representation in spatiotemporal (x,t) domain.
No full text<p>A rectangular object is moving from left to right at different speeds. In the first column, streams of ON/OFF events are generated in the spatiotemporal domain (x,t) for different speeds (slow, mid, fast). The sequences of oriented in the space-time domain corresponding to the speed of the motion. In the second column, sketch of the activated V1 cells in the 2D spatiotemporal domain driven by a slow, mid and fast sweep of input stimulus. The third column shows how cells in area MT encode the speed of the motion via integrate spots of V1 activations in spatiotemporal domain.</p
MT cell selectivity in the spatiotemporal (x, t) domain.
No full text<p>Slow, mid and fast motion selective cells for rightward motion are depicted in green, blue and red, respectively. The diagonal lines of green, blue and red dots represent idealized event responses for slow, mid and fast input motions (with initial inputs generated from the DVS sensor). In accordance to the representation of spatio-temporal inputs increases in speed coincides with an increase in angle relative to the time axis. Model MT cells are suggested to have larger receptive field size in space in comparison to the spatio-temporally selective cells in V1. They also integrate input responses from V1 cells over a temporal period. MT cells with different speed selectivities preferentially integrate V1 responses at the proper spatial offset positions (as depicted in the elliptic outlines). The same representation occurs for leftward motions.</p
Block diagram of V1-MT feedforward and feedback processing.
No full text<p>(A) DVS input. (A1) DVS sensor with half-circular rotational stimulus. (A2) Local changes in intensity (log I) elicit ON or OFF events, depending on the sign of the changes. (A3) <i>e</i><sup><i>on</i></sup> and <i>e</i><sup><i>off</i></sup> identify the event activity (+1) ON and (-1) OFF, respectively. (A4) Event stream which is represented as a sequence of events e at a position <b>p</b> and time t. (A5) illustrates the generated events via DVS sensor in 3-dimensional space (x,y,t). (B) The model of area V1 (B1) Spatiotemporal filter construction. (b1) Spatial filters. (b2) Temporal filters. (b3) The first row represents the products of two spatial and two temporal filters; the second row represents the sum and difference of the product filters. (B2) modulation of area V1 based on the feedback of area MT. (B3) normalization mechanism of area V1. (C) The model of area MT. (C1) filtering representation of MT cells. (C2) modulation signal for area MT based on area MST activation which, here, is set to zero. (C3) MT normalizing mechanism.</p
Motion estimation of 2D structure of corner terminators, temp⋅2.
No full text<p>Two superimposed gratings are moved upward direction through circular aperture. The image input is shown in the first column of the first row. V1 cell and one of the MT cells (V1:MT 1:3) are depicted over the accumulated events as blue circle and green circle, respectively. The upper row shows the estimated motion in areas V1, MT and the modulated V1. The real motion is estimated at the 2D features (corner regions) while the normal flow is estimated along bars contours. The small polar plots show the direction selectivity of selective cells that are located on the bars contours and corner while the whole direction selectivity of the stimulus are depicted in the large polar plot. The histogram shows the angular error between the estimated motion and upward motion ground truth, where the abscissa represents the binning in the range of the angular error Φ which are combined into one bar [<i>θ</i> − 7.5°, <i>θ</i> + 7.5°), and the ordinate represents the number of events.</p
Motion estimation of tiger and ball stimuli.
No full text<p>The stimuli are moved in rightward direction. The first column of each stimulus contains the input image, accumulated events <i>e</i><sup><i>on</i></sup> and <i>e</i><sup><i>off</i></sup>, and a sketch of the ground truth optical flow field. The first rows of the second, third and fourth columns represent the estimated motion in areas V1, MT and the modulated V1, respectively. The direction selectivity for these areas are depicted in the polar plot where blue, green and red lines represent the responses of V1, MT and V1 modulated by MT, respectively. The histogram shows the angular error between the estimated motion and the ground truth of rightward motion direction. The abscissa of the histogram represents the binning in the range of the angular error Φ which are combined into one bar [<i>θ</i> − 7.5°, <i>θ</i> + 7.5°), and the ordinate represents the number of events. The ball image is adopted from johncarlosbaez.wordpress.com.</p
- …
