1,187 research outputs found
Emergent self-awareness in multi-sensor physical agents
No full textThe cognitive approach to the development of autonomous vehicles takes inspiration from human reasoning, and, conversely to the computationalist approach, rejects formulating fixed mathematical models to describe each possible behavior of vehicles and objects around them. The computationalist approach indeed has a weakness: developers cannot examine and mathematically formulate all possible real-world situations that drivers may encounter. Cognitive approaches provide a solution to this problem, as they suggest that vehicles should continually learn through experience as humans do, which would allow them to progressively grasp rare and unexpected behaviors. This thesis mainly addresses the pivotal question of detecting when some unexpected behavior is occurring - referred to as anomaly detection - within a cognitive self-awareness framework. The adopted framework is characterized by several desirable characteristics, such as being Bayesian, hierarchical, multi-sensorial, data-driven, and interpretable.
A set of modules for anomaly detection and localization of an agent are proposed. Dynamic Bayesian Networks are used, as they are interpretable probabilistic models allowing hierarchical representation of variables potentially coming from multiple sensors; moreover, the links between variables inside Dynamic Bayesian Networks can be learned from data. All methods in the thesis adopt a filter called the Markov Jump Particle Filter that can be described through a Dynamic Bayesian Network on a minimum of three levels. When elaborating image data, Variational Autoencoders are adopted to perform dimensionality reduction while maintaining a probabilistic representation; novel methods for joining Variational Autoencoders and Bayesian filters are proposed.
This thesis focuses on the elaboration of vehicular video and odometry data. First, self-awareness anomaly detection approaches to separately handle video and odometry data are proposed; then, an approach fusing the two modalities is introduced; the capability to localize the vehicle is also added.
The proposed methods are evaluated on real-world and simulated data from terrestrial and aerial vehicles.El enfoque cognitivo para el desarrollo de vehículos autónomos se inspira en el razonamiento humano y, a la inversa del enfoque computacional, rechaza la formulación de modelos matemáticos fijos para describir cada posible comportamiento de los vehículos y objetos alrededor de ellos. En efecto, el enfoque computacional tiene una debilidad: los desarrolladores no pueden examinar y formular matemáticamente todas las posibles situaciones del mundo real que los conductores podrían encontrar. Los enfoques cognitivos proporcionan una solución a este problema, ya que sugieren que los vehículos deberían aprender continuamente a través de la experiencia como hacen los humanos, lo que les permitiría reconocer progresivamente comportamientos raros e inesperados. Esta tesis trata principalmente la question fundamental de detectar cuándo está ocurriendo algún comportamiento inesperado - definida como detección de anomalías - dentro de un marco de autoconciencia cognitiva. El marco adoptado tiene características deseables, como ser bayesiano, jerárquico, multisensorial, basado en datos e interpretable.
Se propone un conjunto de módulos para la detección de anomalías y localización de un agente. Se utilizan Redes Bayesianas Dinámicas, ya que son modelos probabilísticos interpretables que permiten una representación jerárquica de variables potencialmente provenientes de sensores múltiples; además, los vínculos entre variables dentro de las Redes Bayesianas Dinámicas se pueden aprender de los datos.
Todos los métodos en la tesis adoptan un filtro llamado Filtro de Partículas de Salto de Markov, que puede describirse a través de una Red Bayesiana Dinámica con un mínimo de tres niveles. Los datos de imágenes se procesan usando un Autocodificadores Variacionales para realizar una reducción de dimensionalidad manteniendo una representación probabilística; se proponen métodos novedosos para unir Autocodificadores Variacionales y filtros Bayesianos.
Esta tesis se enfoca en la elaboración de datos de vehículos de video y odometría. En primer lugar, se proponen enfoques de detección de anomalías para elaborar por separado datos de vídeo y odometría; luego se introduce un enfoque que fusiona las dos modalidades; también se añade la capacidad de localizar el vehículo.
Los métodos propuestos se evalúan con datos del mundo real y simulados, tanto de vehículos terrestres como aéreos
Interpretable anomaly detection using a Generalized Markov Jump Particle Filter
No full textWhen performing anomaly detection on sensory data of an autonomous vehicle, it is fundamental to infer the cause of the found
anomalies. This paper proposes a method for learning prediction models and detecting anomalies by decomposing the evolution of
the state of an agent into its different motion-related parameters. A filter is introduced, based on the concept of Generalized Filtering,
with the objective of increasing the interpretability of the results with respect to previous methods. The proposed anomaly detection
method is tested on data from a real vehicle. We also consider the case in which multiple models are learned, how to extract the salient
discriminatory features of each, and use the proposed anomaly detection method to perform behavior classification
Learning of linear video prediction models in a multi-modal framework for anomaly detection
No full textThis paper proposes a method for performing future-frame prediction and anomaly detection on video data in a multi-modal framework
based on Dynamic Bayesian Networks (DBNs). In particular, odometry data and video data from a moving vehicle are fused. A
Markov Jump Particle Filter (MJPF) is learned on odometry data, and its features are used to aid the learning of a Kalman Variational
Autoencoder (KVAE) on video data. Consequently, anomaly detection can be performed on video data using the learned model. We
evaluate the proposed method using multi-modal data from a vehicle performing different tasks in a closed environment
Probabilistic anomaly detection methods using learned models from time-series data for multimedia self-aware systems
No full textAnomaly detection techniques constitute a fundamental resource in many applications such as medical image analysis, fraud detection or video surveillance. These techniques represent an essential step also for artificial self-aware systems that can continually learn from new situations. In this chapter, we present a semisupervised method for the detection of anomalies for this type of self-aware agents. The described method leverages the message-passing capability of Generalized Dynamic Bayesian Networks (GDBNs) to provide anomalies at different abstraction levels for diverse types of time-series data (i.e., both low-dimensional and high-dimensional). The detected anomalies could consequently be employed to enable the system to evolve by integrating the new acquired knowledge. To present a case study for the description of the anomaly detection method, we propose to use multisensory data from a semiautonomous vehicle performing different tasks in a closed environment
Continual Learning Of Predictive Models In Video Sequences Via Variational Autoencoders
No full textThis paper proposes a method for performing continual learning of
predictive models that facilitate the inference of future frames in
video sequences. For a first given experience, an initial Variational
Autoencoder, together with a set of fully connected neural networks
are utilized to respectively learn the appearance of video frames and
their dynamics at the latent space level. By employing an adapted
Markov Jump Particle Filter, the proposed method recognizes new
situations and integrates them as predictive models avoiding catastrophic forgetting of previously learned tasks. For evaluating the
proposed method, this article uses video sequences from a vehicle
that performs different tasks in a controlled environment
A data-driven approach for the localization of interacting agents via a multi-modal Dynamic Bayesian Network framework
No full textProceedings of 2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 29 Nov. - 2 Dec. 2022, Madrid, SpainThis paper proposes a multi-modal situational inter-action model for collaborative agents by fusing multi-sensorial information in a Multi-Agent Hierarchical Dynamic Bayesian Network (MAH-DBN) framework. The proposed model is learned in a data-driven methodology to estimate the states of interacting agents only from video sequences. This can be regarded as a two-fold methodology for improving visual-based localization and interaction between autonomous agents. In the learning stage, the odometry model is used to drive the video learning model for a robust localization and interaction modeling. During the testing phase, the learned Multi-Agent Hierarchical DBN (MAH-DBN) model is used for the localization of collaborative agents only from video sequences by proposing an inference method called Multi-Agent Coupled Markov Jump Particle Filter (MAC-MJPF)
Giulia Veronica Varisco
No full textThe headword explains the biography and the contribution of the author Giulia Varisco to the children's literatur
Simultaneous localization and anomaly detection from first-person video data through a Coupled Dynamic Bayesian Network model
No full textProceedings of 2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 29 Nov. - 2 Dec. 2022, Madrid, SpainThis paper proposes a method to localize a moving agent - such as an autonomous surveillance vehicle - inside a known environment using First Person Viewpoint video data. Anomalies w.r.t. expected vehicle motion and image content are extracted to guide the localization, signal when the localization results are not trustworthy and explain the reason for the failure. During the training phase, a Dynamic Bayesian Network model is learned, which couples positional and video data. To learn it, clustering is performed on the odometry data, and a modified Kalman Variational Autoencoder is built over the video data. During the testing phase, a Coupled Markov Jump Particle Filter leverages the learned Dynamic Bayesian Network to extract anomalies and to estimate the vehicle’s position, given only camera data. The proposed method is evaluated on two real-world datasets of a vehicle performing perimeter monitoring of a closed environment and of a shopping cart moving in a supermarket
Anomaly Detection in Video Data Based on Probabilistic Latent Space Models
No full textThis paper proposes a method for detecting anomalies in video data. A Variational Autoencoder (VAE) is used
for reducing the dimensionality of video frames, generating
latent space information that is comparable to low-dimensional
sensory data (e.g., positioning, steering angle), making feasible
the development of a consistent multi-modal architecture for
autonomous vehicles. An Adapted Markov Jump Particle Filter
defined by discrete and continuous inference levels is employed to
predict the following frames and detecting anomalies in new video
sequences. Our method is evaluated on different video scenarios
where a semi-autonomous vehicle performs a set of tasks in a
closed environment
- …
