131 research outputs found
Hybrid Inference with Invertible Neural Networks in Factor Graphs
Get PDFThis paper bridges the gap in the literature between neural networks and probabilistic graphical models. Invertible neural networks are incorporated in factor graphs and inference in this model is described by linearization of the network. Consequently, hybrid probabilistic inference in the model is realized through message passing with local constraints on the Bethe free energy. We provide the local Bethe free energy for the invertible neural network node, which allows for evaluation of the performance of the entire probabilistic model. Experimental results show effective hybrid inference in a neural network-based probabilistic model for a binary classification task, paving the way towards a novel class of machine learning models
Variational Log-Power Spectral Tracking for Acoustic Signals
No full textThis paper proposes a generative hierarchical probabilistic model for acoustic signals where both the frequency decomposition and log-power spectrum appear as latent variables. In order to facilitate efficient inference, we represent the model in a factor graph that includes a probabilistic Fourier transform and a Gaussian scale model as modules. We derive novel ways of performing variational message passing-based inference in the Gaussian scale model. As a result, in this model a probabilistic representation of the log-power spectrum of an acoustic signal can be effectively inferred online. The proposed model may find applications as a front end wherever probabilistic log-power spectral features of a signal are needed. We validate the model and message passing-based inference methods by tracking the log-power spectrum of a speech signal
Single trial ERP reading based on parallel factor analysis
No full textThe extraction of task-related single trial ERP features has recently gained much interest, in particular in simultaneous EEG-fMRI applications. In this study, a specific decomposition known as parallel factor analysis (PARAFAC) was used, in order to retrieve the task-related activity from the raw signals. Using visual detection task data, acquired in normal circumstances and simultaneously with fMRI, differences between distinct task-related conditions can be captured in the trial signatures of specific PARAFAC components when applied to ERP data arranged in Channels?x?Time?x?Trials arrays, but the signatures did not correlate with the fMRI data. Despite the need for parameter tuning and careful preprocessing, the approach is shown to be successful, especially when prior knowledge about the expected ERPs is incorporated
Efficient Model Evidence Computation in Tree-structured Factor Graphs
No full textModel evidence is a fundamental performance measure in Bayesian machine learning as it represents how well a model fits an observed data set. Since model evidence is often an intractable quantity, the literature often resorts to computing instead the Bethe Free Energy (BFE), which for cyclefree models is a tractable upper bound on the (negative log-) model evidence. In this paper, we propose a different and faster evidence computation approach by tracking local normalization constants of sum-product messages, termed scale factors. We tabulate scale factor update rules for various elementary factor nodes and by experimental validation we verify the correctness of these update rules for models involving both discrete and continuous variables. We show how tracking scale factors leads to performance improvements compared to the traditional BFE computation approach
Improved Depth Estimation of Bayesian Neural Networks
Get PDFThis paper proposes improvements over earlier work by Nazareth and Blei (2022) for estimating the depth of Bayesian neural networks. Here, we propose a discrete truncated normal distribution over the network depth to independently learn its mean and variance. Posterior distributions are inferred by minimizing the variational free energy, which balances the model complexity and accuracy. Our method improves test accuracy on the spiral data set and reduces the variance in posterior depth estimates
Online Single-Microphone Source Separation using Non-Linear Autoregressive Models
Get PDFIn this paper a modular approach to single-microphone source separation is proposed. A probabilistic model for mixtures of observations is constructed, where the independent underlying source signals are described by non-linear autoregressive models. Source separation in this model is achieved by performing online probabilistic inference through an efficient message passing procedure. For retaining tractability with the non-linear autoregressive models, three different approximation methods are described. A set of experiments shows the effectiveness of the proposed source separation approach. The source separation performance of the different approximation methods is quantified through a set of verification experiments. Our approach is validated in a speech denoising task
Online Single-Microphone Source Separation using Non-Linear Autoregressive Models
No full textIn this paper a modular approach to single-microphone source separation is proposed. A probabilistic model for mixtures of observations is constructed, where the independent underlying source signals are described by non-linear autoregressive models. Source separation in this model is achieved by performing online probabilistic inference through an efficient message passing procedure. For retaining tractability with the non-linear autoregressive models, three different approximation methods are described. A set of experiments shows the effectiveness of the proposed source separation approach. The source separation performance of the different approximation methods is quantified through a set of verification experiments. Our approach is validated in a speech denoising task
- …
