1,720,996 research outputs found
Bayesian Inference for Complex Data Structures: Theoretical and Computational Advances
Full text linkNo abstract availableIn Bayesian Statistics the modeling of data with complex dependence structures is often obtained by composition of simple dependence assumptions. Such representations facilitate the probabilistic assessment and ease the derivation of analytical and computational results in complex models. In the present thesis we derive novel theoretical and computational results on Bayesian inference for probabilistic clustering and flexible dependence models for complex data structures. We focus on models arising from hierarchical specifications in both parametric and nonparametric frameworks.
More precisely, we derive novel conjugacy results for one of the most applied dynamic regression model for binary time series: the dynamic probit model. Exploiting such theoretical results we derive new efficient sampling schemes improving state-of-the-art approximate or sequential Monte Carlo inference. Motivated by an issue of the well-known nested Dirichlet process, we also introduce a novel model, arising from the composition of Dirichlet processes, to cluster populations and observations across populations simultaneously. We derive a closed form expression for the induced distribution of the random partition which allows to gain a deeper understanding of the theoretical properties and inferential implications of the model and we propose a conditional Markov Chain Monte Carlo (MCMC) algorithm to effectively perform inference. Moreover, we generalize the previous composition of discrete random probabilities defining a novel wide class of species sampling priors which allows to predict future observations in different groups and test for homogeneity among sub-populations.
Posterior inference is feasible thanks to a marginal MCMC routine and urn schemes that allow to evaluate posterior and predictive functionals of interest. Finally, we prove a surprising consistency result for the number of clusters for a popular nonparametric model for clustering, that is the Dirichlet process mixture model. In this way we partially answer an open question in the literature
Marta Catalano, Augusto Fasano, and Giovanni Rebaudo’s contribution to the discussion of ‘Martingale posterior distributions’ by Fong, Holmes and Walker
No full textEntropy regularization in probabilistic clustering
Full text linkBayesian nonparametric mixture models are widely used to cluster observations. However, one major drawback of the approach is that the estimated partition often presents unbalanced clusters' frequencies with only a few dominating clusters and a large number of sparsely-populated ones. This feature translates into results that are often uninterpretable unless we accept to ignore a relevant number of observations and clusters. Interpreting the posterior distribution as penalized likelihood, we show how the unbalance can be explained as a direct consequence of the cost functions involved in estimating the partition. In light of our findings, we propose a novel Bayesian estimator of the clustering configuration. The proposed estimator is equivalent to a post-processing procedure that reduces the number of sparsely-populated clusters and enhances interpretability. The procedure takes the form of entropy-regularization of the Bayesian estimate. While being computationally convenient with respect to alternative strategies, it is also theoretically justified as a correction to the Bayesian loss function used for point estimation and, as such, can be applied to any posterior distribution of clusters, regardless of the specific model used
Bayesian forecasting of multivariate longitudinal zero-inflated counts: an application to civil conflict
No full textEfficient expectation propagation for posterior approximation in high-dimensional probit models
No full textContributed discussion to "Giordano, R., Liu, R., Jordan, M.I., Broderick, T. Evaluating Sensitivity to the Stick-Breaking Prior in Bayesian Nonparametrics (with Discussion). Bayesian Analysis. 2023;18(1):287"
No full textBayesian models based on the Dirichlet process and other stick-breaking priors have been proposed as core ingredients for clustering, topic modeling, and other unsupervised learning tasks. However, due to the flexibility of these models, the consequences of prior choices can be opaque. And so prior specification can be relatively difficult. At the same time, prior choice can have a substantial effect on posterior inferences. Thus, considerations of robustness need to go hand in hand with nonparametric modeling. In the current paper, we tackle this challenge by exploiting the fact that variational Bayesian methods, in addition to having computational advantages in fitting complex nonparametric models, also yield sensitivities with respect to parametric and nonparametric aspects of Bayesian models. In particular, we demonstrate how to assess the sensitivity of conclusions to the choice of concentration parameter and stick-breaking distribution for inferences under Dirichlet process mixtures and related mixture models. We provide both theoretical and empirical support for our variational approach to Bayesian sensitivity analysis
Clustering consistency with Dirichlet process mixtures
Full text linkDirichlet process mixtures are flexible non-parametric models, particularly
suited to density estimation and probabilistic clustering. In this work we
study the posterior distribution induced by Dirichlet process mixtures as the
sample size increases, and more specifically focus on consistency for the
unknown number of clusters when the observed data are generated from a finite
mixture. Crucially, we consider the situation where a prior is placed on the
concentration parameter of the underlying Dirichlet process. Previous findings
in the literature suggest that Dirichlet process mixtures are typically not
consistent for the number of clusters if the concentration parameter is held
fixed and data come from a finite mixture. Here we show that consistency for
the number of clusters can be achieved if the concentration parameter is
adapted in a fully Bayesian way, as commonly done in practice. Our results are
derived for data coming from a class of finite mixtures, with mild assumptions
on the prior for the concentration parameter and for a variety of choices of
likelihood kernels for the mixture
Marta Catalano, Augusto Fasano, Matteo Giordano, and Giovanni Rebaudo’s contribution to the Discussion of ‘Root and community inference on the latent growth process of a network’ by Crane and Xu
No full textA closed-form filter for binary time series
Full text linkNon-Gaussian state-space models arise in several applications, and within
this framework the binary time series setting provides a relevant example.
However, unlike for Gaussian state-space models - where filtering, predictive
and smoothing distributions are available in closed form - binary state-space
models require approximations or sequential Monte Carlo strategies for
inference and prediction. This is due to the apparent absence of conjugacy
between the Gaussian states and the likelihood induced by the observation
equation for the binary data. In this article we prove that the filtering,
predictive and smoothing distributions in dynamic probit models with Gaussian
state variables are, in fact, available and belong to a class of unified
skew-normals (SUN) whose parameters can be updated recursively in time via
analytical expressions. Also the key functionals of these distributions are, in
principle, available, but their calculation requires the evaluation of
multivariate Gaussian cumulative distribution functions. Leveraging SUN
properties, we address this issue via novel Monte Carlo methods based on
independent samples from the smoothing distribution, that can easily be adapted
to the filtering and predictive case, thus improving state-of-the-art
approximate and sequential Monte Carlo inference in small-to-moderate
dimensional studies. Novel sequential Monte Carlo procedures that exploit the
SUN properties are also developed to deal with online inference in high
dimensions. Performance gains over competitors are outlined in a financial
application
Efficient computation of predictive probabilities in probit models via expectation propagation
No full text- …
