1,720,998 research outputs found
Fast exact Bayesian inference for sparse signals in the normal sequence model
No full textWe consider exact algorithms for Bayesian inference with model selection priors (including spike-and-slab priors) in the sparse normal sequence model. Because the best existing exact algorithm becomes numerically unstable for sample sizes over n = 500, there has been much attention for alternative approaches like approximate algorithms (Gibbs sampling, variational Bayes, etc.), shrinkage priors (e.g. the Horseshoe prior and the Spike-and-Slab LASSO) or empirical Bayesian methods. However, by introducing algorithmic ideas from online sequential prediction, we show that exact calculations are feasible for much larger sample sizes: for general model selection priors we reach n = 25 000, and for certain spike-and-slab priors we can easily reach n = 100 000. We further prove a de Finetti-like result for finite sample sizes that characterizes exactly which model selection priors can be expressed as spike-and-slab priors. The computational speed and numerical accuracy of the proposed methods are demonstrated in experiments on simulated data, on a differential gene expression data set, and to compare the effect of multiple hyper-parameter settings in the beta-binomial prior. In our experimental evaluation we compute guaranteed bounds on the numerical accuracy of all new algorithms, which shows that the proposed methods are numerically reliable whereas an alternative based on long division is not. AMS 2000 subject classifications: Primary 62G05; secondary 62F15
An asymptotic analysis of distributed nonparametric methods
Full text linkWe investigate and compare the fundamental performance of several distributed learning methods that have been proposed recently. We do this in the context of a distributed version of the classical signal-in-Gaussian-white-noise model, which serves as a benchmark model for studying performance in this setting. The results show how the design and tuning of a distributed method can have great impact on convergence rates and validity of uncertainty quantification. Moreover, we highlight the difficulty of designing nonparametric distributed procedures that automatically adapt to smoothness
Optimal Distributed Composite Testing in High-Dimensional Gaussian Models With 1-Bit Communication
Full text linkIn this paper we study the problem of signal detection in Gaussian noise in a distributed setting where the local machines in the star topology can communicate a single bit of information. We derive a lower bound on the Euclidian norm that the signal needs to have in order to be detectable. Moreover, we exhibit optimal distributed testing strategies that attain the lower bound. </p
Asymptotic frequentist coverage properties of Bayesian credible sets for sieve priors
Full text linkWe investigate the frequentist coverage properties of (certain) Bayesian credible sets in a general, adaptive, nonparametric framework. It is well known that the construction of adaptive and honest confidence sets is not possible in general. To overcome this problem (in context of sieve type of priors), we introduce an extra assumption on the functional parameters, the so-called “general polished tail” condition. We then show that under standard assumptions, both the hierarchical and empirical Bayes methods, result in honest confidence sets for sieve type of priors in general settings and we characterize their size. We apply the derived abstract results to various examples, including the nonparametric regression model, density estimation using exponential families of priors, density estimation using histogram priors and the nonparametric classification model, for which we show that their size is near minimax adaptive with respect to the considered specific pseudometrics
Kolyan Ray and Botond Szabo’s contribution to the Discussion of “Martingale Posterior Distributions” by Fong, Holmes and Walke
No full textDiscussion of the article: martingale posterior distributions by Fong et al
Spike and slab variational Bayes for high dimensional logistic regression
Full text linkNo abstract availabl
Skewed Bernstein-von Mises theorem and skew-modal approximations
No full textGaussian deterministic approximations are routinely employed in Bayesian statistics to ease inference when the target posterior is intractable. Although these approximations are justified, in asymptotic regimes, by Bernstein-von Mises type results, in practice the expected Gaussian behavior might poorly represent the actual shape of the target posterior, thus affecting approximation accuracy. Motivated by these considerations, we derive an improved class of closed-form and valid deterministic approximations of posterior distributions that arise from a novel treatment of a third-order version of the Laplace method yielding approximations within a tractable family of skew-symmetric distributions. Under general assumptions accounting for misspecified models and non-i.i.d. settings, this family of approximations is shown to have a total variation distance from the target posterior whose convergence rate improves by at least one order of magnitude the one achieved by the Gaussian from the classical Bernstein-von Mises theorem. Specializing such a result to the case of regular parametric models shows that the same accuracy improvement can be also established for the posterior expectation of polynomially bounded functions. Unlike available higher-order approximations based on, for example, Edgeworth expansions, our results prove that it is possible to derive closed-form and valid densities which provide a more accurate, yet similarly-tractable, alternative to Gaussian approximations of the target posterior, while inheriting its limiting frequentist properties. We strengthen these arguments by developing a practical skew-modal approximation for both joint and marginal posteriors which preserves the guarantees of its theoretical counterpart by replacing the unknown model parameters with the corresponding maximum a posteriori estimate. Simulation studies and real-data applications confirm that our theoretical results closely match the empirical gains observed in practice
- …
