76 research outputs found
Resampling from the past to improve on MCMC algorithms
No full textWe introduce the idea that resampling from past observations in a Markov Chain Monte Carlo sampler can fasten convergence. We prove that proper resampling from the past does not disturb the limit distribution of the algorithm. We illustrate the method with two examples. The first on a Bayesian analysis of stochastic volatility models and the other on Bayesian phylogeny reconstruction.Monte Carlo methods, Resampling, Stochastic volatility models, Bayesian phylogeny reconstruction.
sj-docx-1-npx-10.1177_1934578X211044564 - Supplemental material for An <i>α</i>-Sophoradiol Glycoside from the Root Wood of <i>Erythrina senegalensis</i> DC. (Fabaceae) with <i>α</i>-Amylase and <i>α</i>-Glucosidase Inhibitory Potential
No full textSupplemental material, sj-docx-1-npx-10.1177_1934578X211044564 for An α-Sophoradiol Glycoside from the Root Wood of Erythrina senegalensis DC. (Fabaceae) with α-Amylase and α-Glucosidase Inhibitory Potential by Cyrille Tchuente Djoko, Isaac Silvère Gade, Alex De Theodore Atchade, Alfred Ngenge Tamfu, Rodica Mihaela Dinica, Eliezer Sangu, Madeleine Tchoffo Djankou, Celine Henoumont, Sophie Laurent and Emmanuel Talla in Natural Product Communications</p
Topics in sparse Bayesian machine learning
No full text2023This dissertation is devoted to addressing several challenging problems in machine learning via the Bayesian approach. One popular approach to Bayesian deep learning is to use Monte Carlo methods, such as Markov Chain Monte Carlo (MCMC), to approximate the posterior distribution. These methods generate a set of samples from the posterior, which can be used to quantify the uncertainty in the parameters and make probabilistic predictions. Bayesian methods in deep learning provide a framework for incorporating uncertainty into the learning process and can lead to more robust models with improved performance on unseen data. They have been applied to a wide range of problems, including image classification, reinforcement learning, and generative models, among others. This dissertation is organized as follows. First chapter is fast asynchronous sampler in sparse bayesian learning. In this chapter, We propose a very fast approximate Markov Chain Monte Carlo(MCMC) sampling framework that is applicable to a large class of sparse Bayesian inference problems, where the computational cost per iteration in several regression models is of order O(n(s + J)), where n is the sample size, s the underlying sparsity of the model, and J is the size of a randomly selected subset of regressors. This cost can be further reduced by data sub-sampling when stochastic gradient Langevin dynamics are employed. The algorithm is an extension of the asynchronous Gibbs sampler of Johnson et al. (2013), but can be viewed from a statistical perspective as a form of Bayesian iterated sure independent screening (Fan et al. (2009)). We show that in high-dimensional linear regression problems, the Markov chain generated by the proposed algorithm admits an invariant distribution that recovers correctly the main signal with high probability under some statistical assumptions. Furthermore we show that its mixing time is at most linear in the number of regressors. We illustrate the algorithm with several models. Second chapter is A one-step Laplace Approximation for high-dimensional variable selection. In this chapter, we introduce a rapid one-step Laplace approximation method, referred to as OLAP, which effectively tackles the computational burden of variable selection in high dimensions. Our findings demonstrate that this approximation offers a consistent variable selection procedure under reasonable assumptions. Additionally, we establish that the mixing time of the Gibbs sampler, employed for sampling from the posterior distribution of OLAP, scales linearly with the dimension p. Through comprehensive simulations, we validate the efficiency and accuracy of our proposed sampler, highlighting its potential to significantly enhance variable selection processes. Third chapter is Sparse(Cyclical) MCMC in Deep Neural Networks. In this chapter, we propose a general cyclical MCMC framework for a class of Bayesian inference problem, aiming to generate samples from one single mode in each cycle andhave mode swapping among different cycles to capture multimodality. We provide extensive results on the performance of prediction, multimodality of different cyclical MCMC methods on high-dimensional gaussian mixture models. We then introduce the sparse cyclical MCMC sampler in deep neural networks and present promising simulation results from the perspective of uncertainty estimation and calibration
On the computational complexity of MCMC-based estimators in large samples
No full textIn this paper we examine the implications of the statistical large sample theory for the computational complexity of Bayesian and quasi-Bayesian estimation carried out using Metropolis random walks. Our analysis is motivated by the Laplace-Bernstein-Von Mises central limit theorem, which states that in large samples the posterior or quasi-posterior approaches a normal density. Using this observation, we establish polynomial bounds on the computational complexity of general Metropolis random walks methods in large samples. Our analysis covers cases, where the underlying log-likelihood or extremum criterion function is possibly nonconcave, discontinuous, and of increasing dimension. However, the central limit theorem restricts the deviations from continuity and log-concavity of the log-likelihood or extremum criterion function in a very specific manner.
EFFICIENCY BOUNDS FOR SEMIPARAMETRIC MODELS WITH SINGULAR SCORE FUNCTIONS
No full textThis paper is concerned with asymptotic efficiency bounds for the estimation of the finite dimension parameter \theta\in \rset^p of semiparametric models that have singular score function for at the true value . The resulting singularity of the matrix of Fisher information means that the standard bound derived by Begun et. al. ([1]) for is not defined. We study the case of single rank deficiency of the score and focus on the case where the derivative of the root density in the direction of the last parameter component, , is nil while the derivatives in the other directions, , are linearly independent. We then distinguish two cases: (i) The second derivative of the root density in the direction of and the first derivative in the direction of are linearly independent and (ii) The second derivative of the root density in the direction of is also nil but the third derivative in is linearly independent of the first derivative in the direction of .
We show that in both cases, efficiency bounds can be obtained for the estimation of , with and 3, respectively and argue that an estimator is efficient if reaches its bound. We provide the bounds in form of convolution and asymptotic minimax theorems. For case (i), we propose a transformation of the Gaussian variable that appears in our convolution theorem to account for the restricted set of values of . This transformation effectively gives the efficiency bound for the estimation of in the model configuration (i). We apply these results to locally under-identified moment condition models and show that the generalized method of moment (GMM) estimator using as weighting matrix, where is the variance of the estimating function, is optimal even in these non standard settings
Topics in Time Series Analysis with Macroeconomic Applications.
No full textTime series is widely used in many real-world applications. In this thesis, we will focus on the scenarios of panel data and state-space model.
Many investigations have used panel methods to study the relationships between fluctuations in economic activity and mortality. A broad consensus has emerged on the overall procyclical nature of mortality: perhaps counter-intuitively, mortality typically rises above its trend during expansions. This consensus has been tarnished by inconsistent reports on the specific age groups and mortality causes involved. We show that these inconsistencies result, in part, from the trend specifications used in previous panel models. Standard econometric panel analysis involves fitting regression models using ordinary least squares, employing standard errors which are robust to temporal autocorrelation. The model specifications include a fixed effect, and possibly a linear trend, for each time series in the panel. We propose alternative methodology based on nonlinear detrending. Applying our methodology on US data, we obtain more precise and consistent results than previous studies.
Iterated filtering is based on a sequence of particle filtering, which could facilitates likelihood-based inference in Dynamic Stochastic General Equilibrium (DSGE) models. Numerous researchers have studied some examples on filtering dynamic economic models. Recent economic turmoil makes reassessment of structural models an urgent problem. We will compare Particle Filter within Markov Chain Monte Carlo (PMCMC) and Iterated Filtering (MIF) in estimating DSGE model using simulated data.
There is a trade-off between numbers of parameter values sampled each filtering and the number of filtering operation needed. PMCMC is at one extreme of this (only 1 new parameter value per filtering operation; thousands of filtering operations needed). Iterated Filtering is at the other extreme (1 new parameter per particle per time point; 50 filtering operations needed). We will propose p-MIF (p (0< p< 1) new parameter per particle per time point on average) schemes as an intermediate algorithm between these two very different extremes. This is shown to perform better for this sort of problem than either existing methods. We also will apply p-MIF to re-evaluate DSGE model using US data .PhDStatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/96027/1/wzhen_1.pd
Bayesian computation for statistical models with intractable normalizing constants
No full textThis paper deals with some computational aspects in the Bayesian analysis of statistical models with intractable normalizing constants. In the presence of intractable normalizing constants in the likelihood function, traditional MCMC methods cannot be applied. We propose an approach to sample from such posterior distributions. The method can be thought as a Bayesian version of the MCMC-MLE approach of Geyer and Thompson (1992). To the best of our knowledge, this is the first general and asymptotically consistent Monte Carlo method for such problems. We illustrate the method with examples from image segmentation and social network modeling. We study as well the asymptotic behavior of the algorithm and obtain a strong law of large numbers for empirical averages.ou
Using Parallel Computation to Improve Independent Metropolis-Hastings Based Estimation
No full textIn this paper, we consider the implications of the fact that parallel raw-power canbe exploited by a generic Metropolis{Hastings algorithm if the proposed values areindependent. In particular, we present improvements to the independent Metropolis{Hastings algorithm that signicantly decrease the variance of any estimator derivedfrom the MCMC output, for a null computing cost since those improvements arebased on a xed number of target density evaluations. Furthermore, the techniquesdeveloped in this paper do not jeopardize the Markovian convergence properties of thealgorithm, since they are based on the Rao{Blackwell principles of Gelfand and Smith(1990), already exploited in Casella and Robert (1996), Atchade and Perron (2005)and Douc and Robert (2010). We illustrate those improvement both on a toy normalexample and on a classical probit regression model but insist on the fact that they areuniversally applicable.
N-fold way simulated tempering for pairwise interaction point processes
No full textPairwise interaction point processes with strong interaction are usually difficult to
sample. We discuss how Besag lattice processes can be used in a simulated tempering
MCMC scheme to help with the simulation of such processes. We show how
the N-fold way algorithm can be used to sample the lattice processes efficiently
and introduce the N-fold way algorithm into our simulated tempering scheme. To
calibrate the simulated tempering scheme we use the Wang-Landau algorithm
Using parallel computation to improve independent Metropolis–Hastings based estimation
No full textIn this paper, we consider the implications of the fact that parallel raw-power can be exploited by a generic Metropolis–Hastings algorithm if the proposed values are independent from the current value of the Markov chain. In particular, we present improvements to the independent Metropolis–Hastings algorithm that significantly decrease the variance of any estimator derived from the MCMC output, at a null computing cost since those improve-ments are based on a fixed number of target density evaluations that can be produced in parallel. The techniques developed in this paper do not jeopardize the Markovian conver-gence properties of the algorithm, since they are based on the Rao–Blackwell principles of Gelfand and Smith (1990), already exploited in Casella and Robert (1996), Atchade ́ and Perron (2005) and Douc and Robert (2011). We illustrate those improvements both on a toy normal example and on a classical probit regression model, but stress the fact that they are applicable in any case where the independent Metropolis–Hastings is applicable
- …
