71 research outputs found
Living on the Edge: An Unified Approach to Antithetic Sampling
No full textWe identify recurrent ingredients in the antithetic sampling literature leading to a unified sampling framework. We introduce a new class of antithetic schemes that includes the most used antithetic proposals. This perspective enables the derivation of new properties of the sampling schemes: i) optimality in the Kullback--Leibler sense; ii) closed-form multivariate Kendall's and Spearman's ; iii) ranking in concordance order and iv) a central limit theorem that characterizes stochastic behaviour of Monte Carlo estimators when the sample size tends to infinity. The proposed simulation framework inherits the simplicity of the standard antithetic sampling method, requiring the definition of a set of reference points in the sampling space and the generation of uniform numbers on the segments joining the points. We provide applications to Monte Carlo integration and Markov Chain Monte Carlo Bayesian estimation
Markov switching multiple-equation tensor regressions
No full textA new flexible tensor model for multiple-equation regressions that accounts for latent regime changes is proposed. The model allows for dynamic coefficients and multi-dimensional covariates that vary across equations. The coefficients are driven by a common hidden Markov process that addresses structural breaks to enhance the model flexibility and preserve parsimony. A new soft PARAFAC hierarchical prior is introduced to achieve dimensionality reduction while preserving the structural information of the covariate tensor. The proposed prior includes a new multi-way shrinking effect to address over-parametrization issues while preserving interpretability and model tractability. Theoretical results are derived to help with the choice of the hyperparameters. An efficient Markov chain Monte Carlo (MCMC) algorithm based on random scan Gibbs and back-fitting strategy is designed with priority placed on computational scalability of the posterior sampling. The validity of the MCMC algorithm is demonstrated theoretically, and its computational efficiency is studied using numerical experiments in different parameter settings. The effectiveness of the model framework is illustrated using two original real data analyses. The proposed model exhibits superior performance compared to the current benchmark, Lasso regression
Markov Switching Tensor Regressions
No full textA new flexible tensor-on-tenor regression model that accounts for latent regime changes is proposed. The coefficients are driven by a common hidden Markov process that addresses structural breaks to enhance the model's flexibility and preserve parsimony. A new soft PARAFAC hierarchical prior is introduced to achieve dimensionality reduction while preserving the structural information of the covariate tensor. The proposed prior includes a new multi-way shrinking effect to address over-parametrization issues while preserving interpretability and model tractability. An efficient MCMC algorithm is introduced based on a random scan Gibbs and back-fitting strategy. The model framework’s effectiveness is illustrated using financial and commodity market volatility data. The proposed model exhibits superior performance compared to the current benchmark, Lasso regression
Embarrassingly Parallel Sequential Markov-chain Monte Carlo for Large Sets of Time Series
No full textBayesian computation crucially relies on Markov chain Monte Carlo (MCMC) algorithms.
In the case of massive data sets, running the Metropolis-Hastings sampler
to draw from the posterior distribution becomes prohibitive due to the large number
of likelihood terms that need to be calculated at each iteration. In order to perform
Bayesian inference for a large set of time series, we consider an algorithm that combines
“divide and conquer” ideas previously used to design MCMC algorithms for big data
with a sequential MCMC strategy. The performance of the method is illustrated using
a large set of financial data
Convergence and Efficiency of Adaptive MCMC
No full textAdaptive Markov Chain Monte Carlo (MCMC) algorithms attempt to ‘learn’ from the results of past iterations so the Markov chain can converge quicker. Unfortunately, adaptive MCMC algorithms are no longer Markovian, so their convergence is difficult to guarantee.
The first part of this thesis approaches the problem via finite adaption. We develop new diagnostics to determine whether the adaption is still improving the convergence. We present an algorithm which automatically stops adapting once it determines further adaption will not increase the convergence speed. Our algorithm allows the computer to tune a ‘good’ Markov chain through multiple phases of adaption, and then run conventional non-adaptive MCMC. In this way, the efficiency gains of adaptive MCMC can be obtained while still ensuring convergence to the target distribution.
The second part of the thesis proves convergence to stationarity of adaptive MCMC algorithms, assuming only simple easily-verifiable upper and lower bounds on transition densities. In particular, the transition and proposal densities are not required to be continuous, thus improving on the previous ergodicity results of Craiu et al. (2015).
The third part of the thesis develops an adaptive algorithm which can locally adjust to an irregularly-shaped target distribution. When the target distribution of a Markov chain is irregularly shaped, a ‘good’ proposal distribution for one part of the state space might be a ‘poor’ one for another part of the state space. We consider a component-wise multiple-try Metropolis (CMTM) algorithm that can automatically choose a better proposal out of a set of proposals from different distributions. The computational efficiency is increased using an adaption rule for the CMTM algorithm that dynamically builds a better set of proposal distributions as the Markov chain runs. We also demonstrated theoretically the ergodicity of the adaptive chain.Ph.D
Convergence and Efficiency of Adaptive MCMC
No full textAdaptive Markov Chain Monte Carlo (MCMC) algorithms attempt to ‘learn’ from the results of past iterations so the Markov chain can converge quicker. Unfortunately, adaptive MCMC algorithms are no longer Markovian, so their convergence is difficult to guarantee.
The first part of this thesis approaches the problem via finite adaption. We develop new diagnostics to determine whether the adaption is still improving the convergence. We present an algorithm which automatically stops adapting once it determines further adaption will not increase the convergence speed. Our algorithm allows the computer to tune a ‘good’ Markov chain through multiple phases of adaption, and then run conventional non-adaptive MCMC. In this way, the efficiency gains of adaptive MCMC can be obtained while still ensuring convergence to the target distribution.
The second part of the thesis proves convergence to stationarity of adaptive MCMC algorithms, assuming only simple easily-verifiable upper and lower bounds on transition densities. In particular, the transition and proposal densities are not required to be continuous, thus improving on the previous ergodicity results of Craiu et al. (2015).
The third part of the thesis develops an adaptive algorithm which can locally adjust to an irregularly-shaped target distribution. When the target distribution of a Markov chain is irregularly shaped, a ‘good’ proposal distribution for one part of the state space might be a ‘poor’ one for another part of the state space. We consider a component-wise multiple-try Metropolis (CMTM) algorithm that can automatically choose a better proposal out of a set of proposals from different distributions. The computational efficiency is increased using an adaption rule for the CMTM algorithm that dynamically builds a better set of proposal distributions as the Markov chain runs. We also demonstrated theoretically the ergodicity of the adaptive chain.Ph.D
Interacting multiple try algorithms with different proposal distributions
No full textWe introduce a new class of interacting Markov chain Monte Carlo (MCMC) algorithms which is designed to increase the efficiency of a modified multiple-try Metropolis (MTM) sampler. The extension with respect to the existing MCMC literature is twofold. First, the sampler proposed extends the basic MTM algorithm by allowing for different proposal distributions in the multiple-try generation step. Second, we exploit the different proposal distributions to naturally introduce an interacting MTM mechanism (IMTM) that expands the class of population Monte Carlo methods and builds connections with the rapidly expanding world of adaptive MCMC. We show the validity of the algorithm and discuss the choice of the selection weights and of the different proposals. The numerical studies show that the interaction mechanism allows the IMTM to efficiently explore the state space leading to higher efficiency than other competing algorithms
Conditional Copula Inference and Efficient Approximate MCMC
No full textThis thesis consists of two main parts. The first part focuses on parametric conditional copula models that allow the copula parameters to vary with a set of covariates according to an unknown calibration function. Flexible Bayesian inference for the calibration function of a
bivariate conditional copula is introduced. The prior distribution over the set of smooth calibration functions is built using a sparse Gaussian Process prior for the Single Index Model. The estimation of parameters from the marginal distributions and the calibration function is done jointly via Markov Chain Monte Carlo sampling from the full posterior distribution. A new Conditional Cross Validated Pseudo-Marginal criterion is used to perform copula selection and is modified using a permutation-based procedure to assess data support for the simplifying assumption.
The first part concludes with methods for establishing data support for the simplifying assumption in a bivariate conditional copula model. After splitting the observed data into training and test sets, the method proposed will use a flexible Bayesian model fit to the training data to define tests based on randomization and standard asymptotic theory. I discuss theoretical justification for the method and implementations in alternative models of interest: Gaussian, Logistic and Quantile regressions. The performance is studied via simulated data.
The second part of the thesis focuses on approximate Bayesian methods. Approximate Bayesian Computation (ABC) and Bayesian Synthetic Likelihood (BSL) are popular simulation based methods for sampling from the posterior distribution when the likelihood is not tractable but simulations for each parameter are easily available. However these methods can be computationally inefficient since a large number of pseudo-data simulations is required. I propose to use perturbed MCMC versions of ABC and BSL algorithms and attempt to significantly accelerate these samplers. The main idea of the proposed strategy is to utilize past samples with k-Nearest-Neighbor approach for likelihood approximation. This general method works for ABC and BSL and greatly reduces computational cost and number of required simulations for these samplers. Performance and computational advantage are examined via series of simulation examples. The second part concludes with theoretical justifications and convergence properties of the proposed strategies.Ph.D
Bayesian Inference for Bivariate Conditional Copula Models with Continuous or Mixed Outcomes
No full textThe main goal of this thesis is to develop Bayesian model for studying the influence of
covariate on dependence between random variables. Conditional copula models are flexible tools for modelling complex dependence structures. We construct Bayesian inference for the conditional copula model adapted to regression settings in which the bivariate outcome is continuous or mixed (binary and continuous) and the copula parameter varies with covariate values. The functional relationship between the copula parameter and the covariate is modelled using cubic splines. We also extend our work to additive models which would allow us to handle more than one covariate while keeping the computational burden within reasonable limits. We perform the proposed joint Bayesian inference via adaptive Markov chain Monte Carlo sampling. The deviance information criterion and cross-validated marginal log-likelihood criterion are employed for three model selection problems: 1) choosing the copula family that best fits the data, 2) selecting the calibration function, i.e., checking if parametric form for copula parameter is suitable and 3) determining the number of independent variables in the additive model. The performance of the estimation and model selection techniques are investigated via simulations and demonstrated on two data sets: 1) Matched Multiple Birth and 2) Burn Injury. In which of interest is the influence of gestational age and maternal age on twin birth weights in the former data, whereas in the later data we are interested in investigating how patient’s age affects the severity of burn injury and the probability of death.Ph
Copulas: New Theory and Methods
No full textIn this thesis, we present new theory and methods related to copulas, which are mathematical objects that allow one to model many complex dependence structures and to understand the dependence structure underlying a random vector independently of its marginals.
Our first contribution is to develop a new class of absolutely continuous copulas parameterized by a function space. We prove a number of desirable properties of this family, and demonstrate its utility with an application that defines patterns of neural connectivity. In addition, we show that the family includes a highly irregular copula, thereby illustrating that although intuitively one would expect them to be well-behaved, absolutely continuous copulas can be quite pathological.
Our second contribution is to integrate copulas within hidden Markov models to develop highly flexible models that allow both the copulas and marginals that generate the observed multivariate data to vary across hidden states. We develop from first principles a new efficient estimation algorithm for this model, examine the asymptotic properties of the resulting estimator, and show -- via simulations and an application that predicts room occupancy -- that the model is informative both for the goal of state prediction and for the goal of understanding the data-generating process.
Our final contribution turns to the Bayesian regime, where multivariate "surrogate" data provides information about a latent variable of interest, both through the marginals of the surrogate vector and the dependence within it. We set up a Bayesian hierarchical model and develop an efficient Gibbs sampler to sample from the posterior distribution, and we use simulations and an obstetrical application that monitors fetal health during labour and delivery to show that considering the copula is again informative for understanding the data-generating process.Ph.D
- …
