1,720,979 research outputs found
Computing Maximum Likelihood Estimates in Log-Linear Models
No full textWe develop computational strategies for extended maximum likelihood estimation, as defined in Rinaldo (2006), for general classes of log-linear models of widespread use, under Poisson and product-multinomial sampling schemes. We derive numerically efficient procedures for generating and manipulating design matrices and we propose various algorithms for computing the extended maximum likelihood estimates of the expectations of the cell counts. These algorithms allow to identify the set of estimable cell means for any given observable table and can be used for modifying traditional goodness-of-fit tests to accommodate for a nonexistent MLE. We describe and take advantage of the connections between extended maximum likelihood estimation in hierarchical log-linear models and graphical models.</p
On Maximum Likelihood Estimation in Log-Linear Models
No full textIn this article, we combine results from the theory of linear exponential families, polyhedral geometry and algebraic geometry to provide analytic and geometric characterizations of log-linear models and maximum likelihood estimation. Geometric and combinatorial conditions for the existence of the Maximum Likelihood Estimate (MLE) of the cell mean vector of a contingency table are given for general log-linear models under conditional Poisson sampling. It is shown that any log-linear model can be generalized to an extended exponential family of distributions parametrized, in a mean value sense, by points of a polyhedron. Such a parametrization is continuous and, with respect to this extended family, the MLE always exists and is unique. In addition, the set of cell mean vectors form a subset of a toric variety consisting of non-negative points satisfying a certain system of polynomial equations. These results of are theoretical and practical importance for estimation and model selection.</p
Properties and Refinements of The Fused Lasso
No full textWe consider estimating an unknown signal, which is both blocky and sparse, corrupted by additive noise. We study three interrelated least squares procedures and their asymptotic properties. The first procedure is the fused lasso, put forward by Friedman et al. (2007), which we modify into a different estimator, called the fused adaptive lasso, with better properties. The other two estimators we discuss solve least squares problems on sieves, one constraining the maximal ℓ1norm and the maximal total variation seminorm, the other restricting the number of blocks and of the number of nonzero coordinates of the signal. We derive conditions for the recovery of the true block partition and the true sparsity pattern by the fused lasso and the fused adaptive lasso, and convergence rates for the sieve estimators, explicitly in terms of the constraining parameters.</p
A Note on the Uniqueness of the Lasso Solution
No full textIn this note we show that, if β₁ and β₂ are two distinct solutions to the lasso problem minβ∈Rp∥ y - Xβ∥2^2 + λ∥β∥1 for some n × p matrix X with p > n, then Xβ1 = Xβ2.</p
Three Centuries of Categorical Data Analysis: Log-linear Models and Maximum Likelihood Estimation
No full textThe common view of the history of contingency tables is that it begins in 1900 with the work of Pearson and Yule, but it extends back at least into the 19th century. Moreover it remains an active area of research today. In this paper we give an overview of this history focussing on the development of log-linear models and their estimation via the method of maximum likelihood. S. N. Roy played a crucial role in this development with two papers co-authored with his students S. K. Mitra and Marvin Kastenbaum, at roughly the mid-point temporally in this development. Then we describe a problem that eluded Roy and his students, that of the implications of sampling zeros for the existence of maximum likelihood estimates for log-linear models. Understanding the problem of on-existence is crucial to the analysis of large sparse contingency tables. We introduce some relevant results from the application of algebraic geometry to the study of this statistical problem.</p
On the Asymptotic Properties of The Group Lasso Estimator for Linear Models
No full textWe establish estimation and model selection consistency, prediction and estimation bounds and persistence for the group-lasso estimator and model selector proposed by Yuan and Lin (2006) for least squares problems when the covariates have a natural grouping structure. We consider the case of a fixed-dimensional parameter space with increasing sample size and the double asymptotic scenario where the model complexity changes with the sample size.</p
The Log-linear Group Lasso Estimator and Its Asymptotic Properties
No full textWe define the group lasso estimator for the natural parameters of the exponential families of distributions representing hierarchical log-linear models under multinomial sampling scheme. Such estimator arises as the unique solution of a convex penalized likelihood program using the group lasso penalty. We illustrate how it is possible to construct, in a straightforward way, an estimator of the underlying log-linear model based on the blocks of non-negative coefficients recovered by the group lasso procedure.
We investigate the asymptotic properties of the group lasso estimator and of the associated model selection criterion in a double-asymptotic framework, in which both the sample size and the model complexity grow simultaneously. We provide conditions guaranteeing that the group lasso estimator is norm consistent and that the group lasso model selection is a consistent procedure, in the sense that, with overwhelming probability as the sample size increases, it will correctly identify all the sets of non-zero interactions among the variables. Provided the sequences of true underlying models is sparse enough, recovery is possible even if the number of cells grows larger than the sample size. Finally, we derive some central limit type of results for the log-linear group lasso estimator.</p
Autoregressive Process Modeling via the Lasso Procedure
No full textThe Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double asymptotic framework where the maximal lag may increase with the sample size. We derive theoretical results establishing various types of consistency. In particular, we derive conditions under which the Lasso estimator for the autoregressive coefficients is model selection consistent, estimation consistent and prediction consistent. Simulation study results are reported.</p
Low-Noise Density Clustering
No full textWe study density-based clustering under low-noise conditions. Our framework allows for sharply defined clusters such as clusters on lower dimensional manifolds. We show that accurate clustering is possible even in high dimensions. We propose two data-based methods for choosing the bandwidth and we study the stability properties of density clusters. We show that a simple graph-based algorithm known as the ``friends-of-friends'' algorithm successfully approximates the high density clusters.</p
Consistency under Sampling of Exponential Random Graph Models
No full textThe growing availability of network data and of scientific interest in distributed systems has led to the rapid development of statistical models of network structure. Typically, however, these are models for the entire network, while the data consists only of a sampled sub-network. Parameters for the whole network, which is what is of interest, are estimated by applying the model to the sub-network. This assumes that the model is consistent under sampling, or, in terms of the theory of stochastic processes, that it defines a projective family. Focussing on the popular class of exponential random graph models (ERGMs), we show that this apparently trivial condition is in fact violated by many popular and scientifically appealing models, and that satisfying it drastically limits ERGM’s expressive power. These results are actually special cases of more general ones about exponential families of dependent random variables, which we also prove. Using such results, we offer easily checked conditions for the consistency of maximum likelihood estimation in ERGMs, and discuss some possible constructive responses.</p
- …
