66 research outputs found
Generalized discriminant analysis via kernel exponential families
No full textThis paper introduces a novel supervised dimension reduction method for classification and regression problems using reproducing kernel Hilbert spaces. The proposed approach takes advantage of the modeling power of kernel exponential families to extract nonlinear summary statistics of the data that are sufficient to preserve information about the target response. For the special case of finite dimensional exponential family distributions, the proposed method is shown to simplify the known solutions for sufficient dimension reduction. A connection with support vector machines is shown and exploited to obtain efficient estimation procedures. Experiments with simulated and real data illustrate the potential of the proposed approach.Fil: Ibañez, Diego Isaías. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral; ArgentinaFil: Forzani, Liliana Maria. Universidad Nacional del Litoral; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: Tomassi, Diego Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentin
Discriminative information in classifiers based on hidden Markov models
No full textFil: Tomassi, Diego Rodolfo. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas; Argentina.Hidden Markov models (HMM) are statistical models which can efficiently deal with sequential data. They provide a way to model complex dependencies between observed data by setting simple dependencies between latent variables: a Markov chain that is not available to the observer. When used in a classification setting, an HMM models the probability density function of the data from each class and label assignement is achieved using a plug-in Bayes classifier. This is a typical example of generative learning, which can be suboptimal when the data does not match the assumed distribution. In this thesis we study methods and algorithms to exploit discriminant information when using HMM to classify sequential data. In the first part, we deal with HMM defined on the wavelet transform of the input sequences. These are hierarchical Markovian structures that use hidden Markov trees as observation models for the wavelet coefficients, given the state of the underlying chain. We derive new training algorithms for these models, specifically targeted to achieve minimum classification error. In the second part of the thesis, we take a look back to HMM with mixtures of Gaussians as observation densities. We focus in scenarios of high-dimensional observed data and derive methods for dimension reduction of the feature space using the approach of statistical sufficiency, which aims to preserve class information in the reduced data. We derive new algorithms and use this framework to analyze information preservation attained by available methods of dimensionality reduction in HMM.Los modelos ocultos de Markov (HMM) son modelos estadísticos usados frecuentemente con datos sequenciales. Proveen un medio eficaz para modelar dependencias complejas a través de dependencias sencillas entre variables latentes que forman una cadena de Markov. Cuando se usan en tareas de clasificación, un HMM modela la función de densidad de probabilidad de los datos de cada clase y la asignación de etiquetas se realiza usando una versión plug-in de la regla de decisión de Bayes. Esto es un ejemplo de aprendizaje generativo, que puede ser subóptimo cuando la distribución de los datos se aparta de la supuesta. En esta tesis se estudian métodos y algoritmos que tienen por objeto aprovechar información discriminante en la clasificación de datos secuenciales modelados con HMM. En la primera parte del trabajo abordamos problemas con HMM definidos sobre la transformada onditas de las secuencias de entrada. Se trata de HMM jerárquicos que usan árboles ocultos de Markov como modelo de observación para los coeficientes de la transforma. Proponemos nuevos algoritmos de entrenamiento para estos modelos, basados directamente en la minimización del error de clasificación como criterio de aprendizaje. En la segunda parte de la tesis revisitamos los HMM más comunes que usan mezclas de gaussianas como modelos de observación, pero nos enfocamos en escenarios de alta dimensionalidad. Derivamos métodos para reducir la dimensión del espacio de características sin perder información discriminante, usando para ello el enfoque de suficiencia estadística. Proponemos nuevos algoritmos y analizamos bajo este marco métodos existentes de reducción dimensional en HMM.Consejo Nacional de Investigaciones Científicas y Técnica
Discriminative information in classifiers based on hidden Markov models
No full textFil: Tomassi, Diego Rodolfo. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas; Argentina.Hidden Markov models (HMM) are statistical models which can efficiently deal with sequential data. They provide a way to model complex dependencies between observed data by setting simple dependencies between latent variables: a Markov chain that is not available to the observer. When used in a classification setting, an HMM models the probability density function of the data from each class and label assignement is achieved using a plug-in Bayes classifier. This is a typical example of generative learning, which can be suboptimal when the data does not match the assumed distribution. In this thesis we study methods and algorithms to exploit discriminant information when using HMM to classify sequential data. In the first part, we deal with HMM defined on the wavelet transform of the input sequences. These are hierarchical Markovian structures that use hidden Markov trees as observation models for the wavelet coefficients, given the state of the underlying chain. We derive new training algorithms for these models, specifically targeted to achieve minimum classification error. In the second part of the thesis, we take a look back to HMM with mixtures of Gaussians as observation densities. We focus in scenarios of high-dimensional observed data and derive methods for dimension reduction of the feature space using the approach of statistical sufficiency, which aims to preserve class information in the reduced data. We derive new algorithms and use this framework to analyze information preservation attained by available methods of dimensionality reduction in HMM.Los modelos ocultos de Markov (HMM) son modelos estadísticos usados frecuentemente con datos sequenciales. Proveen un medio eficaz para modelar dependencias complejas a través de dependencias sencillas entre variables latentes que forman una cadena de Markov. Cuando se usan en tareas de clasificación, un HMM modela la función de densidad de probabilidad de los datos de cada clase y la asignación de etiquetas se realiza usando una versión plug-in de la regla de decisión de Bayes. Esto es un ejemplo de aprendizaje generativo, que puede ser subóptimo cuando la distribución de los datos se aparta de la supuesta. En esta tesis se estudian métodos y algoritmos que tienen por objeto aprovechar información discriminante en la clasificación de datos secuenciales modelados con HMM. En la primera parte del trabajo abordamos problemas con HMM definidos sobre la transformada onditas de las secuencias de entrada. Se trata de HMM jerárquicos que usan árboles ocultos de Markov como modelo de observación para los coeficientes de la transforma. Proponemos nuevos algoritmos de entrenamiento para estos modelos, basados directamente en la minimización del error de clasificación como criterio de aprendizaje. En la segunda parte de la tesis revisitamos los HMM más comunes que usan mezclas de gaussianas como modelos de observación, pero nos enfocamos en escenarios de alta dimensionalidad. Derivamos métodos para reducir la dimensión del espacio de características sin perder información discriminante, usando para ello el enfoque de suficiencia estadística. Proponemos nuevos algoritmos y analizamos bajo este marco métodos existentes de reducción dimensional en HMM.Consejo Nacional de Investigaciones Científicas y Técnica
Sufficient dimension reduction for censored predictors
No full textMotivated by a study conducted to evaluate the associations of 51 inflammatory markers and lung cancer risk, we propose several approaches of varying computational complexity for analyzing multiple correlated markers that are also censored due to lower and/or upper limits of detection, using likelihood-based sufficient dimension reduction (SDR) methods. We extend the theory and the likelihood-based SDR framework in two ways: (i) we accommodate censored predictors directly in the likelihood, and (ii) we incorporate variable selection. We find linear combinations that contain all the information that the correlated markers have on an outcome variable (i.e., are sufficient for modeling and prediction of the outcome) while accounting for censoring of the markers. These methods yield efficient estimators and can be applied to any type of outcome, including continuous and categorical. We illustrate and compare all methods using data from the motivating study and in simulations. We find that explicitly accounting for the censoring in the likelihood of the SDR methods can lead to appreciable gains in efficiency and prediction accuracy, and also outperformed multiple imputations combined with standard SDR.Fil: Tomassi, Diego Rodolfo. Universidad Nacional del Litoral; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: Forzani, Liliana Maria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral; ArgentinaFil: Bura, Efstathia. The George Washington University; Estados UnidosFil: Pfeiffer, Ruth. National Cancer Institute; Estados Unido
Socioeconomic Index for Income and Poverty Prediction: A Sufficient Dimension Reduction Approach
No full textThe present paper introduces a novel method for the construction of Socioeconomic Status (SES) indices that are specic to a target variable of interest. It is based on the Sufficient Dimension Reduction (SDR) paradigm and uses a factorized model-based approach to simultaneously deal with predictor variables of mixed nature (i.e. quantitative, binary, and ordinal), which are usual in microeconomic data. These SES indices also identify relevant predictor variables using a two-step regularized matrix factorization approach. Using data from household surveys for Argentina (Encuesta Permanente de Hogares-EPH), the proposed method is compared with other existing dimension reduction algorithms such as standard Principal Component Analysis (PCA) and its version for mixed variables, regression on the full set of variables and Least Absolute Shrinkage and Selection Operator (LASSO) regression.Fil: Duarte, Sabrina Lorena. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: Forzani, Liliana Maria. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: Llop Orzan, Pamela Nerina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: García Arancibia, Rodrigo. Universidad Nacional del Litoral. Facultad de Ciencias Económicas. Instituto de Economía Aplicada Litoral; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; ArgentinaFil: Tomassi, Diego Rodolfo. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentin
Sufficient reductions in regression with mixed predictors
No full textMost data sets comprise of measurements on continuous and categorical variables. Yet,modeling high-dimensional mixed predictors has received limited attention in the regressionand classication statistical literature. We study the general regression problem of inferringon a variable of interest based on high dimensional mixed continuous and binary predictors.The aim is to nd a lower dimensional function of the mixed predictor vector that containsall the modeling information in the mixed predictors for the response, which can be eithercontinuous or categorical. The approach we propose identies sucient reductions byreversing the regression and modeling the mixed predictors conditional on the response.We derive the maximum likelihood estimator of the sucient reductions, asymptotic testsfor dimension, and a regularized estimator, which simultaneously achieves variable (feature)selection and dimension reduction (feature extraction). We study the performance of theproposed method and compare it with other approaches through simulations and real dataexamples.Fil: Bura, Efstathia. Technische Universitat Wien; AustriaFil: Forzani, Liliana Maria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; ArgentinaFil: García Arancibia, Rodrigo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ciencias Económicas. Instituto de Economía Aplicada Litoral; ArgentinaFil: Llop Orzan, Pamela Nerina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; ArgentinaFil: Tomassi, Diego Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentin
Sufficient reductions in regression with mixed predictors
No full textMost data sets comprise of measurements on continuous and categorical variables. Yet,modeling high-dimensional mixed predictors has received limited attention in the regressionand classication statistical literature. We study the general regression problem of inferringon a variable of interest based on high dimensional mixed continuous and binary predictors.The aim is to nd a lower dimensional function of the mixed predictor vector that containsall the modeling information in the mixed predictors for the response, which can be eithercontinuous or categorical. The approach we propose identies sucient reductions byreversing the regression and modeling the mixed predictors conditional on the response.We derive the maximum likelihood estimator of the sucient reductions, asymptotic testsfor dimension, and a regularized estimator, which simultaneously achieves variable (feature)selection and dimension reduction (feature extraction). We study the performance of theproposed method and compare it with other approaches through simulations and real dataexamples.Fil: Bura, Efstathia. Technische Universitat Wien; AustriaFil: Forzani, Liliana Maria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; ArgentinaFil: García Arancibia, Rodrigo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ciencias Económicas. Instituto de Economía Aplicada Litoral; ArgentinaFil: Llop Orzan, Pamela Nerina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; ArgentinaFil: Tomassi, Diego Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química; Argentin
Sufficient dimension reduction for compositional data
No full textRecent efforts to characterize the human microbiome and its relation to chronic diseases have led to a surge in statistical development for compositional data. We develop likelihood-based sufficient dimension reduction methods (SDR) to find linear combinations that contain all the information in the compositional data on an outcome variable, i.e., are sufficient for modeling and prediction of the outcome. We consider several models for the inverse regression of the compositional vector or transformations of it, as a function of outcome. They include normal, multinomial, and Poisson graphical models that allow for complex dependencies among observed counts. These methods yield efficient estimators of the reduction and can be applied to continuous or categorical outcomes. We incorporate variable selection into the estimation via penalties and address important invariance issues arising from the compositional nature of the data. We illustrate and compare our methods and some established methods for analyzing microbiome data in simulations and using data from the Human Microbiome Project. Displaying the data in the coordinate system of the SDR linear combinations allows visual inspection and facilitates comparisons across studies.Fil: Tomassi, Diego Rodolfo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; Argentina. Université de Technologie de Troyes; FranciaFil: Forzani, Liliana Maria. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; ArgentinaFil: Duarte, Sabrina Lorena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe; Argentina. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; ArgentinaFil: Pfeiffer, Ruth M.. National Cancer Institute; Estados Unido
Sufficient dimension reduction for a novel class of zero-inflated graphical models
No full textGraphical models allow modeling of complex dependencies among components of a random vector. In many applicationsof graphical models, however, for example microbiome data, the data have an excess number of zero values. Wepresent new pairwise graphical models with distributions in an exponential family, that accommodate excess numbersof zeros in the random vector components. First we characterise these multivariate distributions in terms of univariateconditional distributions. We then model predictors that arise from such a pairwise graphical model with excess zerosas a function of an outcome, and derive the corresponding first order sufficient dimension reduction (SDR). That is,we find linear combinations of the predictors that contain all the information for the regression of the outcome as afunction of the predictors. We estimate the SDR using pseudo-likelihood with a hierarchical penalty that accounts forthe graphical model structure, for variable selection, by allowing interactions only for variables that are associatedwith outcome also through main effects. This method yields consistent estimators of the reduction and can be appliedto continuous or categorical outcomes. We then illustrate our methods by studying normal, Poisson and truncatedPoisson graphical models with excess zeros in simulations and by analyzing microbiome data from the AmericanGut Project. Our models provided robust variable selection and the Poisson zero-inflation pairwise graphical modelresulted in predictive performance that was equal or better than that obtained from applying other available methodsfor the analysis of microbiome data.Fil: Koplin, Eric Lionel. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Forzani, Liliana Maria. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Tomassi, Diego Rodolfo. Universidad Nacional del Litoral. Facultad de Ingeniería Química. Departamento de Matemáticas; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Pfeiffer, Ruth M.. National Cancer Institute; Estados Unido
Wavelet shrinkage using adaptive structured sparsity constraints
No full textStructured sparsity approaches have recently received much attention in the statistics, machine learning, and signal processing communities. A common strategy is to exploit or assume prior information about structural dependencies inherent in the data; the solution is encouraged to behave as such by the inclusion of an appropriate regularisation term which enforces structured sparsity constraints over sub-groups of data. An important variant of this idea considers the tree-like dependency structures often apparent in wavelet decompositions. However, both the constituent groups and their associated weights in the regularisation term are typically defined a priori. We here introduce an adaptive wavelet denoising framework whereby a sparsity-inducing regulariser is modified based on information extracted from the signal itself. In particular, we use the same wavelet decomposition to detect the location of salient features in the signal, such as jumps or sharp bumps. Given these locations, the weights in the regulariser associated to the groups of coefficients that cover these time locations are modified in order to favour retention of those coefficients. Denoising experiments show that, not only does the adaptive method preserve the salient features better than the non-adaptive constraints, but it also delivers significantly better shrinkage over the signal as a whole
- …
