1,720,974 research outputs found
Estimation of correlations between cross-sectional estimates from repeated surveys: an application to the variance of change
No full textMeasuring change over time is a central problem for many users of social, economic and demographic data and is of interest in many areas of economics and social sciences. Smith et al. (2003) recognised that assessing change is one of the most important challenges in survey statistics. The primary interest of many users is often in changes or trends from one time period to another. A common problem is to compare two cross-sectional estimates for the same study variable taken on two different waves or occasions, and to judge whether the observed change is statistically significant. This involves the estimation of the sampling variance of the estimator of change. Estimation of variance of change would be relatively straightforward if cross-sectional estimates were based upon the same sample. Unfortunately, samples from different waves are usually not completely overlapping sets of units, because of rotations used in repeated surveys. This implies that crosssectional estimates are not independent. Correlation plays an important role in estimating the variance of a change between the cross-sectional estimates. The unbiasedness of an estimator of a correlation is crucial, because a small bias can significantly over-estimate or under-estimate the variance of change (Berger, 2004). Several methods can be used to estimate correlations, some of which use re-sampling and/or Taylor linearization. We propose to use a multivariate linear regression approach to estimate the correlation. The proposed estimator is not a model-based estimator, as this estimator is valid even if the model does not fit the data. We show that the regression approach gives design-consistent estimator for the correlation when the finite population corrections are negligible. We show how the proposed estimator can accommodate stratified and two-stage sampling designs. We also show how the proposed estimator can be used for estimator of correlation between complex estimators of change
Symmetric generative methods and tSNE: a short survey
No full textIn data visualization, a family of methods is dedicated to the symmetric numerical matrices which contain the distances or similarities between high-dimensional data vectors. The method t-Distributed Stochastic Neighbor Embedding and its variants lead to competitive nonlinear embeddings which are able to reveal the natural classes. For comparisons, it is surveyed the recent probabilistic and model-based alternative methods from the literature (LargeVis, Glove, Latent Space Position Model, probabilistic Correspondence Analysis, Stochastic Block Model) for nonlinear embedding via low dimensional positions.</p
Generalized topographic block model
No full textCo-clustering leads to parsimony in data visualisation with a number of parameters dramatically reduced in comparison to the dimensions of the data sample. Herein, we propose a new generalized approach for nonlinear mapping by a re-parameterization of the latent block mixture model. The densities modeling the blocks are in an exponential family such that the Gaussian, Bernoulli and Poisson laws are particular cases. The inference of the parameters is derived from the block expectation–maximization algorithm with a Newton–Raphson procedure at the maximization step. Empirical experiments with textual data validate the interest of our generalized model
A simple variance estimator of change for rotating repeated surveys: an application to the EU-SILC household surveys
No full textA common problem is to compare two cross-sectional estimates for the same study variable taken on two different waves or occasions, and to judge whether the change observed is statistically significant. This involves the estimation of the sampling variance of the estimator of change. The estimation of this variance would be relatively straightforward if cross-sectional estimates were based on the same sample. Unfortunately, samples are not completely overlapping, because of rotations used in repeated surveys. We propose a simple approach based on a multivariate (general) linear regression model. The variance estimator proposed is not a model-based estimator. We show that the estimator proposed is design consistent when the sampling fractions are negligible. It can accommodate stratified and two-stage sampling designs. The main advantage of the approach proposed is its simplicity and flexibility. It can be applied to a wide class of sampling designs and can be implemented with standard statistical regression techniques. Because of its flexibility, the approach proposed is well suited for the estimation of variance for the European Union Statistics on Income and Living Conditions surveys. It allows us to use a common approach for variance estimation for the different types of design. The approach proposed is a useful tool, because it involves only modelling skills and requires limited knowledge of survey sampling theory
Multiway latent block model for pca tensor decomposition
No full textA multiway latent block model (WLBM) is proposed for modeling the reduction and the clustering of m-modes tensors via matricial latent variables. BEM-like algorithms allow the parameters inference of the models for tensors with multiple modes. This results into summarizing high-dimensional tensors for pca reduction and visualization in a generic multidimensional setting. The parameterizations are -related or approximated- Canonical Polyadic and Tucker decompositions with dramatically less dimensions than their usual full versions for the latent components
Probabilistic Elastic Embedding Model: Comparison of Alternative Models
No full textIn data visualization, Elastic Embedding adds an exponential penalty to an Euclidean criterion. It is able to separate the natural classes but its lacks a probabilistic generative setting which brings more flexibility to the modeling and the inference. Hence, it is proposed a new generative interpretation of Elastic Embedding which is closely related to LargeVis. Numerical experiments compare the proposed model and several alternative ones via two new visual indicators among different approaches
Visualization of generalized mean estimators using auxiliary information in survey sampling: additive case
No full textThe mean estimators with ratio depend on multiple auxiliary variables and unknown parameters in a finite population setting. Recently a new generic approach for modeling multivariate mean estimators with matrices has been proposed in order to compute automatically their minimum mean squared error. This brings naturally a graphical analysis for comparing mean estimators via nonlinear curves of their approximated mean squared error or their bias. Herein generalized additive ratio estimators with two auxiliary variables and higher order expansions in the approximations are proposed. This is just after a brief review of the new generic method, with an extension to constrained parameters. This leads to complete the main matrix in stake with higher-order moments of the auxiliary and target variables while keeping an underlying regression model for the optimization. A perspective is the visualization of alternative models under this framework when empirical means are associated with ratio functions of auxiliary variables
Narrative and Emotional Structures For Generation Of Short Texts For Advice
No full textIt is proposed a new method for the generation of texts using a large language model (LLM) with a double structural foundation. The supervised generation requires two proposed sets of narrative structures (SN) and emotional dynamics (DE). An explicit definition of 51 SN et 9 DE associated to a user table of topics and characters allows to generate automatically short texts. The main purpose of such texts is to provide advices or learning concepts like wanted in health or security but not often met in practice. A synthetic corpus of 36 texts automatically written on the topic of the urban walk in the experimental section demonstrates the feasibility of the approach. This allows a diversification and a planification of the texts contents as validated in the experiments
Linear and Deep Models Basics with Pytorch, Numpy, and Scikit-Learn
No full textInternational audienceThis book is an introduction to computational statistics for the generalized linear models (glm) and to machine learning with the python language. Extensions of the glm with nonlinearities come from hidden layer(s) within a neural network for linear and nonlinear regression or classification. This allows to present side by side classical statistics and current deep learning. The loglikelihoods and the corresponding loss functions are explained. The gradient and hessian matrix are discussed and implemented for these linear and nonlinear models. Several methods are implemented from scratch with numpy for prediction (linear, logistic, poisson regressions) and for reduction (principal component analysis, random projection). The gradient descent, newton-raphson, natural gradient and l-fbgs algorithms are implemented. The datasets in stake are with 10 to 10^7 rows, and are tabular such that images or texts are vectorized. The data are stored in a compressed format (memmap or hdf5) and loaded by chunks for several case studies with pytorch or scikit-learn. Pytorch is presented for training with minibatches via a generic implementation for studying with computer programs. Scikit-learn is presented for processing large datasets via the partial fit, after the small examples. Sixty exercises are proposed at the end of the chapters with selected solutions to go beyond the contents. Code available at https://github.com/rpriam/book
Visualization of generalized mean estimators using auxiliary information in survey sampling
No full textIn this communication, it is proposed a generalized method for modeling mean estimators. The mean estimators depend on multiple auxiliary variates and unknown parameters in a finite population setting. Our approach brings naturally a graphical analysis for comparing and improving mean estimators
- …
