412 research outputs found
A new method for detecting differential item functioning in the Rasch model
Full text linkDifferential item functioning (DIF) can lead to an unfair advantage or disadvantage for certain subgroups in educational and psychological testing. Therefore, a variety of statistical methods has been suggested for detecting DIF in the Rasch model. Most of these methods are designed for the comparison of pre-specified focal and reference groups, such as males and females. Latent class approaches, on the other hand, allow to detect previously unknown groups exhibiting DIF. However, this approach provides no straightforward interpretation of the groups with respect to person characteristics.
Here we propose a new method for DIF detection based on model-based recursive partitioning that can be considered as a compromise between those two extremes. With this approach it is possible to detect groups of subjects exhibiting DIF, which are not prespecified, but result from combinations of observed covariates. These groups are directly interpretable and can thus help understand the psychological sources of DIF.
The statistical background and construction of the new method is first introduced by means of an instructive example, and then applied to data from a general knowledge quiz and a teaching evaluation
sj-pdf-1-epm-10.1177_00131644221143051 – Supplemental material for What Affects the Quality of Score Transformations?
No full textSupplemental material, sj-pdf-1-epm-10.1177_00131644221143051 for What Affects the Quality of Score Transformations? by Carolina Fellinghauer, Rudolf Debelak and Carolin Strobl in Educational and Psychological Measurement</p
Statistical Sources of Variable Selection Bias in Classification Tree Algorithms Based on the Gini Index
Full text linkEvidence for variable selection bias in classification tree algorithms based on the Gini Index is reviewed from the literature and embedded into a broader explanatory scheme: Variable selection bias in classification tree algorithms based on the Gini Index can be caused not only by the statistical effect of multiple comparisons, but also by an increasing estimation bias and variance of the splitting criterion when plug-in estimates of entropy measures like the Gini Index are employed. The relevance of these sources of variable selection bias in the different simulation study designs is examined. Variable selection bias due to the explored sources applies to all classification tree algorithms based on empirical entropy measures like the Gini Index, Deviance and Information Gain, and to both binary and multiway splitting algorithms
DSI Insights: Wege aus der Angst vor Algorithmen
Full text linkWie mindert man "Algorithm Anxiety"? DSI-Fellow Jamie Gloor schlägt mit Carolin Strobl und Rudolf Debelak Ansätze für technisch versierte und weniger versierte Adressaten vor
Conditional Variable Importance for Random Forests
Full text linkRandom forests are becoming increasingly popular in many scientific fields because they can cope with ``small n large p'' problems, complex interactions and even highly correlated predictor variables. Their variable importance measures have recently been suggested as screening tools for, e.g., gene expression studies. However, these variable importance measures show a bias towards correlated predictor variables. We identify two mechanisms responsible for this finding: (i) A preference for the selection of correlated predictors in the tree building process and (ii) an additional advantage for correlated predictor variables induced by the unconditional permutation scheme that is employed in the computation of the variable importance measure. Based on these considerations we develop a new, conditional permutation scheme for the computation of the variable importance measure. The resulting conditional variable importance is shown to reflect the true impact of each predictor variable more reliably than the original marginal approach
The potential of model-based recursive partitioning in the social sciences - Revisiting Ockham's Razor
Full text linkA variety of new statistical methods from the field of machine learning have the potential to offer new impulses for research in the social, educational and behavioral sciences. In this article we focus on one of these methods: model-based recursive partitioning. This algorithmic approach is reviewed and illustrated by means of instructive examples and an application to the Mincer equation. For readers unfamiliar with algorithmic methods, the explanation starts with the introduction of the predecessor method classification and regression trees.
With respect to the application and interpretation of model-based recursive partitioning, we address the principle of parsimony and illustrate that the model-based recursive partitioning approach can be employed to test whether a postulated model is in accordance with Ockham's Razor or whether relevant covariates have been omitted. Finally, an overview of available statistical software is provided to facilitate the applicability in social science
research
Anchor methods for DIF detection: A comparison of the iterative forward, backward, constant and all-other anchor class
Full text linkIn the analysis of differential item functioning (DIF) using item response theory (IRT), a common metric is necessary to compare item parameters between groups of test-takers. In the Rasch model, the same restriction is placed on the item parameters in each group in order to define a common metric. However, the question how the items in the restriction - termed anchor items - are selected appropriately is still a major challenge. This article proposes a conceptual framework for categorizing anchor methods: The anchor class to describe characteristics of the anchor methods and the anchor selection strategy to guide how the anchor items are determined. Furthermore, a new anchor class termed the iterative forward anchor class is proposed. Several anchor classes are implemented with two different anchor selection strategies (the all-other and the single-anchor selection strategy) and are compared in an extensive simulation study. The results show
that the newly proposed anchor class combined with the single-anchor selection strategy is superior in situations where no prior knowledge about the direction of DIF is available. Moreover, it is shown that
the proportion of DIF items in the anchor - rather than the fact whether the anchor includes DIF items at all (termed contamination
in previous studies) - is crucial for suitable DIF analysis
Accounting for Individual Differences in Bradley-Terry Models by Means of Recursive Partitioning
Full text linkThe preference scaling of a group of subjects may not be homogeneous, but different
groups of subjects with certain characteristics may show different preference scalings,
each of which can be derived from paired comparisons by means of the Bradley-Terry model.
Usually, either different models are fit in predefined subsets of the
sample, or the effects of subject covariates are explicitly specified in a parametric
model. In both cases, categorical covariates can be employed directly to distinguish
between the different groups, while numeric covariates are typically discretized
prior to modeling.
Here, a semi-parametric approach for recursive partitioning of Bradley-Terry models is
introduced as a means for identifying groups of subjects with homogeneous preference scalings
in a data-driven way. In this approach, the covariates that -- in main effects or
interactions -- distinguish between groups of subjects with different preference
orderings, are detected automatically from the set of candidate covariates. One main
advantage of this approach is that sensible partitions in numeric covariates are
also detected automatically
Flexible Rasch Mixture Models with Package psychomix
Full text linkMeasurement invariance is an important assumption in the Rasch model and mixture models constitute a flexible way of checking for a violation of this assumption by detecting unobserved heterogeneity in item response data. Here, a general class of Rasch mixture models is established and implemented in R, using conditional maximum likelihood estimation of the item parameters (given the raw scores) along with flexible specification of two model building blocks: (1) Mixture weights for the unobserved classes can be treated as model parameters or based on covariates in a concomitant variable model. (2) The distribution of raw score probabilities can be parametrized in two possible ways, either using a saturated model or a specification through mean and variance. The function raschmix() in the R package "psychomix" provides these models, leveraging the general infrastructure for fitting mixture models in the "flexmix" package. Usage of the function and its associated methods is illustrated on artificial data as well as empirical data from a study of verbally aggressive behavior.mixed Rasch model, Rost model, mixture model, flexmix, R
- …
