162,324 research outputs found
Flexible Rasch Mixture Models with Package psychomix
No full textMeasurement invariance is an important assumption in the Rasch model and mixture models constitute a flexible way of checking for a violation of this assumption by detecting unobserved heterogeneity in item response data. Here, a general class of Rasch mixture models is established and implemented in R, using conditional maximum likelihood estimation of the item parameters (given the raw scores) along with flexible specification of two model building blocks: (1) Mixture weights for the unobserved classes can be treated as model parameters or based on covariates in a concomitant variable model. (2) The distribution of raw score probabilities can be parametrized in two possible ways, either using a saturated model or a specification through mean and variance. The function raschmix() in the R package "psychomix" provides these models, leveraging the general infrastructure for fitting mixture models in the "flexmix" package. Usage of the function and its associated methods is illustrated on artificial data as well as empirical data from a study of verbally aggressive behavior.mixed Rasch model, Rost model, mixture model, flexmix, R
Making Rasch decisions: the use of Rasch analysis in the construction of preference based health related quality of life instruments
No full textObjective: To set out the methodological process for using Rasch analysis alongside traditional psychometric methods in the development of a health state classification that is amenable to valuation. Methods: The overactive bladder questionnaire is used to illustrate a four step process for deriving a reduced health state classification from an existing nonpreference based health related quality of life instrument. Step I excludes items that do not meet the initial validation process and step II uses criteria based on Rasch analysis and psychometric testing to select the final items for the health state classification. In step III, item levels are examined and Rasch analysis is used to explore the possibility of reducing the number of item levels. Step IV repeats steps I to III on alternative data sets in order to validate the selection of items for the health state classification. Conclusions: The techniques described enable the construction of a health state classification amenable for valuation exercises that will allow the derivation of preference weights. Thus, the health related quality of life of patients with conditions, like overactive bladder, can be valued and quality adjustment weights such as quality adjusted life years derived.Rasch analysis; health related quality of life; condition specific measure; preference-based measures; overactive bladder syndrome
Estimation of Models in a Rasch Family for Polytomous Items and Multiple Latent Variables
No full textThe Rasch family of models considered in this paper includes models for polytomous items and multiple correlated latent traits, as well as for dichotomous items and a single latent variable. An R package is described that computes estimates of parameters and robust standard errors of a class of log-linear-by-linear association (LLLA) models, which are derived from a Rasch family of models. The LLLA models are special cases of log-linear models with bivariate interactions. Maximum likelihood estimation of LLLA models in this form is limited to relatively small problems; however, pseudo-likelihood estimation overcomes this limitation. Maximizing the pseudo-likelihood function is achieved by maximizing the likelihood of a single conditional multinomial logistic regression model. The parameter estimates are asymptotically normal and consistent. Based on our simulation studies, the pseudo-likelihood and maximum likelihood estimates of the parameters of LLLA models are nearly identical and the loss of efficiency is negligible. Recovery of parameters of Rasch models fit to simulated data is excellent.
Deconstructing therapy outcome measurement with Rasch analysis of a measure of general clinical distress: the Symptom Checklist-90-Revised
No full textRasch analysis was used to illustrate the usefulness of item-level analyses for evaluating a common therapy outcome measure of general clinical distress, the Symptom Checklist-90-Revised (SCL-90-R; Derogatis, 1994). Using complementary therapy research samples, the instrument's 5-point rating scale was found to exceed clients' ability to make reliable discriminations and could be improved by collapsing it into a 3-point version (combining scale points 1 with 2 and 3 with 4). This revision, in addition to removing 3 misfitting items, increased person separation from 4.90 to 5.07 and item separation from 7.76 to 8.52 (resulting in alphas of .96 and .99, respectively). Some SCL-90-R subscales had low internal consistency reliabilities; SCL-90-R items can be used to define one factor of general clinical distress that is generally stable across both samples, with two small residual factors
Estimating the Multilevel Rasch Model: With the lme4 Package
No full textTraditional Rasch estimation of the item and student parameters via marginal maximum likelihood, joint maximum likelihood or conditional maximum likelihood, assume individuals in clustered settings are uncorrelated and items within a test that share a grouping structure are also uncorrelated. These assumptions are often violated, particularly in educational testing situations, in which students are grouped into classrooms and many test items share a common grouping structure, such as a content strand or a reading passage. Consequently, one possible approach is to explicitly recognize the clustered nature of the data and directly incorporate random effects to account for the various dependencies. This article demonstrates how the multilevel Rasch model can be estimated using the functions in R for mixed-effects models with crossed or partially crossed random effects. We demonstrate how to model the following hierarchical data structures: a) individuals clustered in similar settings (e.g., classrooms, schools), b) items nested within a particular group (such as a content strand or a reading passage), and c) how to estimate a teacher x content strand interaction.
A new method for detecting differential item functioning in the Rasch model
No full textDifferential item functioning (DIF) can lead to an unfair advantage or disadvantage for certain subgroups in educational and psychological testing. Therefore, a variety of statistical methods has been suggested for detecting DIF in the Rasch model. Most of these methods are designed for the comparison of pre-specified focal and reference groups, such as males and females. Latent class approaches, on the other hand, allow to detect previously unknown groups exhibiting DIF. However, this approach provides no straightforward interpretation of the groups with respect to person characteristics. Here we propose a new method for DIF detection based on model-based recursive partitioning that can be considered as a compromise between those two extremes. With this approach it is possible to detect groups of subjects exhibiting DIF, which are not prespecified, but result from combinations of observed ovariates. These groups are directly interpretable and can thus help understand the psychological sources of DIF. The statistical background and construction of the new method is first introduced by means of an instructive example, and then applied to data from a general knowledge quiz and a teaching evaluation.item response theory, IRT, Rasch model, di erential item functioning, DIF, structural change, multidimensionality.
The application of measurement theory to tests in mathematics: a study of the goodness-of-fit of rasch model to the alis mathematics test
No full textThe scores provided by the International Test of Developed Ability (ITDA) have been used as an alternative baseline for comparing the progress of students in the A-level Information System (ALIS) project of U.K. The responses of 26,964 examinees to the mathematics items of ITDA in year 2000 were fitted by using the Rasch model. Five subject groups (the population, 2 gender groups and 2 ability groups) and 25 random samples (5 from each group) were generated from the responses of the examinees. The unconditional maximum likelihood estimates of the item difficulty and examinee ability parameters for various groups/samples were produced by the RASCAL program. The scatterplots among different sets of sample item difficulty parameters reflected that the feature of item and ability invariance was not preserved in the groups of extreme abilities. The assumptions of unidimensionality, equal item discrimination, zero guessing factor and non-speededness were generally not supported in the two ability groups. In particular, the result indicated that the ITDA Mathematical Test might be a speeded test. It was quite interesting in this study to see that the item difficulty parameters and examinee abilities estimated from the Classical Test Theory (CTT) and those from the Rasch model were very comparable and both frameworks exhibit more or less the same feature in terms of invariance. On the other hand, more items were "found" unfit by the CTT method than the Rasch approach indicating that the former looks more sensitive to the lack of fit than the latter. To study the effect of speededness, the analysis was repeated with the last 11 items (which has the highest omits) deleted. Disappointingly, the results showed no significant improvement. Further research on the fitness of data with speed incorporated into the estimation of ability level is recommended
Improving the measurement of QALYs in dementia: developing patient- and carer-reported health state classification systems using Rasch analysis
No full textObjectives: Cost-utility analysis is increasingly used to inform resource allocation. This requires a means of valuing health states before and after intervention. Although generic measures are typically used to generate values, these do not perform well with people with dementia. We report the development of a health state classification system amenable to valuation for use in studies of dementia, derived from the DEMQOL system, a measure of health-related quality of life in dementia by patient self-report (DEMQOL) and carer proxy-report (DEMQOL-Proxy). Methods: Factor analysis was used to determine the dimensional structure of DEMQOL and DEMQOL-Proxy. Rasch analysis was subsequently used to investigate item performance across factors in terms of item-level ordering, functioning across subgroups, model fit and severity-range coverage. This enabled the selection of one item from each factor for the classification system. A sample of people with a diagnosis of mild/moderate dementia (n=644) and a sample of carers of those with mild/moderate dementia (n=683) were used. Results: Factor analysis found different 5-factor solutions for DEMQOL and DEMQOL-Proxy. Following item reduction and selection using Rasch analysis, a 5-dimension classification for DEMQOL and a 4-dimension classification for DEMQOL-Proxy were developed. Each item contained 4 health state levels. Conclusion: Combining Rasch and classical psychometric analysis is a valid method of selecting items for dementia health state classifications from both the patient and carer perspectives. The next stage is to obtain preference weights so that the measure can be used in the economic evaluation of treatment, care and support arrangements for dementia.quality adjusted life years; health related quality of life; Rasch analysis; preference-based measures of health; health states; dementia
Avaliações em larga escala com itens de respostas construídas no contexto do modelo multifacetas de Rasch
No full textTese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Produção, Florianópolis, 2015.Esta tese apresenta um estudo sobre as avaliações com itens de respostas construídas em larga escala no contexto do modelo multifacetas de Rasch (LINACRE, 1989 apud LINACRE, 1994). Essas avaliações necessitam de avaliadores para julgar o desempenho das pessoas quanto à habilidade que está sendo medida por meio do teste. Entre as avaliações com itens de respostas construídas mais utilizadas no âmbito educacional e de seleção estão as provas das diversas disciplinas do Ensino Médio, as provas de redação do ENEM e dos concursos vestibulares e as provas com itens abertos de concursos para provimento de vagas de trabalho. Os resultados das avaliações com itens de respostas construídas não dependem apenas do nível de habilidade dos examinandos quanto ao construto avaliado e da dificuldade das tarefas, dependem também da severidade dos avaliadores que julgam os desempenhos e da estrutura da escala de classificação. Um dos principais problemas nessas avaliações é a pontuação de um mesmo desempenho com graus diferentes de severidade. Quando existem vários avaliadores, o ideal é que todos atribuam exatamente a mesma pontuação para os mesmos desempenhos observados, essa é a condição principal para se ter confiabilidade de pontuação. Entretanto, são muitos os fatores que podem causar variabilidade nessas pontuações. O modelo multifacetas de Rasch vem sendo cada vez mais utilizado para aferir a qualidade das avaliações com itens de respostas construídas, por permitir a inclusão de outras variáveis aos sistemas avaliativos, além da capacidade dos indivíduos e da dificuldade das tarefas. Algumas dessas variáveis consistem em importantes fontes geradoras de vieses nos processos avaliativos. Como exemplos têm-se as características pessoais dos avaliadores, as diferenças entre a severidade dos avaliadores, as tendências dos avaliadores em julgamentos sistemáticos, as diferenças entre as dificuldades de tarefas distintas e a variação quanto ao entendimento e utilização das categorias da escala de classificação por parte dos avaliadores. O modelo multifacetas de Rasch permite a inclusão de cada variável que pode interferir na avaliação, além de possibilitar análises para os efeitos causados por cada elemento que faz parte da avaliação individualmente, o que torna a utilização desse modelo muito vantajosa. O objetivo deste estudo é estabelecer como o modelo multifacetas de Rasch pode contribuir para a determinação da qualidade das avaliações com itens de respostas construídas. A abordagem utilizada pelo modelo multifacetas de Rasch proporciona análises sobre a qualidade das medidas relacionadas aos examinandos, aos avaliadores, às tarefas, aos itens e às escalas de classificação utilizadas para a pontuação das tarefas. Este trabalho também apresenta uma aplicação do modelo multifacetas de Rasch aos dados provenientes de uma avaliação real, na qual estabelece as principais análises sobre a qualidade dessa avaliação.Abstract : This thesis presents a study about the large-scale construct-response item evaluations in the context of the many-facet Rasch model (LINACRE, 1989 apud LINACRE 1994). These evaluations require raters in order to judge the performance of the people regarding the ability that is being measured through test. Among the evaluations with constructed-responses items most frequently used in the educational and hiring ambit are those with open questions of the disciplines of the High School, the writing test of the Brazilian High School National Exam and of the university entrance exams and the tests with open questions of contests. The results of the construct-response item evaluations do not depend only on the ability level of the examinants regarding the evaluated construct and the difficulty of the tasks; they depend also on the severity of the raters that judge the performance and the structure of the classification scale. One of the main problems of these evaluations is the rating of a same performance with different severity degrees. When there are many raters, it would be the ideal if all would give exactly the same rating for the same performances observed, this is the main condition in order to have reliability of rating. However, many are the factors that can cause variability in these ratings. The many-facet Rach model have been even more used to check the quality of the construct-response item evaluations, since it allows the inclusion of other variables to the evaluating systems, besides the capabilities of the individuals and the difficulty of the tasks. Some of these variables consists of important sources generator of biases in the evaluating processes. As examples are the personal characteristics of the raters, the differences between the severity of the raters, the tendencies of the raters in systematic judgements, the differences between the difficulties of the distinct tasks and the variation regarding the understanding and use of the categories of the classification scale by the raters. The many-facet Rach model allows the inclusion of each variable that can interfere in the evaluation besides allowing analyzes for the effects caused by each element that is individually part of the evaluation, which makes the use of the many-facet Rach model very advantageous. The objective of this study is to establish how the many-facet Rach model can contribute to the determination of the quality of the evaluations with construct-response items. The approach used by the many-facet Rach model provides analyzes on the quality of the measure related to the examinees, to the the raters, to the tasks, to the questions and to the classification scales used for the rating of the tasks. This work also presents an application of multi-faceted Rasch model to data from a real assessment, which establishes the main analyzes of the quality of the evaluation
Developing a health state classification system from NEWQOL for epilepsy using classical psychometric techniques and Rasch analysis: a technical report
No full textAims: Resource allocation amongst competing health care interventions is informed by evidence of both clinical- and cost-effectiveness. Cost-utility analysis is increasingly used to assess cost effectiveness through the use of Quality Adjusted Life Years (QALYs). This requires health state values. Generic measures of health related quality of life (HRQL) are usually used to produce these values, but there are concerns about their relevance and sensitivity in epilepsy. This study develops a health state classification system for epilepsy from the NEWQOL battery, a validated questionnaire measuring QoL in epilepsy. The classification system will be amenable to valuation for calculating QALYs. Methods: Factor and other psychometric analyses were undertaken to investigate the factor structure of the battery, and assess the validity and responsiveness of the items. These analyses were used alongside Rasch analysis to select the dimensions included in the classification system, and the items used to represent each domain. Analysis was carried out on a trial dataset of patients with epilepsy (n=1611). Rasch and factor analysis were performed on one half of the sample and validated on the remaining half. Dimensions and items were selected that performed well across all analyses. Results: The battery was found to demonstrate reliability and validity but responsiveness across time periods for many of the items was low. A six dimension classification system was developed: worry about seizures, depression, memory, cognition, stigmatism and control, each with four response levels. Conclusions: It is feasible to develop a health state classification system from a battery of instruments using a combination of classical psychometric, factor and Rasch analysis. This is the first condition-specific health state classification developed for epilepsy and the next stage will produce preference weights to enable the measure to be used in cost-utility analysis.quality adjusted life years; health related quality of life; Rasch analysis; preference-based measures of health; health states; epilepsy
- …
