227 research outputs found
Quantifying Bimodality Part 2: A Likelihood Ratio Test for the Comparison of a Unimodal Normal Distribution and a Bimodal Mixture of Two Normal Distributions. Bruno D. Zumbo is
No full textScientists in a variety of fields are often faced with the question of whether a sample is best described as unimodal or bimodal. In an earlier paper (Frankland & Zumbo, 2002), a simple and convenient method for assessing bimodality was described. That method is extended by developing and demonstrating a likelihood ratio test (LRT) for bimodality for the comparison of a unimodal normal distribution and a bimodal mixture of two normal distributions. As in Frankland and Zumbo (2002), the LRT approach is demonstrated using algorithms in SPSS
Reliability, power, functions, and relations: A reply to Humphreys
No full textIndex terms: difference scores, error of measurement,
power, significance tests, t test, test reliability, true scores.Zimmerman, Donald W.; Williams, Richard H.; Zumbo, Bruno D.. (1993). Reliability, power, functions, and relations: A reply to Humphreys. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/116224
Propensity_DIF_writing_Appendix_0822 – Supplemental material for A Propensity Score Method for Investigating Differential Item Functioning in Performance Assessment
No full textSupplemental material, Propensity_DIF_writing_Appendix_0822 for A Propensity Score Method for Investigating Differential Item Functioning in Performance Assessment by Michelle Y. Chen, Yan Liu and Bruno D. Zumbo in Educational and Psychological Measurement</p
Development of a scale of Sense of Community in university online courses
No full textCreating a sense of community in online classes contributes to student retention and to their overall satisfaction with the course
itself. This study aimed to develop a scale of sense of community of students attending online university courses. A series of ordinal
exploratory factor analyses were conducted on data obtained from 839 students enrolled in Italian universities. Using an item analysis
method, we were able to select the 36 most valid items from an original set of 60 items we had previously defined. These items are
distributed across three related factors measuring membership, influence, and fulfillment of needs. This factorial structure replicates
the McMillan and Chavis’s model of sense of community, upon the basis of which this scale was developed. The three
factors presented good ordinal alpha and adequate convergent/divergent validity coefficients. The scale represents an efficient
tool for the design, monitoring, and evaluation of online courses
Reliability of measurement and power of significance tests based on differences
No full textThe power of significance tests based on difference
scores is indirectly influenced by the reliability
of the measures from which differences are obtained.
Reliability depends on the relative magnitude of
true score and error score variance, but statistical
power is a function of the absolute magnitude of
these components. Explicit power calculations
reaffirm the paradox put forward by Overall &
Woodward (1975, 1976)-that significance tests of
differences can be powerful even if the reliability
of the difference scores is 0. This anomaly arises
because power is a function of observed score
variance but is not a function of reliability unless
either true score variance or error score variance is
constant. Provided that sample size, significance
level, directionality, and the alternative hypothesis
associated with a significance test remain the same,
power always increases when population variance
decreases, independently of reliability. Index
terms: difference scores, error of measurement, power,
significance tests, t test, test reliability, true scores.Zimmerman, Donald W.; Williams, Richard H.; Zumbo, Bruno D.. (1993). Reliability of measurement and power of significance tests based on differences. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/116219
Differential item functioning of the center for epidemiologic studies depression scale among Chinese adolescents
No full textThis study examined differential item functioning (DIF) in the Center for Epidemiologic Studies Depression Scale (CES-D) between Chinese and White adolescents (aged 13 to 17 years) living in Canada. A series of ordinal logistic regressions were used to test for uniform and non-uniform DIF on items in the CES-D. The DIF analyses identified non-uniform DIF for Item 7 (“I felt that everything I did was an effort”). Controlling for gender and strength of ethnic identity in the DIF analyses did not alter the DIF results. The results of this study suggest that CES-D Item 7 does not appear to discriminate at higher levels of depression in Chinese adolescents. The results of this study hold notable implications for the use of the CES-D given that Chinese adolescents demonstrated a systematic difference in expression/experience of depression.Peer reviewedFinal article published.DC Author's celebration 2022differential item functioningdepressionethnic identityThe center for epidemiologic studies depression scale (CES-D)chinese adolescen
Process and product in computer-based assessments: Clearing the ground for a holistic validity framework
No full textThere is no consensus among assessment researchers about many of the central problems of response process data, including what is it and what is it comprised of. The Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014) locate process data within their five sources of validity evidence. However, we rarely see a conceptualization of response processes; rather, the focus is on the techniques and methods of assembling response process indices or statistical models. The method often overrides clear definitions, and, as a field, we may therefore conflate method and methodology – much like we have conflated validity and validation (Zumbo, 2007). In this paper, we aim to clear the conceptual ground to explore the scope of a holistic framework for the validation of process and product. We review prominent conceptualizations of response processes and their sources and explore some fundamental questions: Should we make a theoretical and practical distinction between response processes and response data? To what extent do the uses of process data reflect the principles of deliberate, educational, and psychological measurement? To answer these questions, we consider the case of item response times and the potential for variation associated with disability and neurodiversity
How does the uncaptured uniqueness of survey respondents impact the analysis of group differences?
No full textSurvey-driven research frequently relies on the analysis of differences between subpopulations that share social identities, such as gender or ethnicity. One method of gathering identity information from research participants is the identity item, which I define as a closed-form, single response survey item that prompts a respondent to endorse an option that corresponds to the identity group they belong to. A potential concern for researchers using these items is “identity item mismatch,” where a respondent’s social identity does not fully conform to any of the options available on an identity item. Here, I examine whether this mismatch between the social identities of respondents and the response options on an identity item can bias the multiple regression analysis of survey response data. This thesis relies upon established literature in the cognitive aspects of survey methodology and social identity theory to first build a working theory of how identity item mismatch is produced through an interaction between respondents, the social identities of the respondents, and identity items. This theory then motivates a Monte Carlo simulation of a survey-driven research study, which uses a two-stage beta-Bernoulli event to characterise the response process necessary to answer a gender identity item. In two studies, I use this simulation to demonstrate that an increase in identity item mismatch behaviour biases estimates of population-level differences between identity groups. This bias also emerges when certain social identities within a sample of respondents are not listed as a response category on an identity item. Ultimately, this research illustrates how the data generated through identity items could impact the quality of the inferences generated from survey responses, which motivates the development of revised or alternative survey methods for the capturing of social identity.Education, Faculty ofEducational and Counselling Psychology, and Special Education (ECPS), Department ofGraduat
- …
