1,721,023 research outputs found
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Full text link“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Full text linkWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
Dispelling the Myths Behind First-author Citation Counts
Full text linkWe conducted a full-scale evaluative citation analysis study of scholars in the XML research field to explore just how different from each other author rankings resulting from different citation counting methods actually are, and to demonstrate the capability of emerging data and tools on the Web in supporting more realistic citation counting methods. Our results contest some common arguments for the continued
use of first-author citation counts in the evaluation of scholars, such as high correlations between author rankings by first-author citation counts and other citation
counting methods, and high costs of using more realistic citation counting methods that are not well-supported by the ISI databases. It is argued that increasingly available digital full text research papers make it possible for citation analysis studies to go beyond what the ISI databases have directly supported and to employ more
sophisticated methods
An Examination of Parameter Recovery Using Different Multiple Matrix Booklet Designs
Full text linkEducational large-scale assessments examine students’ achievement in various content
domains and thus provide key findings to inform educational research and evidence-based
educational policies. To this end, large-scale assessments involve hundreds of items to test
students’ achievement in various content domains. Administering all these items to single
students will over-burden them, reduce participation rates, and consume too much time and
resources. Hence multiple matrix sampling is used in which the test items are distributed into
various test forms called “booklets”; and each student administered a booklet, containing a
subset of items that can sensibly be answered during the allotted test timeframe. However,
there are numerous possibilities as to how these booklets can be designed, and this manner of booklet design could influence parameter recovery precision both at global and subpopulation levels. One popular booklet design with many desirable characteristics is the
Balanced Incomplete 7-Block or Youden squares design. Extensions of this booklet design
are used in many large-scale assessments like TIMSS and PISA. This doctoral project
examines the degree to which item and population parameters are recovered in real and
simulated data in relation to matrix sparseness, when using various balanced incomplete
block booklet designs. To this end, key factors (e.g., number of items, number of persons,
number of items per person, and the match between the distributions of item and person
parameters) are experimentally manipulated to learn how these factors affect the precision
with which these designs recover true population parameters. In doing so, the project expands
the empirical knowledge base on the statistical properties of booklet designs, which in turn
could help improve the design of future large-scale studies.
Generally, the results show that for a typical large-scale assessment (with a sample size of at
least 3,000 students and more than 100 test items), population and item parameters are recovered accurately and without bias in the various multi-matrix booklet designs. This is
true both at the global population level and at the subgroup or sub-population levels. Further,
for such a large-scale assessment, the match between the distribution of person abilities and
the distribution of item difficulties is found to have an insignificant effect on the precision
with which person and item parameters are recovered, when using these multi-matrix booklet
designs.
These results give further support to the use of multi-matrix booklet designs as a reliable test
abridgment technique in large-scale assessments, and for accurate measurement of
performance gaps between policy-relevant subgroups within populations. However, item position effects were not fully considered, and different results are possible if similar studies
are performed (a) with conditions involving items that poorly measure student abilities (e.g.,
with students having skewed ability distributions); or, (b) simulating conditions where there
is a lot of missing data because of non-response, instead of just missing by design. This
should be further investigated in future studies.Die Erfassung des Leistungsstands von Schülerinnen und Schülern in verschiedenen
Domänen durch groß angelegte Schulleistungsstudien (sog. Large-Scale Assessments) liefert
wichtige Erkenntnisse für die Bildungsforschung und die evidenzbasierte Bildungspolitik.
Jedoch erfordert die Leistungstestung in vielen Themenbereichen auch immer den Einsatz
hunderter Items. Würden alle Testaufgaben jeder einzelnen Schülerin bzw. jedem einzelnen
Schüler vorgelegt werden, würde dies eine zu große Belastung für die Schülerinnen und
Schüler darstellen und folglich wären diese auch weniger motiviert, alle Aufgaben zu
bearbeiten. Zudem wäre der Einsatz aller Aufgaben in der gesamten Stichprobe sehr zeit- und
ressourcenintensiv. Aus diesen Gründen wird in Large-Scale Assessments oft auf ein Multi-
Matrix Design zurückgegriffen bei dem verschiedene, den Testpersonen zufällig zugeordnete,
Testheftversionen (sog. Booklets) zum Einsatz kommen. Diese enthalten nicht alle Aufgaben,
sondern lediglich eine Teilmenge des Aufgabenpools, wobei nur ein Teil der Items zwischen
den verschiedenen Booklets überlappt. Somit wird sichergestellt, dass die Schülerinnen und
Schüler alle ihnen vorgelegten Items in der vorgegebenen Testzeit bearbeiten können. Jedoch
gibt es zahlreiche Varianten wie diese Booklets zusammengestellt werden können. Das
jeweilige Booklet Design hat wiederum Auswirkungen auf die Genauigkeit der
Parameterschätzung auf Populations- und Teilpopulationsebene. Ein bewährtes Booklet
Design ist das Balanced-Incomplete-7-Block Design, auch Youden-Squares Design genannt,
das in unterschiedlicher Form in vielen Large-Scale Assessments, wie z.B. TIMSS und PISA,
Anwendung findet. Die vorliegende Arbeit untersucht sowohl auf Basis realer als auch
simulierter Daten die Genauigkeit mit der Item- und Personenparameter unter Anwendung
verschiedener Balanced-Incomplete-Block Designs und in Abhängigkeit vom Anteil
designbedingt fehlender Werte geschätzt werden können. Dafür wurden verschiede
Designparameter variiert (z.B. Itemanzahl, Stichprobenumfang, Itemanzahl pro Booklet,
Ausmaß der Passung von Item- und Personenparametern) und anschließend analysiert, in
welcher Weise diese die Genauigkeit der Schätzung von Populationsparametern beeinflussen. Die vorliegende Arbeit hat somit zum Ziel, das empirische Wissen um die statistischen Eigenschaften von Booklet Designs zu erweitern, wodurch ein Beitrag zur Verbesserung zukünftiger Large-Scale Assessments geleistet wird.
Die Ergebnisse der vorliegenden Arbeit zeigten, dass für ein typisches Large-Scale
Assessment (mit einer Stichprobengröße von mindestens 3000 Schülerinnen und Schülern
und mindestens 100 Items) die Personen- und Itemparameter sowohl auf Populations- als
auch auf Teilpopulationsebene mit allen eingesetzten Varianten des Balanced-Incomplete-
Block Designs präzise geschätzt wurden. Außerdem konnte gezeigt werden, dass für
Stichproben mit mindestens 3000 Schülerinnen und Schülern die Passung zwischen der
Leistungsverteilung und der Verteilung der Aufgabenschwierigkeit keinen bedeutsamen
Einfluss auf die Genauigkeit hatte, mit der verschiedene Booklet Designs Personen- und
Itemparameter schätzten.
Die Ergebnisse untermauern, dass unter Verwendung von multi-matrix Designs
bildungspolitisch relevante Leistungsunterschiede zwischen Gruppen von Schülerinnen und
Schülern in der Population reliabel und präzise geschätzt werden können. Eine
Einschränkung der vorliegenden Studie liegt darin, dass Itempositionseffekte nicht umfassend
berücksichtigt wurden. So kann nicht ausgeschlossen werden, dass die Ergebnisse abweichen würden, wenn (a) Items verwendet werden würden, welche die Leistung der Schülerinnen und Schüler schlecht schätzen (z.B. bei einer schiefen Verteilungen der Leistungswerte) oder (b) hohe Anteile an fehlenden Werten vorliegen, die nicht durch das Multi-Matrix Design erzeugt wurden. Dies sollte in zukünftigen Studien untersucht werden
Modeling Competence Data in Large-Scale Educational Assessments
No full textBamberg, Univ., kumulative Diss., 2015Over the last decades, large-scale assessments focusing on skills and knowledge of individuals have expanded significantly in order to obtain information about conditions for and consequences of competence acquisition. To provide valid and accurate scores of the subjects under investigation, it is necessary to thoroughly check the psychometric properties of the competence instruments that are administered and to choose appropriate scaling models for the competence data. In this thesis, various challenges in modeling competence data were addressed that arose from different recently developed competence tests in large-scale assessments. The different tests posed specific demands on the scaling of the data such as dealing with multidimensionality, incorporating different response formats, or linking competence scores. By investigating these challenges associated with each of the competence tests, the aim of the thesis was to draw implications for the specification of the scaling models for the competence data.
First, a new metacognitive knowledge test for early elementary school children was investigated. As earlier findings on metacognitive knowledge in secondary school pointed to empirically distinguishable components of metacognitive knowledge, especially the dimensionality of the newly developed test was studied. Therefore, uni- and multidimensional models were applied to the competence data and their model fit was compared. By applying multidimensional latent-change models, the homogeneity of change was observed as further indicator for dimensionality. Overall, the new test instrument exhibited good psychometric properties including fairness of the items for various subgroups. In accordance with previous studies in other age groups the results indicated a multidimensional structure of the newly developed test instrument. In the discussion, theoretical as well as empirical arguments were compiled that should be considered for the choice of a uni- or a multidimensional model for the metacognitive knowledge data.
The next objective in the thesis was to study a series of reading competence tests intended to measure the same latent trait across a large age span. The different reading competence tests, developed in a longitudinal large-scale study, were based on the same conceptual framework and were administered from fifth grade to adulthood. We specifically investigated whether the test scores were comparable across such a large age span enabling to interpret change across time. The analyses on the reading competence tests showed that the coherence of measurement could not fully be assured across the wide age range. The application of strict linking models allowing for the interpretation of developmental progress seemed to be justified within secondary school, but not between secondary school and adulthood.
The last purpose in the thesis was to find out how to adequately incorporate different response formats in a scaling model. Therefore, multiple choice (MC) and complex multiple choice (CMC) items were regarded as they are most frequently used in large-scale assessments. Specifically, we explored whether the two response formats form distinct empirical dimensions and which a priori scoring schemes for the two response formats appropriately model the competence data. The results demonstrated that the response formats built a unidimensional measure across domains, studies, and age cohorts justifying to use a unidimensional scale score. A differentiated scoring of the CMC items yielded a better discrimination between persons and was, thus, preferred. The a priori weighting scheme of giving each subtask of a CMC item half the weight of a MC item described the empirical competence data well
Using Response Times for Modeling Missing Responses in Large-Scale Assessments
Full text linkExaminees differ in how they interact with assessments. In low-stakes large-scale assessments (LSAs), missing responses pose an obvious example of such differences. Understanding the underlying mechanisms is paramount for making appropriate decisions on how to deal with missing responses in data analysis and drawing valid inferences on examinee competencies. Against this background, the present work aims at providing approaches for a nuanced modeling and understanding of test-taking behavior associated with the occurrence of missing responses in LSAs. These approaches are aimed at a) improving the treatment of missing responses in LSAs, b) supporting a better understanding of missingness mechanisms in particular and examinee test-taking behavior in general, and c) considering differences in test-taking behavior underlying missing responses when drawing inferences about examinee competencies. To that end, the present work leverages the additional information contained in response times and integrates research on modeling missing responses with research on modeling response times associated with observed responses. By documenting lengths of interactions, response times contain valuable information on how examinees interact with assessments and may as such critically contribute to understanding the processes underlying both observed and missing responses.
This work presents four modeling approaches that focus on different aspects and mechanisms of missing responses. The first two approaches focus on modeling not-reached items. The second two approaches aim at modeling omitted items.
The first approach employs the framework for the joint modeling of speed and ability by van der Linden (2007) for modeling the mechanism underlying not-reached items due to lack of working speed. On the basis of both theoretical considerations as well as a comprehensive simulation study, it is argued that by accounting for differences in speed this framework is well suited for modeling the mechanism underlying not-reached items due to lack thereof. In assessing empirical test-level response times, it is, however, also illustrated that some examinees quit the assessment before reaching the end of the test or being forced to stop working due to a time limit.
Building on these results, the second approach of this work aims at disentangling and jointly modeling multiple mechanisms underlying not-reached items. Employing information on response times, not-reached items due to lack of speed are distinguished from not-reached items due to quitting. The former is modeled by considering examinee speed. Quitting behavior - defined as stopping to work before the time limit is reached while there are still unanswered items - is modeled as a survival process, with the item position at which examinees are most likely to quit being governed by their test endurance, conceptualized as a third latent variable besides speed and ability.
The third approach presented in this work focuses on jointly modeling omission behavior and response behavior, thus providing a better understanding of how these two types of behavior differ. For doing so, the approach extends the framework for jointly modeling speed and ability by a model component for the omission process and introduces the concept of different speed levels examinees operate on when generating responses and omitting items. This approach supports a more nuanced understanding of both the missingness mechanism underlying omissions and examinee pacing behavior through assessment of whether examinees employ different pacing strategies when generating responses or omitting items
The fourth approach builds on previous theoretical work relating omitted responses to examinee disengagement and provides a model-based approach that allows for identifying and modeling examinee disengagement in terms of both omission and guessing behavior. Disengagement is identified at the item-by-examinee level by employing a mixture modeling approach that allows for different data-generating processes underlying item responses and omissions as well as different distributions of response times associated with engaged and disengaged behavior. Item-by-examinee mixing proportions themselves are modeled as a function of additional person and item parameters. This allows relating disengagement to ability and speed as well as identifying items that are likely to evoke disengaged test-taking behavior.
The approaches presented in this work are tested and illustrated by a) evaluating their statistical performance under conditions typically encountered in LSAs by means of comprehensive simulation studies, b) illustrating their advances over previously developed approaches, and c) applying them to real data from major LSAs, thereby illustrating their potential for understanding examinee test-taking behavior in general and missingness mechanisms in particular. The potential of the approaches developed in this work for deepening the understanding of results from LSAs is discussed and implications for the improvement of assessment procedures - ranging from construction and administration to analysis, interpretation and reporting - are derived. Limitations of the proposed approaches are discussed and suggestions for future research are provided
Students’ self-directed use of technology in language learning out of classroom in Chinese secondary context
Full text linkWith the explosion of technological language learning resources, there is a need to provide a guide for students, educators, and providers of learning resources on how to effectively choose, recommend and design those available resources. The guide should cover the current situation regarding students' self-directed use of available technology-based resources for learning, the factors that influence their choices, and the learning outcomes that result from their different choices. The current study presented such a guide on these three aspects. I was inspired by a previous study a classification framework in technological learning experiences (Lai et al., 2017) to investigate students’ actual use of technology in language learning beyond classroom. Thus, I partially replicated their methodology and provided more solid theoretical support, and referred to all the other literatures behind, adapting their three-type classification (1. instruction-oriented; 2. entertainment-and information-oriented; 3. social-oriented) of students' self-directed technological language learning experiences to a new context - Chinese secondary school students.
The study adopted exploratory sequential mixed methods and began with interviews with 15 students, followed by a survey (n=429) designed from interview results. The classification framework derived from the interviews undergone some changes in the factors after exploratory factor analysis (EFA). Taking into account the findings of the qualitative study as well as the EFA, I discussed the potential reasons for the changes and tentatively identified a four-category classification framework to differentiate various types of students’ engagement in their self-directed language learning with technology beyond classroom. The difference of the four-type classification from the previous three-type one was that the second category was divided into two types in current research (1. Instruction-oriented; 2. Entertainment-oriented; 3. Information-oriented; 4. Social-oriented). This division was approved to be meaningful, as influencing factors and the learning output of these two types were approved to varied. This reliable and valid classification framework in current study was approved to be replicable into other contexts. Moreover, among four types, information-oriented technological leaning experiences was the only one that significantly predict learning outcomes. Thus, the results encourage students to utilize both affective and cognitive strategies in their self-directed language learning with technology, provided guidance to educators and educational product provides to choose or design engaging and authentic learning materials to best facilitate students in achieving better learning outcomes.Angesichts der explosionsartigen Zunahme technologischer Ressourcen für das Sprachenlernen besteht die Notwendigkeit, einen Leitfaden für Schüler, Lehrkräfte und Anbieter von Lernressourcen zu erstellen, der aufzeigt, wie diese verfügbaren Ressourcen effektiv ausgewählt, empfohlen und gestaltet werden können. Der Leitfaden sollte die aktuelle Situation in Bezug auf die selbstgesteuerte Nutzung verfügbarer technologiebasierter Lernressourcen durch die Schüler, die Faktoren, die ihre Wahl beeinflussen, und die Lernergebnisse, die sich aus ihren verschiedenen Entscheidungen ergeben, abdecken. In der vorliegenden Studie wurde ein solcher Leitfaden zu diesen drei Aspekten vorgestellt. Ich habe mich von einer früheren Studie inspirieren lassen, in der ein Klassifizierungsrahmen für technologische Lernerfahrungen (Lai et al., 2017) entwickelt wurde, um die tatsächliche Nutzung von Technologie durch Studierende beim Sprachenlernen außerhalb des Klassenzimmers zu untersuchen. Daher habe ich ihre Methodik teilweise übernommen, eine solidere theoretische Grundlage geschaffen und mich auf die gesamte übrige Literatur bezogen, indem ich ihre Drei-Typen-Klassifizierung (1. unterrichtsorientiert; 2. unterhaltungs- und informationsorientiert; 3. sozialorientiert) der selbstgesteuerten technologischen Sprachlernerfahrungen von Schülern an einen neuen Kontext - chinesische Sekundarschüler - angepasst habe.
Die Studie verwendete explorative, sequenzielle, gemischte Methoden und begann mit Interviews mit 15 Schülern, gefolgt von einer Umfrage (n=429), die aus den Interviewergebnissen entwickelt wurde. Der aus den Interviews abgeleitete Klassifizierungsrahmen erfuhr nach der explorativen Faktorenanalyse (EFA) einige Änderungen in den Faktoren. Unter Berücksichtigung der Ergebnisse der qualitativen Studie sowie der EFA diskutierte ich die möglichen Gründe für die Veränderungen und ermittelte vorläufig einen Klassifizierungsrahmen mit vier Kategorien, um verschiedene Arten des Engagements der Studierenden beim selbstgesteuerten Sprachenlernen mit Technologie außerhalb des Klassenzimmers zu unterscheiden. Der Unterschied zwischen der Vier-Kategorien-Klassifizierung und der früheren Drei-Kategorien-Klassifizierung bestand darin, dass die zweite Kategorie in der aktuellen Forschung in zwei Typen unterteilt wurde (1. unterrichtsorientiert; 2. unterhaltungsorientiert; 3. informationsorientiert; 4. sozialorientiert). Diese Unterteilung wurde als sinnvoll erachtet, da die Einflussfaktoren und der Lernoutput dieser beiden Typen als unterschiedlich eingestuft wurden. Dieser verlässliche und gültige Klassifizierungsrahmen wurde in der aktuellen Studie als übertragbar auf andere Kontexte befunden. Darüber hinaus war unter den vier Typen die informationsorientierte technologische Lernerfahrung der einzige, der die Lernergebnisse signifikant vorhersagte. Die Ergebnisse ermutigen die Studierenden, sowohl affektive als auch kognitive Strategien beim selbstgesteuerten Sprachenlernen mit Technologie zu nutzen, und geben Pädagogen und Anbietern von Bildungsprodukten eine Orientierungshilfe bei der Auswahl oder Gestaltung von ansprechendem und authentischem Lernmaterial, um den Studierenden bessere Lernergebnisse zu ermöglichen
- …
