1,720,992 research outputs found
A Model Comparison between Neural Architectures of Human Bilingual Sentence Processing
Get PDFThis work investigates phenomena related to human bilingual sentence processing in neural language models. We ask ourselves the question if and how the emergence of these phenomena depends on the model architecture. For this purpose, we train SRNs, LSTMs, and Transformers with different hidden layer sizes as bilingual- and monolingual language models. We test these models on three phenomena that have been shown to emerge in at least one of the architectures in the literature. We refer to them as reading time prediction; an agreement between monolingual vs. bilingual reading with models trained on monolingual vs. bilingual data, the cognate facilitation effect; a faster processing of form and meaning-similar words, and the grammaticality illusion; a preference for the ungrammatical version of a certain class of sentences that is reversed in some languages.
Surprisingly, we found reading time prediction to depend not only on architecture and layer size, but also on the specific random initialization. Failure of reproduction of the effect was confirmed by the author, suggesting the original study to be due to coincidence. As for the cognate facilitation effect, we found it to be present in the SRN and LSTM, providing further evidence for its emergence in humans to be due to the cumulative frequency of cognates. The effect was found to decrease in magnitude for large layer sizes in the LSTM, which can be linked to the LSTM relying less on corpus frequency. However surprisingly, the effect was found to increase for small layer sizes in the SRN. We do not have an adequate explanation for this trend. Furthermore, it was not found in the Transformer, suggesting that Transformers exhibit less cross-linguistic transfer than the other architectures. The grammaticality illusion was found to be present in the SRN, but not in the LSTM and Transformer. This provides further evidence for the effect to arise as a result of short-distance language statistics rather than universal working memory constraints. The effect was found to stay fairly constant over layer size, syntactic linguistic transfer to be small. Furthermore, the Transformer displayed a consistent preference for grammatical sentences, suggesting super-human syntactic proficiency on this particular task
Learning the easy way: the role of form similarity in language learning and processing
Get PDFA key question about bilingual lexical access is whether lexical representations are
activated selectively (within one language) or non-selectively (across languages). A
great deal of the research addressing this question has focused on cognates, translation
equivalents with the same or similar forms across languages. These studies show that,
in non-native language (L2) processing, cognates are usually recognised and produced
faster than non-cognates, suggesting that the form overlap between translation equivalents
contributes to a processing advantage. Traditionally, the advantage is assumed
to demonstrate on-line non-selective activation: For cognates, lexical activation stems
from two sources rather than one, as is the case with non-cognates. This converging
activation leads to their facilitated processing. However, some researchers argue that
cognate facilitation could be due to differences in how cognates and non-cognates are
learned, which leads to qualitatively different representations for cognates.
In this thesis, I focus on learning-based explanations of cognate effects and ask
to what extent these can account for cognate facilitation. First, I review findings of
cognate effects in both comprehension and production, evaluating whether they are more
consistent with learning-based or on-line accounts. Second, I explore whether neural
language models trained on two languages exhibit cognate facilitation and use this to
test learning-based hypotheses of the effect. Following this, I present two behavioural
experiments investigating cognate effects in human non-native speakers. The first
investigates how the language of instruction affects cognate facilitation in trilinguals to
examine whether cognate effects only occur between languages involved in learning.
The second tests whether bilinguals exhibit cognate effects in L2 prediction to shed
light on whether predictive processing is language selective. Taken together, this thesis
provides evidence that cognate facilitation can, in principle, be explained by learning. I
argue that while non-selective activation may occur during on-line processing, it is not
required to explain cognate effects. I call for a more nuanced approach to theories of
bilingual lexical access that allow for a more flexible role for language selectivity
Learning constructions from bilingual exposure: Computational studies of argument structure acquisition
Get PDFRecommended from our members
Analyzing and modeling free word associations
Get PDFHuman free association (FA) norms are believed to reflect the strength of links between words in the lexicon of an average speaker. Large-scale FA norms are commonly used as a data source both in psycholinguistics and in computational modeling. However, few studies aim to analyze FA norms themselves, and it is not known what are the most important factors that guide speakers’ lexical choices in the FA task. Here, we first provide a statistical analysis of a large-scale data set of English FA norms. Second, we argue that such analysis can inform existing computational models of semantic memory, and present a case study with the topic model to support this claim. Based on our analysis, we provide the topic model with dictionary-based knowledge about word synonymy/antonymy, and demonstrate that the resulting model predicts human FA responses better than the topic model without this information
Recommended from our members
Modeling Sentence Processing Effects in Bilingual Speakers: A Comparison of Neural Architectures
Get PDFNeural language models are commonly used to study language processing in human speakers, and several studies trained such models on two languages to simulate bilingual speakers. Surprisingly, no work systematically evaluates different neural architectures on bilingual speakers’ data, despite the abundance of such studies in the monolingual domain. In this work, we take the first step in this direction. We train three neural architectures (SRN, LSTM, and Transformer) on Dutch and English data and evaluate them on two data sets from experimental studies. Our goal is to investigate which architectures can reproduce the cognate facilitation effect and grammaticality illusion observed in bilingual speakers. While all three architectures can correctly predict the cognate effect, only the SRN succeeds at the grammaticality illusion. We additionally show how the observed patterns change as a function of the models’ hidden layer size, a hyperparameter that we argue may be more important in bilingual models
Going Beyond Counting First Authors in Author Co-citation Analysis
Get PDFThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Variations on the Author
Get PDF“Variations on the Author” discusses two of Eduardo Coutinho’s recent films (Um Dia na Vida, from 2010, and Últimas Conversas, posthumously released in 2015) and their contribution to the general question of documentary authorship. The director’s filmography is characterized by a consistent yet self-effacing form of authorial self-inscription: Coutinho often features as an interviewer that rather than express opinions propels discourses; an interviewer that is good at listening. This mode of self-inscription characterizes him as an author who is not expressive but who is nonetheless markedly present on the screen. In Um Dia na Vida, however, Coutinho is completely absent form the image, while Últimas Conversas, on the contrary, includes a confessional prologue that moves the director from the margins to the center of his films. This article examines the ways in which these works stand out in the filmography of a director who offers new insights into the notion of cinematic authorship
Appropriate Similarity Measures for Author Cocitation Analysis
Get PDFWe provide a number of new insights into the methodological discussion about author cocitation analysis. We first argue that the use of the Pearson correlation for measuring the similarity between authors’ cocitation profiles is not very satisfactory. We then discuss what kind of similarity measures may be used as an alternative to the Pearson correlation. We consider three similarity measures in particular. One is the well-known cosine. The other two similarity measures have not been used before in the bibliometric literature. Finally, we show by means of an example that our findings have a high practical relevance.information science;Pearson correlation;cosine;similarity measure;author cocitation analysis
- …
