153 research outputs found
Linzen, Kasyanenko and Gouskova 2013 audio files, corpus and experimental results, code
No full textNational Science Foundation BCS-122465
Disentangling dialects: a neural approach to Indo-Aryan historical phonology and subgrouping
No full textThis paper seeks to uncover patterns of sound change across Indo-Aryan languages using an LSTM encoder-decoder architecture. We augment our models with embeddings representing language ID, part of speech, and other features such as word embeddings. We find that a highly augmented model shows highest accuracy in predicting held-out forms, and investigate other properties of interest learned by our models’ representations. We outline extensions to this architecture that can better capture variation in Indo-Aryan sound change
Identifying robust markers of Parkinson’s disease in typing behaviour using a CNN-LSTM network
No full textThere is urgent need for non-intrusive tests that can detect early signs of Parkinson's disease (PD), a debilitating neurodegenerative disorder that affects motor control. Recent promising research has focused on disease markers evident in the fine-motor behaviour of typing. Most work to date has focused solely on the timing of keypresses without reference to the linguistic content. In this paper we argue that the identity of the key combinations being produced should impact how they are handled by people with PD, and provide evidence that natural language processing methods can thus be of help in identifying signs of disease. We test the performance of a bi-directional LSTM with convolutional features in distinguishing people with PD from age-matched controls typing in English and Span-ish, both in clinics and online.
Quantity doesn't buy quality syntax with neural language models
No full textThis repository contains the 125 LSTM models analyzed in van Schijndel, Mueller, and Linzen (2019) "Quantity doesn't buy quality syntax with neural language models". Each archive contains 25 models trained on a specific number of training tokens.
The naming convention for each model is:
LSTM_[Hidden Units]_[Training Tokens]_[Training Partition]_[Random Seed]-d[Dropout Rate].pt
Hidden Units: The number of hidden units per layer (there are two layers in each model) {100, 200, 400, 800, 1600}
Training Tokens: The number of tokens used to train each model {2m, 10m, 20m, 40m, 80m}
Training Partition: Five distinct training partitions were created for each amount of training data {a, b, c, d, e}
Random Seed: The random seed used to train each model*
Dropout Rate: All models used a dropout rate of 0.2
*A scripting bug led to a random seed of 0 for all models trained on less than 40 million tokens. This does not substantively affect the analyses since each model is distinct in terms of the model configuration or training data, so we opted to not retrain the models with unique random seeds to save time and computational resources.</p
Analysing Word Representation from the Input and Output Embeddings in Neural Network Language Models
No full textResearchers have recently demonstrated that tying the neural weights between the input look-up table and the output classification layer can improve training and lower perplexity on sequence learning tasks such as language modelling. Such a procedure is possible due to the design of the softmax classification layer, which previous work has shown to comprise a viable set of semantic representations for the model vocabulary, and these these output embeddings are known to perform well on word similarity benchmarks. In this paper, we make meaningful comparisons between the input and output embeddings and other SOTA distributional models to gain a better understanding of the types of information they represent. We also construct a new set of word embeddings using the output embeddings to create locally-optimal approximations for the intermediate representations from the language model. These locally-optimal embeddings demonstrate excellent performance across all our evaluations
Analyzing the Use of Metaphors in News Editorials for Political Framing
No full textMetaphorical language is a pivotal element in the realm of political framing. Existing work from linguistics and the social sciences provides compelling evidence regarding the distinctiveness of conceptual framing for political ideology perspectives. However, the nature and utilization of metaphors and the effect on audiences of different political ideologies within political discourses are hardly explored. To enable research in this direction, in this work we create a dataset, originally based on news editorials and labeled with their persuasive effects on liberals and conservatives and extend it with annotations pertaining to metaphorical usage of language. To that end, first, we identify all single metaphors and composite metaphors. Secondly, we provide annotations of the source and target domains for each metaphor. As a result, our corpus consists of 300 news editorials annotated with spans of texts containing metaphors and the corresponding domains of which these metaphors draw from. Our analysis shows that liberal readers are affected by metaphors, whereas conservatives are resistant to them. Both ideologies are affected differently based on the metaphor source and target category. For example, liberals are affected by metaphors in the Darkness & Light (e.g., death) source domains, where as the source domain of Nature affects conservatives more significantly
SLOG: A Structural Generalization Benchmark for Semantic Parsing
No full textThe goal of compositional generalization benchmarks is to evaluate how well models generalize to new complex linguistic expressions. Existing benchmarks often focus on lexical generalization, the interpretation of novel lexical items in syntactic structures familiar from training. Structural generalization tasks, where a model needs to interpret syntactic structures that are themselves unfamiliar from training, are often underrepresented, resulting in overly optimistic perceptions of how well models can generalize. We introduce SLOG, a semantic parsing dataset that extends COGS (Kim and Linzen, 2020) with 17 structural generalization cases. In our experiments, the generalization accuracy of Transformer models, including pretrained ones, only reaches 40.6%, while a structure-aware parser only achieves 70.8%. These results are far from the near-perfect accuracy existing models achieve on COGS, demonstrating the role of SLOG in foregrounding the large discrepancy between models' lexical and structural generalization capacities.</p
SLOG: A Structural Generalization Benchmark for Semantic Parsing
No full textThe goal of compositional generalization benchmarks is to evaluate how well
models generalize to new complex linguistic expressions. Existing benchmarks
often focus on lexical generalization, the interpretation of novel lexical
items in syntactic structures familiar from training; structural generalization
tasks, where a model needs to interpret syntactic structures that are
themselves unfamiliar from training, are often underrepresented, resulting in
overly optimistic perceptions of how well models can generalize. We introduce
SLOG, a semantic parsing dataset that extends COGS (Kim and Linzen, 2020) with
17 structural generalization cases. In our experiments, the generalization
accuracy of Transformer models, including pretrained ones, only reaches 40.6%,
while a structure-aware parser only achieves 70.8%. These results are far from
the near-perfect accuracy existing models achieve on COGS, demonstrating the
role of SLOG in foregrounding the large discrepancy between models' lexical and
structural generalization capacities.Comment: Accepted to EMNLP 202
- …
