33 research outputs found
An extensible toolkit for computational semantics
No full textIn this paper we focus on the software for computational semantics provided by the Python-based Natural Language Toolkit (nltk). The semantics modules in nltk are inspired in large part by the approach developed in Blackburn and Bos (2005) (henceforth referred to as B&B). Since Blackburn and Bos have also provided a software suite to accompany their excellent textbook, one might ask what the justification is for the nltk offering, which is similarly slanted towards teaching computational semantics
Part-of-Speech Tagging for Code-Switched, Transliterated Texts without Explicit Language Identification
No full textThe Development and Incorporation of Introductory Aerospace Curriculum into Tennessee Secondary Education Systems
No full textThis project discusses the commercial and general aviation industries and presents research demonstrating the importance of each. It then argues using modern research that the industries are greatly in need of pilots. The project then submits the solution in the form of basic introductory aerospace curriculum, titled The World of Aerospace, for secondary education systems in Tennessee. The methodology of the curriculum is discussed, followed by the curriculum itself in the appendices, along with reflections and revisions by the author after The World of Aerospace’s implementation in a Tennessee high school. While various facets of the industry are discussed and covered, the pilot sector will receive the most focus
Automatic Transcription in Colonial Contexts: OCR for the Primeros Libros
No full textThe PDF images in the Primeros Libros digital collection, an effort to produce digital facsimiles of all books printed before 1601 in the Americas, pose several challenges for Optical Character Recognition (OCR) systems. The Ocular system, designed by Taylor Berg-Kirkpatrick et al., jointly models the physical operation of hand-press printing and the language of the written document, allowing it to ‘learn’ to read early printed books. Ocular cannot, however, handle the orthographic variation and code switching prevalent in the American context. Working with PDF images of trilingual texts in Spanish, Latin, and Nahuatl, we set out to modify Ocular for use on the Primeros libros collection. In this paper, we present our OCR tool for the Primeros Libros collection, an extension of Ocular which can handle multilingual documents, and which includes an interface for the incorporation of orthographic idiosyncrasies. At the same time, we argue for a situated analysis of digitization tools which considers Ocular\u27s statistical models within the context of the Primeros Libros collection. As Walter Mignolo has shown, books from early colonial Mexico embody a larger project of language codification which was deeply embedded in the colonization and religious conversion of New Spain. The mathematical simplicity of Ocular\u27s statistical models suggests a neutral engagement with the text that disguises a deep engagement with these colonial processes. Automatic transcription in this context becomes a process with significant implications for the ideological positioning of digitization projects
Examining Modularity in Multilingual LMs via Language-Specialized Subnetworks
No full textRecent work has proposed explicitly inducing language-wise modularity in
multilingual LMs via sparse fine-tuning (SFT) on per-language subnetworks as a
means of better guiding cross-lingual sharing. In this work, we investigate (1)
the degree to which language-wise modularity naturally arises within models
with no special modularity interventions, and (2) how cross-lingual sharing and
interference differ between such models and those with explicit SFT-guided
subnetwork modularity. To quantify language specialization and cross-lingual
interaction, we use a Training Data Attribution method that estimates the
degree to which a model's predictions are influenced by in-language or
cross-language training examples. Our results show that language-specialized
subnetworks do naturally arise, and that SFT, rather than always increasing
modularity, can decrease language specialization of subnetworks in favor of
more cross-lingual sharing
How do languages influence each other? Studying cross-lingual data sharing during LM fine-tuning
No full textMultilingual large language models (MLLMs) are jointly trained on data from
many different languages such that representation of individual languages can
benefit from other languages' data. Impressive performance on zero-shot
cross-lingual transfer shows that these models are capable of exploiting data
from other languages. Yet, it remains unclear to what extent, and under which
conditions, languages rely on each other's data. In this study, we use TracIn
(Pruthi et al., 2020), a training data attribution (TDA) method, to retrieve
the most influential training samples seen during multilingual fine-tuning for
a particular test language. This allows us to analyse cross-lingual sharing
mechanisms of MLLMs from a new perspective. While previous work studied
cross-lingual sharing at the level of model parameters, we present the first
approach to study cross-lingual sharing at the data level. We find that MLLMs
rely on data from multiple languages from the early stages of fine-tuning and
that this reliance gradually increases as fine-tuning progresses. We further
study how different fine-tuning languages influence model performance on a
given test language and find that they can both reinforce and complement the
knowledge acquired from data of the test language itself
Unsupervised Code-Switching for Multilingual Historical Document Transcription
No full textTranscribing documents from the printing press era, a challenge in its own right, is more complicated when documents interleave multiple languages—a common feature of 16th century texts. Additionally, many of these documents precede consistent ortho-graphic conventions, making the task even harder. We extend the state-of-the-art his-torical OCR model of Berg-Kirkpatrick et al. (2013) to handle word-level code-switching between multiple languages. Further, we en-able our system to handle spelling variabil-ity, including now-obsolete shorthand systems used by printers. Our results show average rel-ative character error reductions of 14 % across a variety of historical texts.
Cross-Lingual Transfer with Language-Specific Subnetworks for Low-Resource Dependency Parsing
No full textLarge multilingual language models typically share their parameters across all languages, which enables cross-lingual task transfer, but learning can also be hindered when training updates from different languages are in conflict. In this article, we propose novel methods for using language-specific subnetworks, which control cross-lingual parameter sharing, to reduce conflicts and increase positive transfer during fine-tuning. We introduce dynamic subnetworks, which are jointly updated with the model, and we combine our methods with meta-learning, an established, but complementary, technique for improving cross-lingual transfer. Finally, we provide extensive analyses of how each of our methods affects the models
