539 research outputs found
Sort by
What do the eyes really see? An eye-tracking account of language processing
No full textThis experimental study aims to investigate the translation process from English to Czech in a multimodal scenario by using an eye tracker.
We investigate specific aspects of translating ambiguous and unambiguous sentences, and simultaneously, we focus on the possible impact of visual information on the translation process. Thus, we show how mechanisms of visual search, as well as the presence and attention mechanisms involved in such translation processes, can be explored based on various eye-movement data, i.e., cognitive mechanisms involved in reading original sentences and producing the corresponding translation are studied using a plethora of eye-tracking-specific metrics.
Among other things, the paper demonstrates how the Stroop effect is visible in the experimental setup
Transformer Models
No full textTransformer is a machine learning architecture introduced in 2017 that quickly became very popular, surpassing the state of the art in many areas, including language modeling, machine translation, question answering, and chatbots. This chapter describes the main components of the Transformer architecture, such as tokenization, embeddings, and self-attention, followed by the basics of training Transformer models and generating texts. It introduces some of the most well-known pre-trained language models (e.g., BERT and GPT), highlighting their impact and the rise of large language models (LLMs). It discusses how these pre-trained models can be tailored to specific tasks such as text classification, named entity recognition, text generation, and sentence embeddings for similarity analysis. The chapter introduces principles of in-context learning, prompt engineering, chain-of-thought prompting, and fine-tuning; it also discusses computational resources and available cloud platforms. Finally, it underscores the significance of responsible AI development, emphasizing the need to mitigate biases and ensure the ethical use of these powerful language models
EMMT: A simultaneous eye-tracking, 4-electrode EEG and audio corpus for multi-modal reading and translation scenarios
No full textWe present the Eyetracked Multi-Modal Translation (EMMT) corpus, a dataset containing monocular eye movement recordings, audio and 4-electrode electroencephalogram (EEG) data of 43 participants. The objective was to collect cognitive signals as responses of participants engaged in a number of language intensive tasks involving different text-image stimuli settings when translating from English to Czech.
Each participant was exposed to 32 text-image stimuli pairs and asked to (1) read the English sentence, (2) translate it into Czech, (3) consult the image, (4) translate again, either updating or repeating the previous translation. The text stimuli consisted of 200 unique sentences with 616 unique words coupled with 200 unique images as the visual stimuli.
The recordings were collected over a two week period and all the participants included in the study were Czech natives with strong English skills. Due to the nature of the tasks involved in the study and the relatively large number of participants involved, the corpus is well suited for research in Translation Process Studies, Cognitive Sciences among other disciplines
Detecting Emotive Quotes in Oral Testimonies
No full textThe engagement with archives related to narratives of personal trauma can be intimidating for non-experts due to the unsettling subject matter and the lack of clear starting points. One way to incentivize potential readers to learn more about victims' and survivors' stories is to quote, which has a long tradition of piquing interest in philosophy, literature, news, and pop culture.
While these media are crafted for memorability, in this paper, we investigate whether this "quoting language" also emerges in unscripted speech and can be automatically extracted to enrich the presentation of archives and make the interaction with them more memorable.
We train a sequence-to-sequence model for quote detection on available annotated corpora and evaluate how well its knowledge transfers to oral testimonies of Holocaust survivors to identify phrases that stand out even compared with the rest of these important stories. We manually annotate hundreds of automatically extracted text excerpts and assess their characteristics to identify what constitutes a meaningful quote in this domain, and show that this can be decided computationally
NLP with Transformer Models (Transformer Pretrained & Large Language Models, Dialogue & Text Generation with Transformers)
No full textThe second part of tutorial on transformer-based models, focusing on decoding, training and applications in dialogue and generation
Thesis Proposal: Efficient Methods for Natural Language Generation/Understanding Systems
No full textWhile Large Language Models (LLMs) have shown remarkable performance in various Natural Language Processing (NLP) tasks, their effectiveness seem to be heavily biased toward high-resource languages. This proposal aims to address this gap by developing efficient training strategies for low-resource languages. We propose various techniques for efficient learning in simluated low-resource settings for English. We then plan to adapt these methods for low-resource languages. We plan to experiment with both natural language generation and understanding models. We evaluate the models on similar benchmarks as the BabyLM challenge for English. For other languages, we plan to use treebanks and translation techniques to create our own silver test set to evaluate the low-resource LMs
LLM Agents Implement an NLG System from Scratch: Building Interpretable Rule-Based RDF-to-Text Generators
No full textWe present a novel neurosymbolic framework for RDF-to-text generation, in which the model is “trained” through collaborative interactions among multiple LLM agents rather than traditional backpropagation. The LLM agents produce rule-based Python code for a generator for the given domain, based on RDF triples only, with no in-domain human reference texts. The resulting system is fully interpretable, requires no supervised training data, and generates text nearly instantaneously using only a single CPU.
Our experiments on the WebNLG and OpenDialKG data show that outputs produced by our approach reduce hallucination, with only slight fluency penalties compared to finetuned or prompted language models
You are an LLM teaching a smaller model everything you know: Multi-task pretraining of language models with LLM-designed study plans
No full textTento článek navrhuje víceúčelový předtrénink jazykových modelů bez použití textových korpusů.
Metoda využívá existující velký jazykový model (LLM) k vygenerování rozmanitého korpusu obsahujícího trénovací data pro 56 automaticky navržených úloh a používá generované popisky k vylepšení trénovacího signálu.
Metoda se nespoléhá na skryté stavy ani na výstupní distribuce modelu učitele, takže ji lze použít v situacích, kdy je LLM učitele dostupný pouze prostřednictvím API.
Provedené experimenty ukazují, že modely trénované na navrhovaných syntetických korpusech dosahují konkurenceschopného nebo lepšího výkonu ve srovnání s modely trénovanými na stejně velkých lidských textech
FreshTab: Sourcing Fresh Data for Table-to-Text Generation Evaluation
No full textTable-to-text generation (insight generation
from tables) is a challenging task that requires
precision in analyzing the data. In addition,
the evaluation of existing benchmarks is af-
fected by contamination of Large Language
Model (LLM) training data as well as domain
imbalance. We introduce FreshTab, an on-the-
fly table-to-text benchmark generation from
Wikipedia, to combat the LLM data contam-
ination problem and enable domain-sensitive
evaluation. While non-English table-to-text
datasets are limited, FreshTab collects datasets
in different languages on demand (we experi-
ment with German, Russian and French in addi-
tion to English). We find that insights generated
by LLMs from recent tables collected by our
method appear clearly worse by automatic met-
rics, but this does not translate into LLM and
human evaluations. Domain effects are visi-
ble in all evaluations, showing that a domain-
balanced benchmark is more challenging
When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition
No full textThe increasing trust in large language models (LLMs), especially in the form of chatbots, is often undermined by the lack of their extrinsic evaluation. This holds particularly true in nutrition, where randomised controlled trials (RCTs) are the gold standard, and experts demand them for evidence-based deployment. LLMs have shown promising results in this field, but these are limited to intrinsic setups. We address this gap by running the first RCT involving LLMs for nutrition. We augment a rule-based chatbot with two LLM-based features: (1) message rephrasing for conversational variety and engagement, and (2) nutritional counselling through a fine-tuned model. In our seven-week RCT (n=81), we compare chatbot variants with and without LLM integration. We measure effects on dietary outcome, emotional well-being, and engagement. Despite our LLM-based features performing well in intrinsic evaluation, we find that they did not yield consistent benefits in real-world deployment. These results highlight critical gaps between intrinsic evaluations and real-world impact, emphasising the need for interdisciplinary, human-centred approaches