539 research outputs found
Sort by
AARGH! End-to-end Retrieval-Generation for Task-Oriented Dialog
No full textWe introduce AARGH, an end-to-end task-oriented dialog system combining retrieval and generative approaches in a single model, aiming at improving dialog management and lexical diversity of outputs. The model features a new response selection method based on an action-aware training objective and a simplified single-encoder retrieval architecture which allow us to build an end-to-end retrieval-enhanced generation model where retrieval and generation share most of the parameters. On the MultiWOZ dataset, we show that our approach produces more diverse outputs while maintaining or improving state tracking and context-to-response generation performance, compared to state-of-the-art baselines
CorefUD 1.0
No full textCorefUD is a collection of previously existing datasets annotated with coreference, which we converted into a common annotation scheme. In total, CorefUD in its current version 0.2 consists of 17 datasets for 11 languages, and compared to the version 0.2, the file format has been reworked and a number of annotation errors have been fixed
LangTools pro ERSTE
No full textPrezentace jazykových nástrojů pro Českou spořitelnu.
Rozpoznávání mluvené řeči, strojový překlad v rámci okamžité odpovědi, mezijazyčné vyhledávání informací, odpovídání na otázky, automatické shrnování schůzek
FINDINGS OF THE IWSLT 2022 EVALUATION CAMPAIGN
No full textThe evaluation campaign of the 19th International Conference on Spoken Language Translation featured eight shared tasks: (i) Simultaneous speech translation, (ii) Offline speech translation, (iii) Speech to speech translation, (iv) Low-resource speech translation, (v) Multilingual speech translation, (vi) Dialect speech translation, (vii) Formality control for speech translation, (viii) Isometric speech translation. A total of 27 teams participated in at least one of the shared tasks. This paper details, for each shared task, the purpose of the task, the data that were released, the evaluation metrics that were applied, the submissions that were received and the results that were achieved
Learning Interpretable Latent Dialogue Actions With Less Supervision
No full textWe present a novel architecture for explainable modeling of task-oriented dialogues with discrete latent variables to represent dialogue actions. Our model is based on variational recurrent neural networks (VRNN) and requires no explicit annotation of semantic information. Unlike previous works, our approach models the system and user turns separately and performs database query modeling, which makes the model applicable to task-oriented dialogues while producing easily interpretable action latent variables. We show that our model outperforms previous approaches with less supervision in terms of perplexity and BLEU on three datasets, and we propose a way to measure dialogue success without the need for expert annotation. Finally, we propose a novel way to explain semantics of the latent variables with respect to system actions
Computational Literary Studies Infrastructure (CLSINFRA): a H2020 Research Infrastructure Project that aids to connect researchers, data, and methods
No full textThe aim of this poster is to provide an overview of the principal objectives of the newly started H2020
Computational Literary Studies (CLS) project- https://www.clsinfra.io. CLS is a infrastructure project
works to develop and bring together resources of high-quality data, tools and knowledge to aid new
approaches to studying literature in the digital age. Conducting computational literary studies has a
number of challenges and opportunities from multilingual and bringing together distributing
information. At present, the landscape of literary data is diverse and fragmented. Even though many
resources are currently available in digital libraries, archives, repositories, websites or catalogues, a lack
of standardisation hinders how they are constructed, accessed and the extent to which they are reusable
(Ciotti 2014). CLS project aims to federate these resources, with the tools needed to interrogate them,
and with a widened base of users, in the spirit of the FAIR and CARE principles (Wilkinson et al. 2016).
The resulting improvements will benefit researchers by bridging gaps between greater- and lesser-
resourced communities in computational literary studies and beyond, ultimately offering opportunities
to create new research and insight into our shared and varied European cultural heritage.
Rather than building entirely new resources for literary studies, the project is committed to exploiting
and connecting the already-existing efforts and initiatives, in order to acknowledge and utilize the
immense human labour that has already been undertaken. Therefore, the project builds on recently-
compiled high-quality literary corpora, such as DraCor and ELTeC (Fischer et al. 2019, Burnard et al. 2021,
Schöch et al. in press), integrates existing tools for text analysis, e.g. TXM, stylo, multilingual NLP
pipelines (Heiden 2010, Eder et al. 2016), and takes advantage of deep integration with two other
infrastructural projects, namely the CLARIN and DARIAH ERICs. Consequently, the project aims at
building a coherent ecosystem to foster the technical and intellectual findability and accessibility of
relevant data. The ecosystem consists of (1) resources, i.e. text collections for drama, poetry and prose in
several languages, (2) tools, (3) methodological and theoretical considerations, (4) a network of CLS
scholars based at different European institutions, (5) a system of short-term research stays for both early
career researchers and seasoned scholars, (6) a repository for training materials, as well as (7) an
efficient dissemination strategy. This is achieved through a collaboration between participating
institutions: Institute of Polish Language at the Polish Academy of Sciences, Poland; University of
Potsdam, Germany; Austrian Academy of Sciences, Austria; National University of Distance Education,
Spain; École Normale Supérieure de Lyon, France; Humboldt University of Berlin, German; Charles
University, Czech Republic; Digital Research Infrastructure for the Arts and Humanities, France; Ghent
Centre for Digital Humanities, Ghent University, Belgium; Belgrade Centre for Digital Humanities, Serbia;
Huygens Institute for the History of the Netherlands (Royal Netherlands Academy of Arts and Sciences),
Netherlands; Trier Center for Digital Humanities, Trier University, Germany; Moore Institute, National
University of Ireland Galway, Ireland
From COMET to COMES – Can Summary Evaluation Benefit from Translation Evaluation?
No full textComet is a recently proposed trainable neural-based evaluation metric developed to assess the quality of Machine Translation systems. In this paper, we explore the usage of Comet for evaluating Text Summarization systems -- despite being trained on multilingual MT outputs, it performs remarkably well in monolingual settings, when predicting summarization output quality. We introduce a variant of the model -- Comes -- trained on the annotated summarization outputs that uses MT data for pre-training. We examine its performance on several datasets with human judgments collected for different notions of summary quality, covering several domains and languages
Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue
No full textThe SIGDIAL conference is a premier publication venue for research in discourse and dialogue. This year the conference is organized as a hybrid event with both in-person and remote participation on September 7-9, 2022, at Heriot-Watt University, Edinburgh, Scotland, and is hosted by the Interaction Lab and the National Robotarium
Backtranslation Feedback Improves User Confidence in MT, Not Quality
No full textTranslating text into a language unknown to the text’s author, dubbed outbound translation, is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. We demonstrate this by showing three ways in which user confidence in the outbound translation, as well as its overall final quality, can be affected: backward translation, quality estimation (with alignment) and source paraphrasing. In this paper, we describe an experiment on outbound translation from English to Czech and Estonian. We examine the effects of each proposed feedback module and further focus on how the quality of machine translation systems influence these findings and the user perception of success. We show that backward translation feedback has a mixed effect on the whole process: it increases user confidence in the produced translation, but not the objective quality
A Fine-Grained Analysis of BERTScore
No full textBERTScore (Zhang et al., 2020), a recently proposed automatic metric for machine translation quality, uses BERT (Devlin et al., 2019), a large pre-trained language model to evaluate candidate translations with respect to a gold translation. Taking advantage of BERT’s semantic and syntactic abilities, BERTScore seeks to avoid the flaws of earlier approaches like BLEU, instead scoring candidate translations based on their semantic similarity to the gold sentence. However, BERT is not infallible; while its performance on NLP tasks set a new state of the art in general, studies of specific syntactic and semantic phenomena have shown where BERT’s performance deviates from that of humans more generally.
This naturally raises the questions we address in this paper: what are the strengths and weaknesses of BERTScore? Do they relate to
known weaknesses on the part of BERT? We find that while BERTScore can detect when a candidate differs from a reference in important content words, it is less sensitive to smaller errors, especially if the candidate is lexically or stylistically similar to the reference