539 research outputs found
Sort by
Keyphrase Generation: A Text Summarization Struggle
No full textAuthors' keyphrases assigned to scientific articles are essential for recognizing content and topic aspects. Most of the proposed supervised and unsupervised methods for keyphrase generation are unable to produce terms that are valuable but do not appear in the text. In this paper, we explore the possibility of considering the keyphrase string as an abstractive summary of the title and the abstract. First, we collect, process and release a large dataset of scientific paper metadata that contains 2.2 million records. Then we experiment with popular text summarization neural architectures. Despite using advanced deep learning models, large quantities of training data and many days of computation, our systematic evaluation on four test datasets reveals that the explored text summarization methods could not produce better keyphrases than the much simpler unsupervised methods or the existing supervised ones
MTM19 Tutorial: Transfer Learning for Low-Resource Languages
No full textThis tutorial will show you how to use the Tensor2Tensor and how to apply Transfer Learning to low-resource languages. It should be easy to follow for everyone, even people that never trained Machine Translation models
Promoting the Knowledge of Source Syntax in Transformer NMT Is Not Needed
No full textThe utility of linguistic annotation in neural machine translation seemed to had been established in past papers. The experiments were however limited to recurrent sequence-to-sequence architectures and relatively small data settings.
We focus on the state-of-the-art Transformer model and use comparably larger corpora. Specifically, we try to promote the knowledge of source-side syntax using multi-task learning either through simple data manipulation techniques or through a dedicated model component. In particular, we train one
of Transformer attention heads to produce source-side dependency tree.
Overall, our results cast some doubt on the utility of multi-task setups with linguistic information. The data manipulation techniques, recommended in previous works, prove ineffective in large data settings.
The treatment of self-attention as dependencies seems much more promising: it helps in translation and reveals that Transformer model can very easily grasp the syntactic structure.
An important but curious result is, however, that identical gains are obtained by using trivial ``linear trees'' instead of true dependencies. The reason for the gain thus may not be coming from the added linguistic knowledge but from some simpler regularizing effect we induced on self-attention matrices
CUNI Systems for the Unsupervised News Translation Task in WMT 2019
No full textIn this paper we describe the CUNI translation system used for the unsupervised news shared task of the ACL 2019 Fourth Conference on Machine Translation (WMT19). We follow the strategy of Artetxe ae at. (2018b), creating a seed phrase-based system where the phrase table is initialized from cross-lingual embedding mappings trained on monolingual data, followed by a neural machine translation system trained on synthetic parallel data. The synthetic corpus was produced from a monolingual corpus by a tuned PBMT model refined through iterative back-translation. We further focus on the handling of named entities, i.e. the part of vocabulary where the cross-lingual embedding mapping suffers most. Our system reaches a BLEU score of 15.3 on the German-Czech WMT19 shared task
SAO WMT19 Test Suite: Machine Translation of Audit Reports
No full textThis paper describes a machine translation test set of documents from the auditing domain and its use as one of the “test suites” in the WMT19 News Translation Task for translation directions involving Czech, English and German.
Our evaluation suggests that current MT systems optimized for the general news domain can perform quite well even in the particular domain of audit reports. The detailed manual evaluation however indicates that deep factual knowledge of the domain is necessary. For the naked eye of a non-expert, translations by many systems seem almost perfect and automatic MT evaluation with one reference is practically useless for considering these details.
Furthermore, we show on a sample document from the domain of agreements that even the best systems completely fail in preserving the semantics of the agreement, namely the identity of the parties
CUNI System for the WMT19 Robustness Task
No full textWe present our submission to the WMT19 Robustness Task. Our baseline system is the CUNI Transformer system trained for the WMT18 shared task on News Translation. Quantitative results show that the CUNI Transformer system is already far more robust to noisy input than the LSTM-based baseline provided by the task organizers. We further improved the performance of our model by fine-tuning on the in-domain noisy data
A Test Suite and Manual Evaluation of Document-Level NMT at WMT19
No full textAs the quality of machine translation rises and
neural machine translation (NMT) is moving
from sentence to document level translations,
it is becoming increasingly difficult to evaluate
the output of translation systems.
We provide a test suite for WMT19 aimed at
assessing discourse phenomena of MT systems participating in the News Translation
Task. We have manually checked the outputs
and identified types of translation errors that
are relevant to document-level translation
A Speech Test Set of Practice Business Presentations with Additional Relevant Texts
No full textWe present a test corpus of audio recordings and transcriptions of presentations of students' enterprises together with their slides and web-pages. The corpus is intended for evaluation of automatic speech recognition (ASR) systems, especially in conditions where the prior availability of in-domain vocabulary and named entities is benefitable.
The corpus consists of 39 presentations in English, each up to 90 seconds long.
The speakers are high school students from European countries with English as their second language.
We benchmark three baseline ASR systems on the corpus and show their imperfection
Grammatical Error Correction in Low-Resource Scenarios
No full textGrammatical error correction in English is a long studied problem with many existing systems and datasets. However, there has been only a limited research on error correction of other languages. In this paper, we present a new dataset AKCES-GEC on grammatical error correction for Czech. We then make experiments on Czech, German and Russian and show that when utilizing synthetic parallel corpus, Transformer neural machine translation model can reach new state-of-the-art results on these datasets. AKCES-GEC is published under CC BY-NC-SA 4.0 license at https://hdl.handle.net/11234/1-3057 and the source code of the GEC model is available at https://github.com/ufal/low-resource-gec-wnut2019
CUNI Submission for Low-Resource Languages in WMT News 2019
No full textThis paper describes the CUNI submission
to the WMT 2019 News Translation Shared
Task for the low-resource languages: GujaratiEnglish and Kazakh-English. We participated
in both language pairs in both translation directions. Our system combines transfer learning from a different high-resource language
pair followed by training on backtranslated
monolingual data. Thanks to the simultaneous training in both directions, we can iterate
the backtranslation process. We are using the
Transformer model in a constrained submission