539 research outputs found
Sort by
Findings of the 2018 Conference on Machine Translation (WMT18)
No full textThis paper presents the results of the premier
shared task organized alongside the Confer-
ence on Machine Translation (WMT) 2018.
Participants were asked to build machine
translation systems for any of 7 language pairs
in both directions, to be evaluated on a test set
of news stories. The main metric for this task
is human judgment of translation quality. This
year, we also opened up the task to additional
test suites to probe specific aspects of transla-
tion
Training Tips for the Transformer Model
No full textThis article describes our experiments in neural machine translation using the recent Tensor2Tensor
framework and the Transformer sequence-to-sequence model (Vaswani et al., 2017).
We examine some of the critical parameters that affect the final translation quality, memory
usage, training stability and training time, concluding each experiment with a set of recommendations
for fellow researchers. In addition to confirming the general mantra “more data
and larger models”, we address scaling to multiple GPUs and provide practical tips for improved
training regarding batch size, learning rate, warmup steps, maximum sentence length
and checkpoint averaging. We hope that our observations will allow others to get better results
given their particular hardware and data constraints
Morphological and Language-Agnostic Word Segmentation for NMT
No full textThe state of the art of handling rich morphology in neural machine translation (NMT) is to break word forms into subword units, so that the overall vocabulary size of these units fits the practical limits given by the NMT model and GPU memory capacity. In this paper, we compare two common but linguistically uninformed methods of subword construction (BPE and STE, the method implemented in Tensor2Tensor toolkit) and two linguistically-motivated methods: Morfessor and one novel method, based on a derivational dictionary. Our experiments with German-to-Czech translation, both morphologically rich, document that so far, the non-motivated methods perform better. Furthermore, we identify a critical difference between BPE and STE and show a simple pre-processing step for BPE that considerably increases translation quality as evaluated by automatic measures
Neural Monkey: The Current State and Beyond
No full textNeural Monkey is an open-source toolkit for sequence-to-sequence learning. The focus of this paper is to present the current state of the toolkit to the intended audience, which includes students and researchers, both active in the deep learning community and newcomers. For each of these target groups, we describe the most relevant features of the toolkit, including the simple configuration scheme, methods of model inspection that promote useful intuitions, or a modular design for easy prototyping. We summarize relevant contributions to the research community which were made using this toolkit and discuss the characteristics of our toolkit with respect to other existing systems. We conclude with a set of proposals for future development
Promises and Pitfalls of Neural MT
No full textIn my talk, I presented the current translation quality achieved by neural machine translation and thoroughly discussed the expected benefits and risks for interpreters arising from the EU project ELITR
Slavic Forest, Norwegian Wood
No full textWe once had a corp,
or should we say, it once had us
They showed us its tags,
isn’t it great, unified tags
They asked us to parse
and they told us to use everything
So we looked around
and we noticed there was near nothing
We took other langs,
bitext aligned: words one-to-one
We played for two weeks,
and then they said, here is the test
The parser kept training till morning,
just until deadline
So we had to wait and hope what we get
would be just fine
And, when we awoke,
the results were done, we saw we’d won
So, we wrote this paper,
isn’t it good, Norwegian wood
Jak pracuje internetový vyhledávač
No full textThis is something everyone uses, but it is not absolutely clear how it works. For a given query it is not possible to simply traverse all the existing billions of websites, and return the few millions that contain the sought words. I will show what a reverse index is, how results are sorted, and I will also add more advanced techniques (TF.IDF, vector model, lemmatization, synonyms, named entities)
Neural Monkey: An Open-source Tool for Sequence Learning
No full textIn this paper, we announce development of Neural Monkey — an open-source neural machine translation (NMT) and general sequence-to-sequence learning system built over TensorFlow machine learning library.
The system provides a high-level API with support for fast prototyping of complex architectures with multiple sequence encoders and decoders. These models’ overall architecture is specified in easy-to-read configuration files. The long-term goal of Neural Monkey project is to create and maintain a growing collection of implementations of recently proposed components or methods, and therefore it is designed to be easily extensible. The trained models can be deployed either for batch data processing or as a web service. In the presented paper, we describe the design of the system and introduce the reader to running experiments using Neural Monkey
Task3 Patient-Centred Information Retrieval: Team CUNI
No full textIn this paper we present our participation as the team of the Charles University at Task3 Patient-Centred Information Retrieval in CLEF eHealth Evaluation lab 201