539 research outputs found
Sort by
Findings of the 2017 Conference on Machine Translation (WMT17)
No full textThis paper presents the results of the
WMT17 shared tasks, which included
three machine translation (MT) tasks
(news, biomedical, and multimodal), two
evaluation tasks (metrics and run-time estimation
of MT quality), an automatic
post-editing task, a neural MT training
task, and a bandit learning task
Paying Attention to Multi-Word Expressions in Neural Machine Translation
No full textProcessing of multi-word expressions (MWEs) is a known problem for any natural language
processing task. Even neural machine translation (NMT) struggles to overcome it. This paper
presents results of experiments on investigating NMT attention allocation to the MWEs and
improving automated translation of sentences that contain MWEs in English→Latvian and
English→Czech NMT systems. Two improvement strategies were explored—(1) bilingual
pairs of automatically extracted MWE candidates were added to the parallel corpus used to
train the NMT system, and (2) full sentences containing the automatically extracted MWE
candidates were added to the parallel corpus. Both approaches allowed to increase automated
evaluation results. The best result—0.99 BLEU point increase—has been reached with the first
approach, while with the second approach minimal improvements achieved. We also provide
open-source software and tools used for MWE extraction and alignment inspection
Curriculum Learning and Minibatch Bucketing in Neural Machine Translation
No full textWe examine the effects of particular orderings of sentence pairs on the on-line training of neural machine translation (NMT). We focus on two types of such orderings: (1) ensuring that each minibatch contains sentences similar in some aspect and (2) gradual inclusion of some sentence types as the training progresses
(so called “curriculum learning”). In our English-to-Czech experiments, the internal homogeneity of minibatches has no effect on the training but some of our “curricula” achieve a small improvement over the baseline
Results of the WMT17 Metrics Shared Task
No full textThis paper presents the results of the
WMT17 Metrics Shared Task. We asked
participants of this task to score the outputs
of the MT systems involved in the
WMT17 news translation task and Neural
MT training task. We collected scores
of 14 metrics from 8 research groups. In
addition to that, we computed scores of
7 standard metrics (BLEU, SentBLEU,
NIST, WER, PER, TER and CDER) as
baselines. The collected scores were evaluated
in terms of system-level correlation
(how well each metric’s scores correlate
with WMT17 official manual ranking of
systems) and in terms of segment level
correlation (how often a metric agrees with
humans in judging the quality of a particular
sentence).
This year, we build upon two types of
manual judgements: direct assessment
(DA) and HUME manual semantic judgements
CUNI System for WMT17 Automatic Post-Editing Task
No full textFollowing upon the last year's CUNI system for automatic post-editing of machine translation output,
we focus on exploiting the potential of sequence-to-sequence neural models for this task. In this system description paper, we compare several encoder-decoder architectures on a smaller-scale models and present the system we submitted to WMT 2017 Automatic Post-Editing shared task based on this preliminary comparison. We also show how simple inclusion of synthetic data can improve the overall performance as measured by an automatic evaluation metric. Lastly, we list few example outputs generated by our post-editing system
Attention Strategies for Multi-Source Sequence-to-Sequence Learning
No full textModeling attention in neural multi-source sequence-to-sequence learning remains a relatively unexplored area, despite its usefulness in tasks that incorporate multiple source languages or modalities. We propose two novel approaches to combine the outputs of attention mechanisms over each source sequence, flat and hierarchical. We compare the proposed methods with existing techniques and present results of systematic evaluation of those methods on the WMT16 Multimodal Translation and
Automatic Post-editing tasks. We show the proposed methods achieve competitive results on both tasks
Variable Mini-Batch Sizing and Pre-Trained Embeddings
No full textThis paper describes our submission to
the WMT 2017 Neural MT Training Task.
We modified the provided NMT system in
order to allow for interrupting and con-
tinuing the training of models. This al-
lowed mid-training batch size decremen-
tation and incrementation at variable rates.
In addition to the models with variable
batch size, we tried different setups with
pre-trained word2vec embeddings. Aside
from batch size incrementation, all our ex-
periments performed below the baselin
CUNI System for the WMT17 Multimodal Translation Task
No full textIn this paper, we describe our submissions to the WMT17 Multimodal
Translation Task. For Task 1 (multimodal translation), our best scoring
system is a purely textual neural translation of the source image caption to
the target language. The main feature of the system is the use of additional
data that was acquired by selecting similar sentences from parallel corpora
and by data synthesis with back-translation. For Task 2 (cross-lingual image
captioning), our best submitted system generates an English caption which is
then translated by the best system used in Task 1. We also present negative
results, which are based on ideas that we believe have potential of making
improvements, but did not prove to be useful in our particular setup
CUNI NMT System for WAT 2017 Translation Tasks
No full textThe paper presents this year's CUNI submissions to the WAT 2017 Translation Task focusing on the Japanese-English translation, namely Scientific papers subtask, Patents subtask and Newswire subtask. We compare two neural network architectures, the standard sequence-to-sequence with attention (Seq2Seq) (Bahdanau et al., 2014) and an architecture using convolutional sentence encoder (FBConv2Seq) described by Gehring et al. (2017), both implemented in the NMT framework Neural Monkey that we currently participate in developing.
We also compare various types of preprocessing of the source Japanese sentences and their impact on the overall results. Furthermore, we include the results of our experiments with out-of-domain data obtained by combining the corpora provided for each subtask
Visualizing Neural Machine Translation Attention and Confidence
No full textIn this article, we describe a tool for visualizing the output and attention weights of neural
machine translation systems and for estimating confidence about the output based on the
attention.
Our aim is to help researchers and developers better understand the behaviour of their
NMT systems without the need for any reference translations. Our tool includes command
line and web-based interfaces that allow to systematically evaluate translation outputs from
various engines and experiments. We also present a web demo of our tool with examples of
good and bad translations: http://ej.uz/nmt-attentio