539 research outputs found
Sort by
UMC002: English-Hindi Parallel Corpus
No full textEnglish-Hindi parallel corpus collected from several sources. Tokenized and sentence-aligned
Maximum Entropy Translation Model in Dependency-Based MT Framework
No full textMaximum Entropy Principle has been
used successfully in various NLP tasks. In
this paper we propose a forward translation
model consisting of a set of maximum
entropy classifiers: a separate classifier
is trained for each (sufficiently frequent)
source-side lemma. In this way
the estimates of translation probabilities
can be sensitive to a large number of features
derived from the source sentence (including
non-local features, features making
use of sentence syntactic structure,
etc.). When integrated into English-to-
Czech dependency-based translation scenario
implemented in the TectoMT framework,
the new translation model significantly
outperforms the baseline model
(MLE) in terms of BLEU. The performance
is further boosted in a configuration
inspired by Hidden Tree Markov Models
which combines the maximum entropy
translation model with the target-language
dependency tree model
Can Corpus Pattern Analysis Be Used in NLP?
No full textCorpus Pattern Analysis (CPA) [4], coined and implemented
by Hanks as the Pattern Dictionary of English Verbs (PDEV) [3], appears
to be the only deliberate and consistent implementation of Sinclair’s
concept of Lexical Item [12]. In his theoretical inquiries [5] Hanks
hypothesizes that the pattern repository produced by CPA can also support
the word sense disambiguation task. Although more than 670 verb
entries have already been compiled in PDEV, no systematic evaluation
of this ambitious project has been reported yet.
Assuming that the Sinclairian concept of the Lexical Item is correct, we
started to closely examine PDEV with its possible NLP application in
mind. Our experiments presented in this paper have been performed on
a pilot sample of English verbs to provide a first reliable view on whether
humans can agree in assigning PDEV patterns to verbs in a corpus. As
a conclusion we suggest procedures for future development of PDEV
Querying Diverse Treebanks in a Uniform Way
No full textThe paper presents a system for querying treebanks in a uniform way. The system is able to work with both dependency and constituency
based treebanks in any language. We demonstrate its abilities on 11 different treebanks. The query language used by the system
provides many features not available in other existing systems while still keeping the performance efficient. The paper also describes
the conversion of ten treebanks into a common XML-based format used by the system, touching the question of standards and formats.
The paper then shows several examples of linguistically interesting questions that the system is able to answer, for example browsing
verbal clauses without subjects or extraposed relative clauses, generating the underlying grammar in a constituency treebank, searching
for non-projective edges in a dependency treebank, or word-order typology of a language based on the treebank. The performance of
several implementations of the system is also discussed by measuring the time requirements of some of the queries
Perplexity of n-gram and Dependency Language Models
No full textLanguage models (LMs) are essential components of many applications such as speech recognition or machine translation. LMs factorize the probability of a string of words into a product of P(w_i|h_i), where h_i is the context (history) of word w_i. Most LMs use previous words as the context. The paper presents two alternative approaches: post-ngram LMs (which use following words as context) and dependency LMs (which exploit dependency structure of a sentence and can use e.g. the governing word as context). Dependency LMs could be useful whenever a topology of a dependency tree is available, but its lexical labels are unknown, e.g. in tree-to-tree machine translation. In comparison with baseline interpolated trigram LM both of the approaches achieve significantly lower perplexity for all seven tested languages (Arabic, Catalan, Czech, English, Hungarian, Italian, Turkish)
English-to-Czech System Combination
Get PDFThe results of two-month stay devoted to the combination of machine translation outputs when translating to Czech
UMC004: Hindi Web Texts
No full textA corpus of Hindi texts from the web suitable for language modelling: segmented into sentences and tokenized
Bohatá anotace ve frázovém strojovém překladu
No full textWe present a simple tool for the inclusion of rich linguistic annotation in phrase-based machine translation systems
Evaluating Utility of Data Sources in a Large Parallel Czech-English Corpus CzEng 0.9
No full textCzEng 0.9 is the third release of a large parallel corpus of Czech and English. For the current release, CzEng was extended by significant
amount of texts from various types of sources, including parallel web pages, electronically available books and subtitles. This paper
describes and evaluates filtering techniques employed in the process in order to avoid misaligned or otherwise damaged parallel sentences
in the collection. We estimate the precision and recall of two sets of filters. The first set was used to process the data before their inclusion
into CzEng. The filters from the second set were newly created to improve the filtering process for future releases of CzEng. Given the
overall amount and variance of sources of the data, our experiments illustrate the utility of parallel data sources with respect to extractable
parallel segments. As a similar behaviour can be expected for other language pairs, our results can be interpreted as guidelines indicating
which sources should other researchers exploit first