1,721,086 research outputs found
Using Mixed Incentives to Document Xi’an Guanzhong
No full textThis paper describes our use of mixed incentives and the citizen science portal LanguageARC to prepare, collect and quality control a large corpus of object namings for the purpose of providing speech data to document the under-represented Guanzhong dialect of Chinese spoken in the Shaanxi province in the environs of Xi’an.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Multimedia Computin
Paraphrasing and Translation
Get PDFParaphrasing and translation have previously been treated as unconnected natural lan¬
guage processing tasks. Whereas translation represents the preservation of meaning
when an idea is rendered in the words in a different language, paraphrasing represents
the preservation of meaning when an idea is expressed using different words in the
same language. We show that the two are intimately related. The major contributions
of this thesis are as follows:• We define a novel technique for automatically generating paraphrases using
bilingual parallel corpora, which are more commonly used as training data for
statistical models of translation.• We show that paraphrases can be used to improve the quality of statistical ma¬
chine translation by addressing the problem of coverage and introducing a degree
of generalization into the models.• We explore the topic of automatic evaluation of translation quality, and show that
the current standard evaluation methodology cannot be guaranteed to correlate
with human judgments of translation quality.Whereas previous data-driven approaches to paraphrasing were dependent upon
either data sources which were uncommon such as multiple translation of the same
source text, or language specific resources such as parsers, our approach is able to
harness more widely parallel corpora and can be applied to any language which has
a parallel corpus. The technique was evaluated by replacing phrases with their para¬
phrases, and asking judges whether the meaning of the original phrase was retained
and whether the resulting sentence remained grammatical. Paraphrases extracted from
a parallel corpus with manual alignments are judged to be accurate (both meaningful
and grammatical) 75% of the time, retaining the meaning of the original phrase 85%
of the time. Using automatic alignments, meaning can be retained at a rate of 70%.Being a language independent and probabilistic approach allows our method to be
easily integrated into statistical machine translation. A paraphrase model derived from
parallel corpora other than the one used to train the translation model can be used to
increase the coverage of statistical machine translation by adding translations of previously unseen words and phrases. If the translation of a word was not learned, but
a translation of a synonymous word has been learned, then the word is paraphrased and its paraphrase is translated. Phrases can be treated similarly. Results show that
augmenting a state-of-the-art SMT system with paraphrases in this way leads to significantly improved coverage and translation quality. For a training corpus with 10,000
sentence pairs, we increase the coverage of unique test set unigrams from 48% to 90%,
with more than half of the newly covered items accurately translated, as opposed to
none in current approaches
Crowd-Workers: Aggregating Information Across Turkers to Help Them Find Higher Paying Work
No full textThe Mechanical Turk crowdsourcing platform currently fails to provide the most basic piece of information to enable workers to make informed decisions about which tasks to undertake: what is the expected hourly pay? Mechanical Turk advertises a reward amount per assignment, but does not give any indication of how long each assignment will take. We have developed a browser plugin that tracks the length of time it takes to complete a task, and a web service that aggregates the information across many workers. Our web service, crowd-workers.com, allows workers to discovery higher paying work by sorting tasks by estimated hourly rate
Adapting coreference resolution for narrative processing
No full text© 2015 Association for Computational Linguistics. Domain adaptation is a challenge for supervised NLP systems because of expensive and time-consuming manual annotated resources. We present a novel method to adapt a supervised coreference resolution system trained on newswire to short narrative stories without retraining the system. The idea is to perform inference via an Integer Linear Programming (ILP) formulation with the features of narratives adopted as soft constraints. When testing on the UMIREC1 and N22 corpora with the-stateof-the-art Berkeley coreference resolution system trained on OntoNotes3, our inference substantially outperforms the original inference on the CoNLL 2011 metric.status: Publishe
PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale
Get PDFExisting question answering (QA) systems owe much of their success to large,
high-quality training data. Such annotation efforts are costly, and the
difficulty compounds in the cross-lingual setting. Therefore, prior
cross-lingual QA work has focused on releasing evaluation datasets, and then
applying zero-shot methods as baselines. This work proposes a synthetic data
generation method for cross-lingual QA which leverages indirect supervision
from existing parallel corpora. Our method termed PAXQA (Projecting annotations
for cross-lingual (x) QA) decomposes cross-lingual QA into two stages. First,
we apply a question generation (QG) model to the English side. Second, we apply
annotation projection to translate both the questions and answers. To better
translate questions, we propose a novel use of lexically-constrained machine
translation, in which constrained entities are extracted from the parallel
bitexts.
We apply PAXQA to generate cross-lingual QA examples in 4 languages (662K
examples total), and perform human evaluation on a subset to create validation
and test splits. We then show that models fine-tuned on these datasets
outperform prior synthetic data generation models over several extractive QA
datasets. The largest performance gains are for directions with non-English
questions and English contexts. Ablation studies show that our dataset
generation method is relatively robust to noise from automatic word alignments,
showing the sufficient quality of our generations. To facilitate follow-up
work, we release our code and datasets at https://github.com/manestay/paxqa .Comment: EMNLP 2023 (Findings
Artificial Intelligence in mental health and the biases of language based models
Get PDFBackground:
The rapid integration of Artificial Intelligence (AI) into the healthcare field has occurred with little communication between computer scientists and doctors. The impact of AI on health outcomes and inequalities calls for health professionals and data scientists to make a collaborative effort to ensure historic health disparities are not encoded into the future. We present a study that evaluates bias in existing Natural Language Processing (NLP) models used in psychiatry and discuss how these biases may widen health inequalities. Our approach systematically evaluates each stage of model development to explore how biases arise from a clinical, data science and linguistic perspective.//
Design/Methods:
A literature review of the uses of NLP in mental health was carried out across multiple disciplinary databases with defined Mesh terms and keywords. Our primary analysis evaluated biases within ‘GloVe’ and ‘Word2Vec’ word embeddings. Euclidean distances were measured to assess relationships between psychiatric terms and demographic labels, and vector similarity functions were used to solve analogy questions relating to mental health.//
Results:
Our primary analysis of mental health terminology in GloVe and Word2Vec embeddings demonstrated significant biases with respect to religion, race, gender, nationality, sexuality and age. Our literature review returned 52 papers, of which none addressed all the areas of possible bias that we identify in model development. In addition, only one article existed on more than one research database, demonstrating the isolation of research within disciplinary silos and inhibiting cross-disciplinary collaboration or communication.//
Conclusion:
Our findings are relevant to professionals who wish to minimize the health inequalities that may arise as a result of AI and data-driven algorithms. We offer primary research identifying biases within these technologies and provide recommendations for avoiding these harms in the future
Using Mechanical Turk to Build Machine Translation Evaluation Sets
Get PDFBuilding machine translation (MT) test sets is a relatively expensive task. As MT becomes increasingly desired for more and more language pairs and more and more domains, it becomes necessary to build test sets for each case. In this paper, we investigate using Amazon’s Mechanical Turk (MTurk) to make MT test sets cheaply. We find that MTurk can be used to make test sets much cheaper than professionally-produced test sets. More importantly, in experiments with multiple MT systems, we find that the MTurk-produced test sets yield essentially the same conclusions regarding system performance as the professionally-produced test sets yield.This research was supported by the EuroMatrix-Plus project funded by the European Commission, by the DARPA GALE program under Contract No. HR0011-06-2-0001, and the NSF under grant IIS-0713448. Thanks to Amazon Mechanical Turk for providing a $100 credit
Bucking the Trend: Large-Scale Cost-Focused Active Learning for Statistical Machine Translation
Get PDFWe explore how to improve machine translation systems by adding more translation data in situations where we already have substantial resources. The main challenge is how to buck the trend of diminishing returns that is commonly encountered. We present an active learning-style data solicitation algorithm to meet this challenge. We test it, gathering annotations via Amazon Mechanical Turk, and find that we get an order of magnitude increase in performance rates of improvement.Johns Hopkins University Human Language Technology Center of Excellenc
Visual bilingual lexicon induction with transferred ConvNet features
No full textThis paper is concerned with the task of bilingual lexicon induction using image-based features. By applying features from a convolutional neural network (CNN), we obtain state-of-the-art performance on a standard dataset, obtaining a 79% relative improvement over previous work which uses bags of visual words based on SIFT features. The CNN image-based approach is also compared with state-of-the-art linguistic approaches to bilingual lexicon induction, even outperforming these for one of three language pairs on another standard dataset. Furthermore, we shed new
light on the type of visual similarity metric to use for genuine similarity versus relatedness tasks, and experiment with using multiple layers from the same network in an attempt to improve performance.sponsorship: Supported by the PARIS project (IWT-SBO
110067), the SCATE project (IWT-SBO 130041), and the PDM Kort postdoctoral fellowship PDMK14/117 from KU Leuven for Ivan Vulic.status: Accepte
- …
