19 research outputs found
Doctor of Philosophy
No full textdissertationHigh-dimensional representations for words, text, images, knowledge graphs and other structured data are commonly used in different paradigms of machine learning and data mining. These representations have different degrees of interpretability, with efficient distributed representations coming at the cost of the loss of feature to dimension mapping. This implies that there is obfuscation in the way concepts are captured in these embedding spaces. Its effects are seen in many representations and tasks, one particularly problematic one being in language representations where the societal biases, learned from underlying data, are captured and occluded in unknown dimensions and subspaces. As a result, invalid associations (such as different races and their association with a polar notion of good versus bad) are made and propagated by the representations, leading to unfair outcomes in different tasks where they are used. This dissertation addresses some of these problems pertaining to the transparency and interpretability of such representations. A primary focus is the detection, quantification, and mitigation of socially biased associations in language representation. First, using the underlying geometrical structure of the high-dimensional space occupied by the representations, we build an understanding of the relative orientation between point sets. Next, we capture and isolate different feature subspaces or concept subspaces (such as gender or occupations in language representations) within the embedding space. Following this, we develop methods to decouple specific subspaces so as to better prepare the representations for specific downstream tasks. We also build a comprehensive range of probes to understand and highlight the different implicit associations learnt by the representations from underlying data, as an effort to distinguish between meaningful and invalid associations learnt and amplified by the embeddings. This is especially impactful as these representations drive much of the tasks and real world applications based on Natural Language Processing. Finally, we extend and apply these methods to different distributed representations and demonstrate the applications of increased interpretability
MisgenderMender: A Community-Informed Approach to Interventions for Misgendering
No full textContent Warning: This paper contains examples of misgendering and erasure
that could be offensive and potentially triggering.
Misgendering, the act of incorrectly addressing someone's gender, inflicts
serious harm and is pervasive in everyday technologies, yet there is a notable
lack of research to combat it. We are the first to address this lack of
research into interventions for misgendering by conducting a survey of
gender-diverse individuals in the US to understand perspectives about automated
interventions for text-based misgendering. Based on survey insights on the
prevalence of misgendering, desired solutions, and associated concerns, we
introduce a misgendering interventions task and evaluation dataset,
MisgenderMender. We define the task with two sub-tasks: (i) detecting
misgendering, followed by (ii) correcting misgendering where misgendering is
present in domains where editing is appropriate. MisgenderMender comprises 3790
instances of social media content and LLM-generations about non-cisgender
public figures, annotated for the presence of misgendering, with additional
annotations for correcting misgendering in LLM-generated text. Using this
dataset, we set initial benchmarks by evaluating existing NLP systems and
highlighting challenges for future models to address. We release the full
dataset, code, and demo at
https://tamannahossainkay.github.io/misgendermender/.Comment: NAACL 202
MISGENDERED: Limits of Large Language Models in Understanding Pronouns
No full textContent Warning: This paper contains examples of misgendering and erasure
that could be offensive and potentially triggering.
Gender bias in language technologies has been widely studied, but research
has mostly been restricted to a binary paradigm of gender. It is essential also
to consider non-binary gender identities, as excluding them can cause further
harm to an already marginalized group. In this paper, we comprehensively
evaluate popular language models for their ability to correctly use English
gender-neutral pronouns (e.g., singular they, them) and neo-pronouns (e.g., ze,
xe, thon) that are used by individuals whose gender identity is not represented
by binary pronouns. We introduce MISGENDERED, a framework for evaluating large
language models' ability to correctly use preferred pronouns, consisting of (i)
instances declaring an individual's pronoun, followed by a sentence with a
missing pronoun, and (ii) an experimental setup for evaluating masked and
auto-regressive language models using a unified method. When prompted
out-of-the-box, language models perform poorly at correctly predicting
neo-pronouns (averaging 7.7% accuracy) and gender-neutral pronouns (averaging
34.2% accuracy). This inability to generalize results from a lack of
representation of non-binary pronouns in training data and memorized
associations. Few-shot adaptation with explicit examples in the prompt improves
performance for neo-pronouns, but only to 64.7% even with 20 shots. We release
the full dataset, code, and demo at
https://tamannahossainkay.github.io/misgendered/Comment: Accepted at ACL 2023 as a long pape
SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes
No full textWhile generative multilingual models are rapidly being deployed, their safety
and fairness evaluations are largely limited to resources collected in English.
This is especially problematic for evaluations targeting inherently
socio-cultural phenomena such as stereotyping, where it is important to build
multi-lingual resources that reflect the stereotypes prevalent in respective
language communities. However, gathering these resources, at scale, in varied
languages and regions pose a significant challenge as it requires broad
socio-cultural knowledge and can also be prohibitively expensive. To overcome
this critical gap, we employ a recently introduced approach that couples LLM
generations for scale with culturally situated validations for reliability, and
build SeeGULL Multilingual, a global-scale multilingual dataset of social
stereotypes, containing over 25K stereotypes, spanning 20 languages, with human
annotations across 23 regions, and demonstrate its utility in identifying gaps
in model evaluations. Content warning: Stereotypes shared in this paper can be
offensive
MiTTenS: A Dataset for Evaluating Gender Mistranslation
No full textTranslation systems, including foundation models capable of translation, can produce errors that result in gender mistranslation, and such errors can be especially harmful. To measure the extent of such potential harms when translating into and out of English, we introduce a dataset, MiTTenS, covering 26 languages from a variety of language families and scripts, including several traditionally under-represented in digital resources. The dataset is constructed with handcrafted passages that target known failure patterns, longer synthetically generated passages, and natural passages sourced from multiple domains. We demonstrate the usefulness of the dataset by evaluating both neural machine translation systems and foundation models, and show that all systems exhibit gender mistranslation and potential harm, even in high resource languages.GitHub repository https://github.com/google-research-datasets/mitten
Cultural Re-contextualization of Fairness Research in Language Technologies in India
No full textRecent research has revealed undesirable biases in NLP data and models.
However, these efforts largely focus on social disparities in the West, and are
not directly portable to other geo-cultural contexts. In this position paper,
we outline a holistic research agenda to re-contextualize NLP fairness research
for the Indian context, accounting for Indian societal context, bridging
technological gaps in capability and resources, and adapting to Indian cultural
values. We also summarize findings from an empirical study on various social
biases along different axes of disparities relevant to India, demonstrating
their prevalence in corpora and models.Comment: Accepted to NeurIPS Workshop on "Cultures in AI/AI in Culture". This
is a non-archival short version, to cite please refer to our complete paper:
arXiv:2209.1222
On Measuring and Mitigating Biased Inferences of Word Embeddings
No full textWord embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them. We use this observation to design a mechanism for measuring stereotypes using the task of natural language inference. We demonstrate a reduction in invalid inferences via bias mitigation strategies on static word embeddings (GloVe). Further, we show that for gender bias, these techniques extend to contextualized embeddings when applied selectively only to the static components of contextualized embeddings (ELMo, BERT)
