36 research outputs found
Proactive information retrieval
No full textUsers interact with digital systems with some task in his mind. An example of a task could be writing a research paper on a topic. Tasks can be single or multi-staged. In the process of accomplishing their task objectives, a user often needs to interact with an information retrieval (IR) system to address one or more information needs which arise while working on their task, e.g. for writing their research paper on a chosen topic, the user needs to look for existing research works related to the topic. Traditional IR systems do not take into account a user's task intent while showing search results to the user for a specific query submitted by the user. In our work we propose next generation IR systems (i.e. proactive IR systems) which seek to anticipate the user's underlying task from his interaction with a digital system to automatically identify their information needs and to suggest potentially relevant information sources to help the user to accomplish his task
MADS: A Multi-modal Academic Document Segmentation Dataset for Smart Question Bank Management
No full textIn today’s world, most major academic institutes and organizations conduct competitive exams to assess eligibility of students for admission or recruitment. Due to the rising craze among participants, traditional methods are not optimized enough to get ahead in the race. The inclusion of AI enabled tutoring is mandatory for such exams. One such area of implementation is smart question bank management system. Though we have large volumes of questions of competitive exams in physical mode, however, they are harder to process visually for systems as they consist of several types of text and non-text elements such as numbers, equations, images alongside textual paragraphs. For this purpose, we propose MADS, which is a multi-modal academic document segmentation dataset consisting of images of documents containing heterogeneous questions from the competitive exams like GMAT, GRE, GATE, SAT, UGC-NET. These documents consist of textual paragraphs along with numbers, images and equations. The dataset comes with bounding box annotation in two popular format YOLO and PASCAL-VOC formats to aid the development of efficient document segmentation algorithms. Additionally, benchmarks have been provided for state of the art deep learning based implementations such as Faster RCNN and YOLO-v8. From application point of view, the proposed dataset can identify different objects in an image so that later it can be used for semantic relationship and question answering applications enhancing comprehension and personalized learning experiences, thus, supporting the goal of providing quality education
Unraveling the Influence of Training Data and Internal Structures in Large Language Models for Enhanced Explainability (Student Abstract)
No full textRecent advances in deep learning have expanded the application of large language models (LLMs) across fields such as medicine, finance, and education. Understanding the mechanisms underlying these models is essential to mitigate issues like hallucinations and bias. This study provides deep learning practitioners with insights into how specific training data points and internal structures influence model behaviour. Using influence functions and mechanistic interpretability, we will analyze the impact of data on model predictions across various tasks. Preliminary findings indicate that semantic search techniques, such as FAISS, enable efficient identification of influential training points in GPT-2 small. Future work will extend these methods to additional tasks and more complex models, with a focus on further elucidating LLM structures to improve interpretability
Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning
No full textHuman society had a long history of suffering from cognitive biases leading to social prejudices and mass injustice. The prevalent existence of cognitive biases in large volumes of historical data can pose a threat of being manifested as unethical and seemingly inhumane predictions as outputs of AI systems trained on such data. To alleviate this problem, we propose a bias-aware multi-objective learning framework that given a set of identity attributes (e.g. gender, ethnicity etc.) and a subset of sensitive categories of the possible classes of prediction outputs, learns to reduce the frequency of predicting certain combinations of them, e.g. predicting stereotypes such as ‘most blacks use abusive language’, or ‘fear is a virtue of women’. Our experiments conducted on an emotion prediction task with balanced class priors shows that a set of baseline bias-agnostic models exhibit cognitive biases with respect to gender, such as women are prone to be afraid whereas men are more prone to be angry. In contrast, our proposed bias-aware multi-objective learning methodology is shown to reduce such biases in the predictid emotions
Investigating Adversarial Robustness in Language Models: Adversarial Attacks, Certification, and Defense
No full textJoint estimation of topics and hashtag relevance in cross-lingual tweets
No full textTwitter is a widely used platform for sharing news articles. An
emerging trend in multi-lingual communities is to share non-English
news articles using English tweets in order to spread the news to a
wider audience. In general, the choice of relevant hashtags for such
tweets depends on the topic of the non-English news article. In this
paper, we address the problem of automatically detecting the relevance of the hashtags of such tweets. More specifically, we propose
a generative model to jointly model the topics within an English
tweet and those within the non-English news article shared from
it to predict the relevance of the hashtags of the tweet. For conducting experiments, we compiled a collection of English tweets
that share news articles in Bengali (a South Asian language). Our
experiments on this dataset demonstrate that this joint estimation
based approach using the topics from both the non-English news
articles and the tweets proves to be more effective for relevance
estimation than that of only using the topics of a tweet itself
Tempo-lexical context driven word embedding for cross-session search task extraction
No full textSearch task extraction in information retrieval
is the process of identifying search intents over
a set of queries relating to the same topical information need. Search tasks may potentially
span across multiple search sessions. Most existing research on search task extraction has
focused on identifying tasks within a single
session, where the notion of a session is defined by a fixed length time window. By contrast, in this work we seek to identify tasks
that span across multiple sessions. To identify tasks, we conduct a global analysis of
a query log in its entirety without restricting
analysis to individual temporal windows. To
capture inherent task semantics, we represent
queries as vectors in an abstract space. We
learn the embedding of query words in this
space by leveraging the temporal and lexical
contexts of queries. To evaluate the effectiveness of the proposed query embedding, we
conduct experiments of clustering queries into
tasks with a particular interest of measuring
the cross-session search task recall. Results of
our experiments demonstrate that task extraction effectiveness, including cross-session recall, is improved significantly with the help of
our proposed method of embedding the query
terms by leveraging the temporal and templexical contexts of queries
Multi-Objective Few-shot Learning for Fair Classification
No full textIn this paper, we propose a general framework for mitigating the disparities
of the predicted classes with respect to secondary attributes
within the data (e.g., race, gender etc.). Our proposed method involves
learning a multi-objective function that in addition to learning
the primary objective of predicting the primary class labels
from the data, also employs a clustering-based heuristic to minimize
the disparities of the class label distribution with respect to
the cluster memberships, with the assumption that each cluster
should ideally map to a distinct combination of attribute values.
Experiments demonstrate effective mitigation of cognitive biases on
a benchmark dataset without the use of annotations of secondary
at-tribute values (the zero-shot case) or with the use of a small
number of attribute value annotations (the few-shot case)
Procrastination is the thief of time: evaluating the effectiveness of proactive search systems
No full textTraditional search system users actively interact with the system
to complete their search task. We believe that the next generation
of search systems will see a shift towards proactive understanding
of user intent based on analysis of user activities. Such a proactive
system could start recommending documents that are likely to help
users accomplish their tasks without requiring them to explicitly
submit queries to the system. We propose a framework to evaluate
such a search system. The key idea behind our proposed metric is to
aggregate a correlation measure over a search session between the
expected outcome, which in this case refers to the list of documents
retrieved with a true user query, and the predicted outcome, which
refers to the list of documents recommended by a proactive search
system. Experiments on the AOL query log data show that the
ranking of two sample proactive IR systems induced by our metric
conforms to the expected ranking between these systems
