268 research outputs found
Gated relational stacked denoising autoencoder with localized author embedding for global citation recommendation
No full textCitation recommendation is an effective and efficient way to facilitate authors finding desired references. This paper presents a novel neural network based model, called gated relational probabilistic stacked denoising autoencoder with localized author (GRSLA) embedding, for global citation recommendation task. Our model is comprised of two modules with different neural network architecture. For each citing and cited papers, we use a gated paper embedding module, which is extended from probabilistic stacked denoising autoencoder (PSDAE) by adding gated units, to obtain their paper vectors. The added gated units are able to utilize text information of cited paper to refine the vector representation of citing paper in multiple semantic levels. For an author in papers, we first apply topic model to obtain his/her semantic neighbors, and then use a localized author embedding (LAE) module to excavate author vector representation from semantic and explicit neighbors. Unlike most graph convolutional network (GCN) based methods, the LAE module is able to avoid computing global Laplacian in whole graph by taking limited neighbors. Moreover, the LAE module can also be stacked to absorb more neighbors, which makes our model have high extendibility. Based on the generation process of GRSLA, we also derive a learning algorithm of our model by maximum a posteriori (MAP) estimation. We conduct experiments on the AAN, DBLP and CORD-19 datasets, and the results show that GRSLA model works well than previous global citation recommendation methods
Integrating Graphs with Large Language Models: Methods and Prospects
Get PDFLarge language models (LLMs) such as GPT-4 have emerged as frontrunners,
showcasing unparalleled prowess in diverse applications, including answering
queries, code generation, and more. Parallelly, graph-structured data, an
intrinsic data type, is pervasive in real-world scenarios. Merging the
capabilities of LLMs with graph-structured data has been a topic of keen
interest. This paper bifurcates such integrations into two predominant
categories. The first leverages LLMs for graph learning, where LLMs can not
only augment existing graph algorithms but also stand as prediction models for
various graph tasks. Conversely, the second category underscores the pivotal
role of graphs in advancing LLMs. Mirroring human cognition, we solve complex
tasks by adopting graphs in either reasoning or collaboration. Integrating with
such structures can significantly boost the performance of LLMs in various
complicated tasks. We also discuss and propose open questions for integrating
LLMs with graph-structured data for the future direction of the field
Neural Information Processing
No full textHaishuai Wang (with Peng Zhang, Jia Wu, Shirui Pan) is a contributing author, Mining Top-k Minimal Redundancy Frequent Patterns over Uncertain Databases.
Book introduction:
The four volume set LNCS 9489, LNCS 9490, LNCS 9491, and LNCS 9492 constitutes the proceedings of the 22nd International Conference on Neural Information Processing, ICONIP 2015, held in Istanbul, Turkey, in November 2015. The 231 full papers presented were carefully reviewed and selected from 375 submissions. The 4 volumes represent topical sections containing articles on Learning Algorithms and Classification Systems; Artificial Intelligence and Neural Networks: Theory, Design, and Applications; Image and Signal Processing; and Intelligent Social Networks.
Paper abstract:
Frequent pattern mining from uncertain data has been paid closed attention due to most of the real life databases contain data with uncertainty. Several approaches have been proposed for mining high significance frequent itemsets over uncertain data, however, previous algorithms yield many redundant frequent itemsets and require to set an appropriate user specified threshold which is difficult for users. In this paper, we formally define the problem of top-k minimal redundancy probabilistic frequent pattern mining, which targets to identify top-k patterns with high-significance and low-redundancy simultaneously from uncertain data. We first design uncertain pattern correlation based on Pearson correlation coefficient, which considers pattern uncertainty. Moreover, we present a new algorithm, UTFP, to mine top-k minimal redundancy frequent patterns of length no less than minimum length min_l without setting threshold. We further propose a set of strategies to prune and reduce search space. Experimental results demonstrate that the proposed algorithm achieves good performance in terms of finding top-k frequent patterns with low redundancy on probabilistic data. Our method represents the first research endeavor for probabilistic data based top-k correlated pattern mining.https://digitalcommons.fairfield.edu/engineering-books/1026/thumbnail.jp
GOOD-D: On Unsupervised Graph Out-Of-Distribution Detection
Get PDFMost existing deep learning models are trained based on the closed-world
assumption, where the test data is assumed to be drawn i.i.d. from the same
distribution as the training data, known as in-distribution (ID). However, when
models are deployed in an open-world scenario, test samples can be
out-of-distribution (OOD) and therefore should be handled with caution. To
detect such OOD samples drawn from unknown distribution, OOD detection has
received increasing attention lately. However, current endeavors mostly focus
on grid-structured data and its application for graph-structured data remains
under-explored. Considering the fact that data labeling on graphs is commonly
time-expensive and labor-intensive, in this work we study the problem of
unsupervised graph OOD detection, aiming at detecting OOD graphs solely based
on unlabeled ID data. To achieve this goal, we develop a new graph contrastive
learning framework GOOD-D for detecting OOD graphs without using any
ground-truth labels. By performing hierarchical contrastive learning on the
augmented graphs generated by our perturbation-free graph data augmentation
method, GOOD-D is able to capture the latent ID patterns and accurately detect
OOD graphs based on the semantic inconsistency in different granularities
(i.e., node-level, graph-level, and group-level). As a pioneering work in
unsupervised graph-level OOD detection, we build a comprehensive benchmark to
compare our proposed approach with different state-of-the-art methods. The
experiment results demonstrate the superiority of our approach over different
methods on various datasets.Comment: Accepted by WSDM 2023. 10 pages, 4 figure
Trajectories of fibers and analysis of yarn quality for compact spinning with pneumatic groove
No full textAn Empirical Survey on Long Document Summarization: Datasets, Models and Metrics
No full textLong documents such as academic articles and business reports have been the standard format to detail out important issues and complicated subjects that require extra attention. An automatic summarization system that can effectively condense long documents into short and concise texts to encapsulate the most important information would thus be significant in aiding the reader’s comprehension. Recently, with the advent of neural architectures, significant research efforts have been made to advance automatic text summarization systems, and numerous studies on the challenges of extending these systems to the long document domain have emerged. In this survey, we provide a comprehensive overview of the research on long document summarization and a systematic evaluation across the three principal components of its research setting: benchmark datasets, summarization models, and evaluation metrics. For each component, we organize the literature within the context of long document summarization and conduct an empirical analysis to broaden the perspective on current research progress. The empirical analysis includes a study on the intrinsic characteristics of benchmark datasets, a multi-dimensional analysis of summarization models, and a review of the summarization evaluation metrics. Based on the overall findings, we conclude by proposing possible directions for future exploration in this rapidly growing field.Full Tex
Learning multi-level weight-centric features for few-shot learning
No full textFew-shot learning is currently enjoying a considerable resurgence of interest, aided by the recent advance of deep learning. Contemporary approaches based on weight-generation scheme delivers a straightforward and flexible solution to the problem. However, they did not fully consider both the representation power for unseen categories and weight generation capacity in feature learning, making it a significant performance bottleneck. This paper proposes a multi-level weight-centric feature learning to give full play to feature extractor's dual roles in few-shot learning. Our proposed method consists of two essential techniques: a weight-centric training strategy to improve the features’ prototype-ability and a multi-level feature incorporating a mid- and relation-level information. The former increases the feasibility of constructing a discriminative decision boundary based on a few samples. Simultaneously, the latter helps improve the transferability for characterizing novel classes and preserve classification capability for base classes. We extensively evaluate our approach to low-shot classification benchmarks. Experiments demonstrate our proposed method significantly outperforms its counterparts in both standard and generalized settings and using different network backbones.No Full Tex
Towards Self-Interpretable Graph-Level Anomaly Detection
No full textGraph-level anomaly detection (GLAD) aims to identify graphs that exhibit notable dissimilarity compared to the majority in a collection. However, current works primarily focus on evaluating graph-level abnormality while failing to provide meaningful explanations for the predictions, which largely limits their reliability and application scope. In this paper, we investigate a new challenging problem, explainable GLAD, where the learning objective is to predict the abnormality of each graph sample with corresponding explanations, i.e., the vital subgraph that leads to the predictions. To address this challenging problem, we propose a Self-Interpretable Graph aNomaly dETection model (SIGNET for short) that detects anomalous graphs as well as generates informative explanations simultaneously. Specifically, we first introduce the multi-view subgraph information bottleneck (MSIB) framework, serving as the design basis of our self-interpretable GLAD approach. This way SIGNET is able to not only measure the abnormality of each graph based on cross-view mutual information but also provide informative graph rationales by extracting bottleneck subgraphs from the input graph and its dual hypergraph in a self-supervised way. Extensive experiments on 16 datasets demonstrate the anomaly detection capability and self-interpretability of SIGNET.Yixin Liu, Kaize Ding, Qinghua Lu, Fuyi Li, Leo Yu Zhang, Shirui Pa
E-MIMIC: Empowering Multilingual Inclusive Communication
No full textPreserving diversity and inclusion is becoming a compelling need in both industry and academia. The ability to use appropriate forms of writing, speaking, and gestures is not widespread even in formal communications such as public calls, public announcements, official reports, and legal documents. The improper use of linguistic expressions can foment unacceptable forms of exclusion, stereotypes as well as forms of verbal violence against minorities, including women. Furthermore, existing machine translation tools are not designed to generate inclusive content. The present paper investigates a joint effort of the research communities of linguistics and Deep Learning Natural Language Understanding in fighting against non-inclusive, prejudiced language forms. It presents a methodology aimed at tackling the improper use of language in formal communication, with a particular attention paid to Romanic languages (Italian, in particular). State-of-the-art Deep Language Modeling architectures are exploited to automatically identify non-inclusive text snippets, suggest alternative forms, and produce inclusive text rephrasing. A preliminary evaluation conducted on a benchmark dataset shows promising results, i.e., 85% accuracy in predicting inclusive/non-inclusive communications
One-Shot Neural architecture search via novelty driven sampling
Get PDFOne-Shot Neural architecture search (NAS) has received wide attentions due to its computational efficiency. Most state-of-the-art One-Shot NAS methods use the validation accuracy based on inheriting weights from the supernet as the stepping stone to search for the best performing architecture, adopting a bilevel optimization pattern with assuming this validation accuracy approximates to the test accuracy after re-training. However, recent works have found that there is no positive correlation between the above validation accuracy and test accuracy for these One-Shot NAS methods, and this reward based sampling for supernet training also entails the rich-get-richer problem. To handle this deceptive problem, this paper presents a new approach, Efficient Novelty-driven Neural Architecture Search, to sample the most abnormal architecture to train the supernet. Specifically, a single-path supernet is adopted, and only the weights of a single architecture sampled by our novelty search are optimized in each step to reduce the memory demand greatly. Experiments demonstrate the effectiveness and efficiency of our novelty search based architecture sampling method
- …
