Association for the Advancement of Artificial Intelligence: AAAI Publications
Not a member yet
26155 research outputs found
Sort by
G2LDetect: A Global-to-Local Approach for Hallucination Detection
No full textHallucination detection has attracted considerable interest due to the tendency of language models to generate texts that contain hallucinations. Most existing methods start with specific local details directly extracted from text, then aggregate to form the final conclusion.
However, this direct extraction approach ignores the global context, leading to isolated details, and is prone to missed or over-detections.
In this paper, we present a global-to-local approach for hallucination detection (G2LDetect), which considers the global information of the text before identifying local details.
We first construct a global representation of the text by transforming it into a hierarchical tree structure.
Afterward, we obtain specific local details from the global tree representation using path-wise identification and perform detection on them.
This global-to-local detection process ensures that local details are context-aware and complete, thus making more accurate and reliable detection results.
Experimental results show that our global-to-local method outperforms existing methods, especially for longer texts
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition - And Ways to Overcome Them
No full textCross-modal contrastive pre-training between natural language and other modalities, e.g., vision and audio, has demonstrated astonishing performance and effectiveness across a diverse variety of tasks and domains. In this paper, we investigate whether such natural language supervision can be used for wearable sensor based Human Activity Recognition (HAR), and discover that--surprisingly--it performs substantially worse than standard end-to-end training and self-supervision. We identify the primary causes for this as: sensor heterogeneity and the lack of rich, diverse text descriptions of activities. To mitigate their impact, we also develop strategies and assess their effectiveness through an extensive experimental evaluation. These strategies lead to significant increases in activity recognition, bringing performance closer to supervised and self-supervised training, while also enabling the recognition of unseen activities and cross modal retrieval of videos. Overall, our work paves the way for better sensor-language learning, ultimately leading to the development of foundational models for HAR using wearables
IDseq: Decoupled and Sequentially Detecting and Grounding Multi-Modal Media Manipulation
No full textDetecting and grounding multi-modal media manipulation
aims to categorize the type and localize the region of manipulation for image-text pairs in both two modalities. Existing
methods have not sufficiently explored the intrinsic properties of the manipulated images, which contain both forgery and content features, leading to inefficient utilization. To address this problem, we propose an Image-Driven Decoupled
Sequential Framework (IDseq), designed to decouple image
features and rationally integrate them to accomplish different
sub-tasks effectively. Specifically, IDseq employs two specially designed disentangled losses to guide the disentangled
learning of forgery and content features. To efficiently leverage these features, we propose a Decoupled Image Manipulation Decoder (DIMD) that processes image tasks within a decoupled schema. We mitigate their exclusive competition by
separating the image tasks into forgery-relevant and content-relevant components and training them without gradient interaction. Additionally, we utilize content features enhanced by
the proposed Manipulation Indicator Generator (MIG) for the
text tasks, which provide the maximal visual information as a
reference while eliminating interference from unverified image data. Extensive experiments show the superiority of our
IDseq, where it notably outperforms SOTA methods on the
fine-grained classification by 3.8% in mAP and the forgery
face grounding by 8.7% in IoUmean, even 1.3% in F1 on the
most challenging manipulated text grounding
StressPrompt: Does Stress Impact Large Language Models and Human Performance Similarly?
No full textHuman beings often experience stress, which can significantly influence their performance. This study explores whether Large Language Models (LLMs) exhibit stress responses similar to those of humans and whether their performance fluctuates under different stress-inducing prompts. To investigate this, we developed a novel set of prompts, termed StressPrompt, designed to induce varying levels of stress. These prompts were derived from established psychological frameworks and carefully calibrated based on ratings from human participants. We then applied these prompts to several LLMs to assess their responses across a range of tasks, including instruction-following, complex reasoning, and emotional intelligence. The findings suggest that LLMs, like humans, perform optimally under moderate stress, consistent with the Yerkes-Dodson law. Notably, their performance declines under both low and high-stress conditions. Our analysis further revealed that these StressPrompts significantly alter the internal states of LLMs, leading to changes in their neural representations that mirror human responses to stress. This research provides critical insights into the operational robustness and flexibility of LLMs, demonstrating the importance of designing AI systems capable of maintaining high performance in real-world scenarios where stress is prevalent, such as in customer service, healthcare, and emergency response contexts. Moreover, this study contributes to the broader AI research community by offering a new perspective on how LLMs handle different scenarios and their similarities to human cognition
Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules
No full textRecent advancements have underscored the exceptional analytical and situational understanding capabilities of Large Language Models (LLMs) in autonomous driving decisions. However, the inherent hallucination issues of LLMs pose significant safety concerns when utilized as standalone decision-making systems. To address these challenges, we propose the Hybrid-Driving framework, which leverages LLMs' situational comprehension and reasoning abilities alongside the specialized driving expertise embedded in knowledge graphs and driving rules, thereby enhancing the safety, robustness, and reliability of autonomous driving decisions. To articulate driving experiences clearly, we introduce the Scenario Evolution Knowledge Graph (SEKG), which integrates scenario prediction and action risk analysis in autonomous driving. By delineating observation areas and defining Time-to-Collision (TTC) levels, we effectively control the number of driving scenario nodes and ensure scenario diversity. Based on the scenario evolution relationships within the SEKG, we predict scenarios and assess associated action risks. Additionally, we implement a rule-filtering mechanism to eliminate unreasonable actions and employ prompt engineering to integrate scenario information, optional actions, and SEKG-based action risk analysis into the LLMs for decision-making. Extensive experiments demonstrate that our approach substantially improves decision success rates compared to using LLMs alone (≥37.5%), as well as surpasses the DiLu framework with LLMs and few-shot driving memory (≥7.5%), and other reinforcement learning methods (≥11%). These results validate the effectiveness of the Hybrid-Driving framework in enhancing LLM reliability for autonomous driving and advocate for its broader application of domain-specific knowledge across other fields
FMPM-DNet: Hyperspectral Pansharpening Dynamic Network Based on Feature Modulation and Probability Mask
No full textCurrently, most Hyperspectral (HS) pansharpening methods have two problems, namely the lack of consideration the spatial variations of HS images and inaccurate feature reconstruction in multi-channel complex mapping relationships, leading to spectral and spatial distortions in the fusion results. To address these issues, we propose a dynamic network based on feature modulation and probability mask (FMPM-DNet) for HS pansharpening, including two stages of spectral-spatial feature modulation and feature reconstruction. In the first stage, to increase the feature representation ability of the model, a wave function is defined based on complex transformation to convert spatial features into wave-like features. On this basis, considering the spatial variations of HS images, a dynamic feature modulation unit (DFMU) is constructed to achieve adaptive modulation and coarse fusion of features by dynamically generating spectral-spatial correction matrix. In the second stage, a feature probability mask unit (FPMU) is designed to realize global feature embedding at different depths and local feature embedding at the same depth to obtain refined fused features. Extensive experiments on three widely used datasets demonstrate that the proposed FMPM-Net achieves significant improvements in both spatial and spectral quality metrics compared to some state-of-the-art (SOTA) methods
Generalizable Sensor-Based Activity Recognition via Categorical Concept Invariant Learning
No full textHuman Activity Recognition (HAR) aims to recognize activities by training models on massive sensor data. In real-world deployment, a crucial aspect of HAR that has been largely overlooked is that the test sets may have different distributions from training sets due to inter-subject variability including age, gender, behavioral habits, etc., which leads to poor generalization performance. One promising solution is to learn domain-invariant representations to enable a model to generalize on an unseen distribution. However, most existing methods only consider the feature-invariance of the penultimate layer for domain-invariant learning, which leads to suboptimal results. In this paper, we propose a Categorical Concept Invariant Learning (CCIL) framework for generalizable activity recognition, which introduces a concept matrix to regularize the model in the training stage by simultaneously concertrating on feature-invariance and logit-invariance. Our key idea is that the concept matrix for samples belonging to the same activity category should be similar. Extensive experiments on four public HAR benchmarks demonstrate that our CCIL substantially outperforms the state-of-the-art approaches under cross-person, cross-dataset, cross-position, and one-person-to-another settings
RaCMC: Residual-Aware Compensation Network with Multi-Granularity Constraints for Fake News Detection
No full textMultimodal fake news detection aims to automatically identify real or fake news, thereby mitigating the adverse effects caused by such misinformation. Although prevailing approaches have demonstrated their effectiveness, challenges persist in cross-modal feature fusion and refinement for classification. To address this, we present a residual-aware compensation network with multi-granularity constraints (RaCMC) for fake news detection, that aims to sufficiently interact and fuse cross-modal features while amplifying the differences between real and fake news. First, a multiscale residual-aware compensation module is designed to interact and fuse features at different scales, and ensure both the consistency and exclusivity of feature interaction, thus acquiring high-quality features. Second, a multi-granularity constraints module is implemented to limit the distribution of both the news overall and the image-text pairs within the news, thus amplifying the differences between real and fake news at the news and feature levels. Finally, a dominant feature fusion reasoning module is developed to comprehensively evaluate news authenticity from the perspectives of both consistency and inconsistency. Experiments on three public datasets, including Weibo17, Politifact and GossipCop, reveal the superiority of the proposed method
Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization
No full textImage forgery detection and localization (IFDL) is of vital importance as forged images can spread misinformation that poses potential threats to our daily life. However, previous methods still struggled to effectively handle forged images processed with diverse forgery operations in real-world scenarios. In this paper, we propose a novel Reinforced Multi-teacher Knowledge Distillation (Re-MTKD) framework for the IFDL task, structured around an encoder-decoder ConvNeXt-UperNet along with Edge-Aware Module, named Cue-Net. First, three Cue-Net models are separately trained for the three main types of image forgeries, i.e., copy-move, splicing and inpainting, which then serve as the multi-teacher models to train the target student model with Cue-Net through self-knowledge distillation. A Reinforced Dynamic Teacher Selection (Re-DTS) strategy is developed to dynamically assign weights to the involved teacher models, which facilitates specific knowledge transfer and enables the student model to effectively learn both the common and specific natures of diverse tampering traces. Extensive experiments demonstrate that, compared with other state-of-the-art methods, the proposed method achieves superior performance on several recently emerged datasets comprised of various kinds of image forgeries
Symbolic Functional Decomposition: A Reconfiguration Approach
No full textFunctional decomposition is the process of breaking down a function f into a composition f=g(f_1,...,f_k) of simpler functions f_1,...,f_k belonging to some class F. This fundamental notion can be used to model applications arising in a wide variety of contexts, ranging from machine learning to formal language theory. In this work, we study functional decomposition by leveraging on the notion of functional reconfiguration. In this setting, constraints are imposed not only on the factor functions f_1,...,f_k but also on the intermediate functions arising during the composition process.
We introduce a symbolic framework to address functional reconfiguration and decomposition problems. In our framework, functions arising during the reconfiguration process are represented symbolically, using ordered binary decision diagrams (OBDDs). The function g used to specify the reconfiguration process is represented by a Boolean circuit C. Finally, the function class F is represented by a second-order finite automaton A. Our main result states that functional reconfiguration, and hence functional decomposition, can be solved in fixed-parameter linear time when parameterized by the width of the input OBDD, by structural parameters associated with the reconfiguration circuit C, and by the size of the second-order finite automaton A