Association for the Advancement of Artificial Intelligence: AAAI Publications
Not a member yet
26155 research outputs found
Sort by
Toward a Generalized Model of Human-AI Team Effectiveness
No full textThis extended abstract proposes a generalized model for evaluating human–AI team effectiveness, capturing its dynamic nature and the complex relationships among contributing variables. While future research may enrich or expand upon the set of variables, the underlying framework is intended to remain conceptually robust
The Need for Human-AI Collaborative Methods for Conducting Audits of Machine Learning Models
No full textConducting application audits of ML models is essential for ensuring their safe and responsible deployment, particularly in high-stakes applications. However, the auditing of ML models deployed in domain-specific applications remains largely a manual process, relying on domain experts to identify model errors. The manual nature of the process limits scalability of audits and hinders the discovery of problematic model behaviors. We posit that a human-AI collaborative paradigm is key to conducting effective application audits. In this abstract, we propose a research agenda to develop Human-AI collaborative methods for conducting application audits of ML models
Enhancing Human-Autonomous System Interaction and Team Dynamics in Automated Driving Systems (ADS)
No full textAs Automated Driving Systems (ADS) advance toward higher levels of automation (SAE Levels 3-5), the role of human drivers is shifting from active control to supervision and intervention. However, traditional human-automation interaction frameworks do not fully account for the dynamic team-based coordination required for effective ADS integration. Poor adaptability and coordination between drivers and ADS can result in critical safety failures, as seen in real-world incidents. This research focuses on human-autonomy teaming (HATs) in ADS, emphasizing collaborative adaptation, shared decision-making, and real-time interaction metrics to enhance safety and user experience. By incorporating team cognition theory and layered dynamical models, this research aims to develop human-machine teaming metrics that facilitate adaptive and context-aware cooperation. Key research questions include: (1) How do human drivers and ADS dynamically interact? (2) What team cognition metrics effectively measure human-ADS adaptability? (3) How can real-time analytics improve driver-ADS collaboration and safety
AI Agents in Education: Four Trends and a Practical Workflow
No full textAI agents based on large language models (LLMs) are transforming education by offering personalized tutoring, faster grading, innovative content generation, and streamlined administrative tasks. This paper explores four major trends of AI agents in the education industry: tutoring and mentoring, automated grading, content creation, and administrative automation. This study also introduces a common AI agent prototype, focusing on customization through fine-tuning and retrieval-augmented generation (RAG), and highlights the role of external tools in enhancing versatility. By reviewing current applications and technical methods, this work demonstrates the growing role of AI in improving instructional quality and operational efficiency
HurriCast: Synthetic Tropical Cyclone Track Generation for Hurricane Forecasting
No full textThe generation of synthetic tropical cyclone tracks for Risk assessment is a critical application of preparedness for the impacts of climate change and disaster relief, particularly in North America. Insurance companies use these synthetic tracks to estimate the potential risks and financial impacts of future tropical cyclones. For governments and policymakers, understanding the potential impacts of tropical cyclones helps in developing effective emergency response strategies, updating building codes, and prioritizing investments in resilience and mitigation projects. In this study, many hypothetical but plausible TC scenarios are created based on historical TC data HURDAT2 (HURricane DATa 2nd generation). A hybrid methodology, combining the ARIMA and K-MEANS methods with Autoencoder, is employed to capture better historical TC behaviors and project future trajectories and intensities. It demonstrates an efficient and reliable in the field of climate modeling and risk assessment. By effectively capturing past hurricane patterns and providing detailed future projections, this approach not only validates the reliability of this method but also offers crucial insights for a range of applications, from disaster preparedness and emergency management to insurance risk analysis and policy formulation
GenAI at the Edge: Comprehensive Survey on Empowering Edge Devices
No full textGenerative Artificial Intelligence (GenAI) applies models and algorithms such as Large Language Model (LLM) and Foundation Model (FM) to generate new data. GenAI, as a promising approach, enables advanced capabilities in various applications, including text generation and image processing. In current practice, GenAI algorithms run mainly on the cloud server, leading to high latency and raising security concerns. Consequently, these challenges encourage the deployment of GenAI algorithms directly on edge devices. However, the large size of such models and their significant computational resource requirements pose obstacles when deploying them in resource-constrained systems. This survey provides a comprehensive overview of recent proposed techniques that optimize GenAI for efficient deployment on resource-constrained edge devices. For this aim, this work highlights three main categories for bringing GenAI to the edge: software optimization, hardware optimization, and frameworks. The main takeaways for readers of this survey will be a clear roadmap to design, implement, and refine GenAI systems for real-world implementation on edge devices
LLM-ACTR: from Cognitive Models to LLMs in Manufacturing Solutions
No full textUsing off-the-shelf large language models (LLMs) in manufacturing decision-making often results in broad but noisy behavior. Previous approaches that employ LLMs for decision-making struggle with complex reasoning tasks that require deliberate cognition over fast and intuitive inference. These approaches often report issues related to insufficient grounding, such as human-level but unhuman-like behaviors. In the present paper, we toward addressing this gap and ask whether language models can learn from cognitive models for human-like decisions. We introduce VSM-ACTR 2.0, an ACT-R cognitive model for manufacturing solutions, and LLM-ACTR, a developing framework for knowledge transfer from cognitive models to language models. The ACT-R cognitive architecture is designed to computationally model the internal mechanisms of human cognitive decision-making. LLM-ACTR extracts knowledge from ACT-R’s internal decision-making processes, represents it as latent neural representations, and injects this content vector into trainable LLM adapter layers. It then fine-tunes the LLMs for downstream decision-making predictions. We find that, after fine-tuning and adding the content vector to the activations during the LLM forward pass, the LLM offers better representations of human decision-making behaviors on a novel Design for Manufacturing problem, compared to an LLM-only model that employs chain-of-thought reasoning strategies. Taken together, the results open up new research directions for equipping LLMs with the necessary knowledge to computationally model and replicate the internal mechanisms of human cognitive decision-making
DELTA: Pre-Train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment
No full textRecent research demonstrates the effectiveness of using pre-trained language models for legal case retrieval. Most of the existing works focus on improving the representation ability for the contextualized embedding of the [CLS] token and calculate relevance using textual semantic similarity. However, in the legal domain, textual semantic similarity does not always imply that the cases are relevant enough. Instead, relevance in legal cases primarily depends on the similarity of key facts that impact the final judgment. Without proper treatments, the discriminative ability of learned representations could be limited since legal cases are lengthy and contain numerous non-key facts.
To this end, we introduce DELTA, a discriminative model designed for legal case retrieval. The basic idea involves pinpointing key facts in legal cases and pulling the contextualized embedding of the [CLS] token closer to the key facts while pushing away from the non-key facts, which can warm up the case embedding space in an unsupervised manner. To be specific, this study brings the word alignment mechanism to the contextual masked auto-encoder. First, we leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Second, we employ the deep decoder to enable ``translation'' between different structures, with the goal of pinpointing key facts to enhance discriminative ability. Comprehensive experiments conducted on publicly available legal benchmarks show that our approach can outperform existing state-of-the-art methods in legal case retrieval.
It provides a new perspective on the in-depth understanding and processing of legal case documents
Risk Controlled Image Retrieval
No full textMost image retrieval research prioritizes improving predictive performance, often overlooking situations where the reliability of predictions is equally important. The gap between model performance and reliability requirements highlights the need for a systematic approach to analyze and address the risks associated with image retrieval. Uncertainty quantification technique can be applied to mitigate this issue by assessing uncertainty for retrieval sets, but it provides only a heuristic estimate of uncertainty rather than a guarantee. To address these limitations, we present Risk Controlled Image Retrieval (RCIR), which generates retrieval sets with coverage guarantee, i.e., retrieval sets that are guaranteed to contain the true nearest neighbors with a predefined probability. RCIR can be easily integrated with existing uncertainty-aware image retrieval systems, agnostic to data distribution and model selection. To the best of our knowledge, this is the first work that provides coverage guarantees to image retrieval. The validity and efficiency of RCIR are demonstrated on four real-world datasets: CAR-196, CUB-200, Pittsburgh, and ChestX-Det
LEGEND: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets
No full textThe success of the reward model in distinguishing between responses with subtle safety differences depends critically on the high-quality preference dataset, which should capture the fine-grained nuances of harmful and harmless responses. This motivates the need to develop the datasets involving preference margins, which accurately quantify how harmless one response is compared to another. In this paper, we take the first step to propose an effective and cost-efficient framework to promote the margin-enhanced preference dataset development. Our framework, Legend, Leverages rEpresentation enGineering to annotate preferENce Datasets. It constructs the specific direction within the LLM's embedding space that represents safety. By leveraging this safety direction, Legend can then leverage the semantic distances of paired responses along this direction to annotate margins automatically. We experimentally demonstrate our effectiveness in both reward modeling and harmless alignment for LLMs. Legend also stands out for its efficiency, requiring only the inference time rather than additional training. This efficiency allows for easier implementation and scalability, making Legend particularly valuable for practical applications in aligning LLMs with safe conversations