1,721,047 research outputs found
Benchmarking Sensitivity of Continual Graph Learning for Skeleton-Based Action Recognition
Full text linkContinual learning (CL) is the research field that aims to build machine
learning models that can accumulate knowledge continuously over different tasks
without retraining from scratch. Previous studies have shown that pre-training
graph neural networks (GNN) may lead to negative transfer (Hu et al., 2020)
after fine-tuning, a setting which is closely related to CL. Thus, we focus on
studying GNN in the continual graph learning (CGL) setting. We propose the
first continual graph learning benchmark for spatio-temporal graphs and use it
to benchmark well-known CGL methods in this novel setting. The benchmark is
based on the N-UCLA and NTU-RGB+D datasets for skeleton-based action
recognition. Beyond benchmarking for standard performance metrics, we study the
class and task-order sensitivity of CGL methods, i.e., the impact of learning
order on each class/task's performance, and the architectural sensitivity of
CGL methods with backbone GNN at various widths and depths. We reveal that
task-order robust methods can still be class-order sensitive and observe
results that contradict previous empirical observations on architectural
sensitivity in CL.Comment: This work is accepted at VISAPP 2024 as a short pape
Pre-trained word embeddings for goal-conditional transfer learning in reinforcement learning
No full text
Policy Compression for Intelligent Continuous Control on Low-Power Edge Devices
Full text linkInterest in deploying deep reinforcement learning (DRL) models on low-power edge devices, such as Autonomous Mobile Robots (AMRs) and Internet of Things (IoT) devices, has seen a significant rise due to the potential of performing real-time inference by eliminating the latency and reliability issues incurred from wireless communication and the privacy benefits of processing data locally. Deploying such energy-intensive models on power-constrained devices is not always feasible, however, which has led to the development of model compression techniques that can reduce the size and computational complexity of DRL policies. Policy distillation, the most popular of these methods, can be used to first lower the number of network parameters by transferring the behavior of a large teacher network to a smaller student model before deploying these students at the edge. This works well with deterministic policies that operate using discrete actions. However, many real-world tasks that are power constrained, such as in the field of robotics, are formulated using continuous action spaces, which are not supported. In this work, we improve the policy distillation method to support the compression of DRL models designed to solve these continuous control tasks, with an emphasis on maintaining the stochastic nature of continuous DRL algorithms. Experiments show that our methods can be used effectively to compress such policies up to 750% while maintaining or even exceeding their teacher’s performance by up to 41% in solving two popular continuous control tasks
Hierarchical Reinforcement Learning: A Survey and Open Research Challenges
Full text linkReinforcement learning (RL) allows an agent to solve sequential decision-making problems by interacting with an environment in a trial-and-error fashion. When these environments are very complex, pure random exploration of possible solutions often fails, or is very sample inefficient, requiring an unreasonable amount of interaction with the environment. Hierarchical reinforcement learning (HRL) utilizes forms of temporal- and state-abstractions in order to tackle these challenges, while simultaneously paving the road for behavior reuse and increased interpretability of RL systems. In this survey paper we first introduce a selection of problem-specific approaches, which provided insight in how to utilize often handcrafted abstractions in specific task settings. We then introduce the Options framework, which provides a more generic approach, allowing abstractions to be discovered and learned semi-automatically. Afterwards we introduce the goal-conditional approach, which allows sub-behaviors to be embedded in a continuous space. In order to further advance the development of HRL agents, capable of simultaneously learning abstractions and how to use them, solely from interaction with complex high dimensional environments, we also identify a set of promising research directions
Feasibility of 100% renewable energy-based electricity production for cities with storage and flexibility
No full text- …
