6 research outputs found
Deep Learning Methods for Socially-Aware Human Trajectory Forecasting
No full textThe ability to forecast human motion, called ``human trajectory forecasting", is a critical requirement for mobility applications such as autonomous driving and robot navigation. Humans plan their path taking into account what might happen in the future. Similarly, the decision-making algorithm of autonomous systems should predict how their environment will evolve in the future. This thesis focuses on developing deep learning methods for forecasting human motion.
In the first part of this thesis, we tackle the fundamental challenges of social interaction modelling and multimodality. Social interactions dictate how the motion of a human is affected by others. Current deep learning methods often struggle to model these interactions between trajectory sequences. To promote interaction-awareness in forecasting models, we develop a training paradigm that explicitly focuses on samples that undergo interactions and incorporates model uncertainty. Furthermore, we build a taxonomy of existing interaction encoders and propose an optimal design that is robust to the real-world noise. In addition to modelling interactions, a good trajectory forecasting model must account for the multimodal nature of the prediction, i.e., the possibility of having multiple plausible futures given the past observations. To tackle multimodality, we present a socially-aware generative adversarial network that leverages recent advances in sequence modelling, and has the ability to model the temporal evolution of social interactions. Furthermore, we develop a collaborative sampling technique that refines the bad generated predictions at test time.
In the second part of this thesis, we focus on two challenges specific to the real-world deployment of forecasting models: interpretability and adaptability. While neural networks have the capacity to learn complex interactions, it is difficult to understand the reason behind their predictions. Thus, we develop a framework that combines the interpretability of the classical models with the predictive power of neural networks. With regards to adaptability, existing deep forecasting models suffer from inferior performance when they encounter novel scenarios. We develop a strategy to adapt a pre-trained forecasting model to a target domain using limited samples. In particular, we introduce motion style adapters that identify and adjust the target domain-specific features. Throughout this thesis, experiments on synthetic and real-world forecasting datasets validate the effectiveness of our proposed methods.VIT
Safety-Compliant Generative Adversarial Networks for Human Trajectory Forecasting
No full textHuman trajectory forecasting in crowds presents the challenges of modelling social interactions and outputting collision-free multimodal distribution. Following the success of Social Generative Adversarial Networks (SGAN), recent works propose various GAN-based designs to better model human motion in crowds. Despite superior performance in reducing distance-based metrics, current networks fail to output socially acceptable trajectories, as evidenced by high collisions in model predictions. To counter this, we introduce SGANv2: an improved safety-compliant SGAN architecture equipped with spatio-temporal interaction modelling and a transformer-based discriminator. The spatio-temporal modelling ability helps to learn the human social interactions better while the transformer-based discriminator design improves temporal sequence modelling. Additionally, SGANv2 utilizes the learned discriminator even at test-time via a collaborative sampling strategy that not only refines the colliding trajectories but also prevents mode collapse, a common phenomenon in GAN training. Through extensive experimentation on multiple real-world and synthetic datasets, we demonstrate the efficacy of SGANv2 to provide socially-compliant multimodal trajectories.VIT
Collaborative Sampling in Generative Adversarial Networks
No full textThe standard practice in Generative Adversarial Networks (GANs) discards the discriminator during sampling. However, this sampling method loses valuable information learned by the discriminator regarding the data distribution. In this work, we propose a collaborative sampling scheme between the generator and the discriminator for improved data generation. Guided by the discriminator, our approach refines the generated samples through gradient-based updates at a particular layer of the generator, shifting the generator distribution closer to the real data distribution. Additionally, we present a practical discriminator shaping method that can smoothen the loss landscape provided by the discriminator for effective sample refinement. Through extensive experiments on synthetic and image datasets, we demonstrate that our proposed method can improve generated samples both quantitatively and qualitatively, offering a new degree of freedom in GAN sampling.VIT
Interpretable Social Anchors for Human Trajectory Forecasting in Crowds
No full textHuman trajectory forecasting in crowds, at its core, is a sequence prediction problem with specific challenges of capturing inter-sequence dependencies (social interactions) and consequently predicting socially-compliant multimodal distributions. In recent years, neural network-based methods have been shown to outperform hand-crafted methods on distance-based metrics. However, these data-driven methods still suffer from one crucial limitation: lack of interpretability. To overcome this limitation, we leverage the power of discrete choice models to learn interpretable rule-based intents, and subsequently utilise the expressibility of neural networks to model scene-specific residual. Extensive experimentation on the interaction-centric benchmark TrajNet++ demonstrates the effectiveness of our proposed architecture to explain its predictions without compromising the accuracy.VIT
Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting
No full textDeep motion forecasting models have achieved great success when trained on a massive amount of data. Yet, they often perform poorly when training data is limited. To address this challenge, we propose a transfer learning approach for efficiently adapting pre-trained forecasting models to new domains, such as unseen agent types and scene contexts. Unlike the conventional fine-tuning approach that updates the whole encoder, our main idea is to reduce the amount of tunable parameters that can precisely account for the target domain-specific motion style. To this end, we introduce two components that exploit our prior knowledge of motion style shifts: (i) a low-rank motion style adapter that projects and adjusts the style features at a low-dimensional bottleneck; and (ii) a modular adapter strategy that disentangles the features of scene context and motion history to facilitate a fine-grained choice of adaptation layers. Through extensive experimentation, we show that our proposed adapter design, coined MoSA, outperforms prior methods on several forecasting benchmarks.VIT
