17 research outputs found
Context-based Cyclist Path Prediction: Crafted and Learned Models for Intelligent Vehicles
No full textThis thesis addresses the problem of path prediction for cyclists.Instead of solely focusing on how to predict the future trajectory based on previous position measurements, this thesis investigates how to leverage additional contextual information that can inform on the future intent of cyclists.This thesis does this with the application of intelligent vehicles in mind. That means all measurements come from the point of view of a vehicle on the road.Additionally, the resulting predictions must be usable by a motion planner. In practice, this means the predictions are a probability distribution over the future position rather than a single point in space.This thesis starts with an investigation of one of the modules that allow path prediction in the first place: 3D object detection. Two existing state-of-the-art 3D object detectors that exploit Lidar data are evaluated beyond the standard metrics of 3D object detection.3D object detectors predict an oriented 3D bounding box. The standard metric determines a correct detection based on the accuracy of the position, extent, and orientation of the bounding box all at once.By loosening the requirements for when a detection is considered correct, the accuracy of the estimated position, extent, and orientation can be evaluated separately.The results show that a large number of detections are considered incorrect largely because of inaccurate bounding box extent rather than bounding box position, which is arguably a more important aspect for path prediction. As a result, the performance of these 3D object detectors when used for path prediction can be considered to be higher than what the common metrics suggest.After this, this thesis investigates how knowledge of the road topology can be used to improve the accuracy of cyclist path prediction. The trajectories of cyclists near an intersection are extracted from a naturalistic cyclist detection dataset.These are categorized and grouped based on the action taken by each cyclist (hard left/right, slight left/right, or straight). A Linear Dynamical System (LDS) is fitted on each group. These LDSs are used together to create a Mixture of Linear Dynamical Systems (MoLDS). During online inference, the relative probability of each underlying LDS allows the MoLDS to evaluate which direction the cyclist is most likely to take. This chapter demonstrates that the highest prediction accuracy is obtained when this model is additionally given prior knowledge on which directions are available for the cyclist to take.Next, context cues related to a specific scenario are considered.In the scenario, a cyclist in front of the ego-vehicle approaches an intersection and has the option to either continue straight or turn left. The three context cues considered are the distance of the cyclist to the intersection, whether the cyclist is raising their arm, and the criticality of the situation. This last context cue is based on the time it will take the ego-vehicle to overtake the cyclist: the lower this is, the more risk a left turn brings.This scenario is first modeled with a Switching Linear Dynamical System (SLDS) with two motion models that represent "cycling straight" and "turning left", respectively. This model does not yet use any context cues.Still, the SLDS is shown to outperform a baseline model that represents the scenario with a single motion model.By letting the context cues inform the SLDS whether switching from one motion model to the other is likely to happen the performance is increased even further. The resulting model is referred to as a Dynamic Bayesian Network (DBN).The context-based path prediction methods described so far have been designed with specific motion models and interplay of context cues in mind: the overall state representation has been hand-crafted. The advantage of this approach is that the state representation is then interpretable, making it easy to understand why a model predicts what it does, even when it fails to predict something correctly.However, methods with a learned state representation often attain higher performances.The next point of investigation of this thesis is then to compare a model with a crafted state representation to a model with a learned one. Specifically, the DBN is compared to a Recurrent Neural Network (RNN), using the cyclist scenario from before.To level the playing field as much as possible two actions are taken. First, the contextual cues are supplied to the RNN as well, and experiments assert that the performance of the RNN does in fact improve when it incorporates these cues.Secondly, the optimization method used in the RNN is applied to the DBN as well, but in such a way that the interpretation of its crafted state representation remains the same.Of the two methods, the RNN attains the highest performance. Still, optimizing the DBN largely closes the performance gap between the two.Finally, this thesis determines whether the DBN is not only performant but also useful in practice: it is integrated in an intelligent vehicle. The cyclist scenario is performed live, in which the intelligent vehicle extracts the relevant context cues directly from sensor data. The resulting predictions are used to create an early warning system for the driver, to warn them if the cyclist intends to turn left.The model is also used for predictions in an autonomously driving intelligent vehicle, but due to safety reasons on a different scenario that contains comparable contextual cues.An automated dummy plays the role of a pedestrian on the sidewalk who walks towards the curbside in order to cross the road. The intelligent vehicle is driving on this road towards the pedestrian and has right of way.In this scenario, a pedestrian is only expected to cross the road if they are unaware of the approaching vehicle.Furthermore, if they will stop, they are expected to only stop at the curbside.The intelligent vehicle determines whether the pedestrian is aware of it by estimating the head orientation of the pedestrian. Additionally, it measures the distance between the pedestrian and the curbside, and predicts the future trajectory of the pedestrian accordingly.With the model in place, the vehicle can autonomously follow a planned trajectory and evade the pedestrian if the pedestrian does indeed cross the road.The real-world experiments confirm the feasibility of the system. By evaluating the entire pipeline at once, from detections to motion planning, this chapter is able to propose future work that bridges these various disciplines and shows what intelligent vehicles can already realistically achieve.Intelligent Vehicle
Optimizing the Uncertainty Bounds for a Robust Control Problem using Moving Horizon Estimation
No full textA classical robust control problem based on randomized algorithms assumes a probability distribution over the uncertainty region to get its controller. This thesis shows that the measurements from the output signal can help to update and improve the probability distribution in a closed loop situation without the requirement of additional input. The probability distribution is updated by repeatedly taking samples from the uncertainty region and computing the probability of these samples using moving horizon estimation. Moving horizon estimation is used as the basis because it allows for constraints to be active on the system, at the cost of solving many convex programming problems every time step. If there are no constraints, moving horizon estimation reduces to a Kalman filter. The updated probability distribution leads to the creation of a new robust controller with an improved performance over the original controller.Delft Center for Systems and ControlMechanical, Maritime and Materials Engineerin
Crafted vs. Learned Representations in Predictive Models - A Case Study on Cyclist Path Prediction
No full textThis paper compares two models for context-based path prediction of objects with switching dynamics: a Dynamic Bayesian Network (DBN) and a Recurrent Neural Network (RNN). These models are instances of two larger model categories, distinguished by whether expert knowledge is explicitly crafted into the state representation (and thus is interpretable) or whether the representation is learned from data, respectively. Both have shown state-of-the-art performance in previous work. In order to provide a fair comparison, we ensure that both models are treated similarly with respect to the use of context cues and parameter estimation. Specifically, we describe (1) how to integrate the context cues (used previously by the DBN) into the RNN, and (2) how to optimize the DBN with back-propagation similar to the RNN, while keeping an interpretable state representation. Experiments are performed on a scenario where a cyclist might turn left at an intersection in front of the ego-vehicle. Results show that the RNN successfully leverages the context cues, and that optimizing the DBN improves its performance with respect to existing work. While the RNN outperforms the optimized DBN in predictive log-likelihood by a significant margin, both models attain similar average Euclidean distance errors (23-39 cm for DBN and 31-34 cm for RNN, predicting 1 s ahead).Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Intelligent Vehicle
Using road topology to improve cyclist path prediction
No full textWe learn motion models for cyclist path prediction on real-world tracks obtained from a moving vehicle, and propose to exploit the local road topology to obtain better predictive distributions. The tracks are extracted from the Tsinghua-Daimler Cyclist Benchmark for cyclist detection, and corrected for vehicle egomotion. Tracks are then spatially aligned to local curves and crossings in the road. We study a standard approach for path prediction in the literature based on Kalman Filters, as well as a mixture of specialized filters related to specific road orientations at junctions. Our experiments demonstrate an improved prediction accuracy (up to 20% on sharp turns) of mixing specialized motion models for canonical directions, and prior knowledge on the road topology. The new track data complements the existing video, disparity and annotation data of the original benchmark, and will be made publicly available.Accepted Author ManuscriptIntelligent Vehicle
3D Object Detection For Intelligent Vehicles
No full textThis master thesis presents an experimental study on 3D person localization (i.e., pedestrians, cyclists)in traffic scenes, using monocular vision and Light Detection And Ranging (LiDAR) data. The performance of two top-ranking methods is analyzed on the 3D object detection KITTI dataset. In this evaluation, the effect of the Intersection over Union (IoU) threshold on the performance in terms of 3D bounding box location, size, and orientation is analysed.Since the KITTI 3D object detection dataset contains relatively few 3D person instances, the analysis will is to the EuroCity Persons 2.5D (ECP2.5D) datasets (both day and night), which is one order of magnitude larger. Using both datasets, additional experiments are performed to evaluate the influence of distance, the number of LiDAR points, occlusion, and intensity on the performance. Domain transfer experiments between the KITTI and ECP2.5D datasets are performed, to examine how these datasets generalize with respect to each other. Furthermore, Part-A2 net is used to evaluate the detection score which is given to the ground truth pedestrians. The relationship between the detection score and the distance, the number of LiDAR points, and occlusion is analyzed. Some objects are not detected although their ground truth detection score is high. This creates the potential to detect these pedestrians. Lastly, this thesis presents a method that uses the detections from the previous frame to increase the performance in the subsequent frame by adding the previous detections to the 3D proposals coming from the Region Proposal Network (RPN).During this master thesis, two articles were written. The first article will be presented at the 2020 IEEE Intelligent Vehicles Symposium (IV2020) titled “An Experimental Study on 3D Person Localization in Traffic Scenes” by Joram R. van der Sluis, Ewoud A.I. Pool, and Dariu M. Gavrila. The second article will be submitted to IEEE Transactions on Intelligent Vehicles (T-IV) titled “An Experimental Study on 3D Person Localization in Traffic Scenes” by Joram R. van der Sluis, Ewoud A.I. Pool, and Dariu M. Gavrila. This second article is an extension of the first article. Since this article is not submitted yet, it could be that the appended version will contain some differences regarding the final submitted version.Mechanical Engineering | Vehicle Engineerin
Predicting pedestrian path using optical flow as context cue
No full textA human driver can gauge the intention and signals given by other road users indicative of their future behaviour. The intentions and signals are identified by looking at the cues originating from vulnerable road users or their surroundings (hand signals, head orientation, posture, traffic signals, distance to curb, etc.). Taking all these cues into account by creating a separate detector for each is an extremely difficult task. Instead, this MSc Thesis will explore the possibility of using a generic contextual cue in optical flow originating from a pedestrian with deep learning methods to improve the path prediction in a naturalistic driving scenario. The contribution of this work is to examine multiple ways to extract relevant information from the optical flow and also explore the possibility of using the entire the high-dimensional optical flow using convolutions and soft-attention to help identify relevant pixels for the prediction task. This work elaborates on the extraction and processing of optical flow features. It proposes 2 Recurrent neural networks (RNN) based model: one to work with the histogram of optical flow features and the other one to take in the dense optical flow directly. Also, visualization of the soft-attention weights is done to add a step that helps in the interpretability of the RNN model incorporating dense optical flow. From the experimental results, optical flow features have shown significant improvements in terms of predicting probabilistic confidence for tracks with some changes in their motion mode. It was seen that the convolution-attention RNN model was able to work with dense optical flow features and position of pedestrians as input to obtain better results among all the combinations of features and models compared in this work
Depth-aware Instance Segmentation with a Discriminative Loss Function
No full textThis work explores the possibility of incorporating depth information into a deep neural network to improve accuracy of RGB instance segmentation. The baseline of this work is semantic instance segmentation with discriminative loss function.The baseline work proposes a novel discriminative loss function with which the semantic net-work can learn a n-D embedding for all pixels belonging to instances. Embeddings of the same instances are attracted to their own centers while centers of different instance embeddings repulse each other. Two limitations are set for attraction and repulsion, namely the in-margin and out-margin. A post-processing procedure (clustering) is required to infer instance indices from embeddings with an important parameter bandwidth, the threshold for clustering. The contribution of the work in this thesis are several new methods to incorporate depth information into the baseline work. One simple method is adding scaled depth directly to RGB embeddings, which is named as scaling. Through theorizing and experiments, this work also proposes that depth pixels can be encoded into 1-D embeddings with the same discriminative loss function and combined with RGB embeddings. Explored combination methods are fusion and concatenation. Additionally, two depth pre-processing methods are proposed, replication and coloring. From the experimental result, both scaling and fusion lead to significant improvements over baseline work while concatenation contributes more to classes with lots of similarities.Mechanical Engineering | Vehicle Engineering | Cognitive Robotic
Multi-class Road User Detection with 3+1D Radar in the View-of-Delft Dataset
No full textNext-generation automotive radars provide elevation data in addition to range-, azimuth- and Doppler velocity. In this experimental study, we apply a state-of-the-art object detector (PointPillars), previously used for LiDAR 3D data, to such 3+1D radar data (where 1D refers to Doppler). In ablation studies, we first explore the benefits of the additional elevation information, together with that of Doppler, radar cross section and temporal accumulation, in the context of multi-class road user detection. We subsequently compare object detection performance on the radar and LiDAR point clouds, object class-wise and as a function of distance. To facilitate our experimental study, we present the novel View-of-Delft (VoD) automotive dataset. It contains 8693 frames of synchronized and calibrated 64-layer LiDAR-, (stereo) camera-, and 3+1D radar-data acquired in complex, urban traffic. It consists of 123106 3D bounding box annotations of both moving and static objects, including 26587 pedestrian, 10800 cyclist and 26949 car labels. Our results show that object detection on 64-layer LiDAR data still outperforms that on 3+1D radar data, but the addition of elevation information and integration of successive radar scans helps close the gap. The VoD dataset is made freely available for scientific benchmarking.Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Intelligent Vehicle
Traffic Information and Learning in Day-to-Day Route Choice
No full textThe aim of the research is to gain insight into the impact of traffic information on day-to-day route choice. The effects of both en-route information and ex-post information are studied. Special attention is given to three other important aspects in day-to-day route choice: learning from experience and information, habits and the effect of uncertainty / reliability. A large experiment was set up and a discrete choice model for panel data was estimated on the resulting data. Conclusions: \u95 The more elaborate traffic information that was provided, the higher the travel time savings that were realized. \u95 Travellers chose less reliable routes more often when they were provided with traffic information. \u95 Whereas travellers who did not receive en-route traffic information based their expected travel time mainly on the most recent experience, travellers with en-route information used more previous experiences. \u95 The role of past choices (used to model implicit learning and habit formation) compared to the role of the expected travel time (used to model explicit learning) was found to be relatively large in the route choice process.Transport & Planning DepartmentCivil Engineering and Geoscience
Context-based path prediction for targets with switching dynamics
No full textAnticipating future situations from streaming sensor data is a key perception challenge for mobile robotics and automated vehicles. We address the problem of predicting the path of objects with multiple dynamic modes. The dynamics of such targets can be described by a Switching Linear Dynamical System (SLDS). However, predictions from this probabilistic model cannot anticipate when a change in dynamic mode will occur. We propose to extract various types of cues with computer vision to provide context on the target’s behavior, and incorporate these in a Dynamic Bayesian Network (DBN). The DBN extends the SLDS by conditioning the mode transition probabilities on additional context states. We describe efficient online inference in this DBN for probabilistic path prediction, accounting for uncertainty in both measurements and target behavior. Our approach is illustrated on two scenarios in the Intelligent Vehicles domain concerning pedestrians and cyclists, so-called Vulnerable Road Users (VRUs). Here, context cues include the static environment of the VRU, its dynamic environment, and its observed actions. Experiments using stereo vision data from a moving vehicle demonstrate that the proposed approach results in more accurate path prediction than SLDS at the relevant short time horizon (1 s). It slightly outperforms a computationally more demanding state-of-the-art method.Intelligent Vehicle
