1,720,983 research outputs found
Point Pair Feature based Object Detection for Random Bin Picking
No full textPoint pair features are a popular representation for free form 3D object detection and pose estimation. In this paper, their performance in an industrial random bin
picking context is investigated. A new method to generate representative synthetic datasets is proposed. This allows to investigate the influence of a high degree of clutter and the presence of self similar features, which are typical to our application. We provide an overview of solutions proposed in literature and discuss their strengths and weaknesses. A simple heuristic method to drastically reduce the computational complexity is introduced, which results in improved robustness, speed and accuracy compared to the naive approach.sponsorship: IWT (TETRA project RaPiDo, #140358)status: Publishe
Unlocking Dynamic Cameras for Visual Navigation
Get PDFThe ability of a mobile robot to navigate reliably in an unknown environment depends on the information perceived from its surroundings, usually available in the form of measurements from sensors. However, most of the current sensor configurations assume a static rigid transformation between the sensors, placing many limitations on its performance, especially when undergoing high dynamic motion profiles. A Dynamic Camera consists of a camera mounted to an actuated mechanism such as a gimbal and offers several advantages compared to the static case. However, In order to accurately relate information across these cameras and other sensors, a precise time-varying extrinsic calibration needs to be determined. In this dissertation, I present novel methods to calibrate a Dynamic Camera to other sensors such as static cameras and IMUs. First, I analyze a Dynamic Camera Cluster system, and present an information theoretic Next-Best-View formulation that directly minimizes the uncertainty on the calibration parameters and test the resulting calibration in a visual-inertial odometry application. Second, I present a novel analysis of the redundant parameters in the calibration system when joint angle values to the actuated mechanism are unavailable and propose a novel minimal parameterization to resolve the degeneracies. Third, I present the online calibration between a Dynamic Camera and an IMU, and analyze the observability properties of the calibration parameters. I perform extensive tests in simulation and in the real world, consisting of a 3-DOF gimbal and a 5-DOF manipulator, demonstrating our ability to recover an accurate calibration, thereby allowing Dynamic Cameras to be used in visual navigation applications.Ph.D
Multi-Channel Generalized-ICP: A robust framework for multi-channel scan registration
No full textAbstractCurrent state of the art scan registration algorithms which use only position information often fall victim to correspondence ambiguity and degeneracy in the optimization solutions. Other methods which use additional channels, such as color or intensity, often use only a small fraction of the available information and ignore the underlying structural information of the added channels. The proposed method incorporates the additional channels directly into the scan registration formulation to provide information within the plane of the surface. This is achieved by calculating the uncertainty both along and perpendicular to the local surface at each point and calculating nearest neighbor correspondences in the higher dimensional space. The proposed method reduces instances of degenerate transformation estimates and improves both registration accuracy and convergence rate. The method is tested on the Ford Vision and Lidar dataset using both color and intensity channels, as well as with Microsoft Kinect data from the Freiburg RGBD Office dataset and data obtained from the University of Waterloo campus
Monocular 3D Object Detection and 3D Multi-object Tracking for Autonomous Vehicles
Get PDFIn this thesis, we first tackle the monocular 3D object detection task. The main challenge in monocular 3D detection lies in predicting depth, which must be inferred due to the lack of range measurement. We propose CaDDN, which predicts a categorical depth distribution for each pixel to project feature information in 3D space. We then use the bird's-eye-view projection and single-stage detector to produce the final output detections. Next, we tackle the 3D multi-object tracking task, where associating tracks to detections remains challenging. Our proposed solution, InterTrack, introduces the Interaction Transformer to generate discriminative object representations for data association. We extract track and detection features, and aggregate global information via attention. We then use each track/detection feature pair to estimate affinities, and use a two-stage data association and track management approach to produce the final tracks. We perform validation on autonomous vehicle datasets and achieve state-of-the-art performance.M.A.S
Leveraging Pre-trained Detection Networks for Label Generation on New Datasets
No full textThe development of 3D object detection solutions for autonomous driving requires labelled real-world datasets for training and evaluation, which are costly to produce. This thesis presents a 3D click labelling approach to individual scene labelling, in which annotators are provided a specially designed user interface to click on objects in the point cloud of each scene. The click labels are subsequently processed by a pre-trained network to generate a label for each click. The proposed annotation scheme demonstrates a 30x lower human annotation time. This thesis also presents a method for automatic labelling of static objects through the use of full drive sequences. In this work proposals from a pre-trained detection network, over the course of a drive sequence, are transformed into a global reference frame. Overlapping detections are processed to produce high recall label proposals for static vehicles in the sequence, eliminating the need for manual annotation.M.A.S
Two Methods for Robust Robotiocs Perception: Visual Measurement Integrity Monitoring for Localization and Confidence Guided Stereo 3D Object Detection
Get PDFSafe autonomous robot operations require robust localization and accurate 3D object detection. This thesis covers two methods to improve the robustness in these two tasks. The first half of this thesis introduces an approach to monitor the integrity of the state estimation in UAV localization by calculating the protection level of the localization solution, i.e. the largest possible translational error. The protection level is validated using the EuRoC dataset and is demonstrated to provide a more reliable bound than the commonly used 3 method. The second half of this thesis proposes a novel stereo-based object Safe autonomous robot operations require robust localization and accurate 3D object
detection. This thesis covers two methods to improve the robustness in these two tasks.
The first half of this thesis introduces an approach to monitor the integrity of the state
estimation in UAV localization by calculating the protection level of the localization
solution, i.e. the largest possible translational error. The protection level is validated
using the EuRoC dataset and is demonstrated to provide a more reliable bound than
the commonly used 3? method. The second half of this thesis proposes a novel stereobased object detection network named CG-Stereo. This pipeline uses separate decoders
for foreground and background depth estimation and leverages the confidence estimation
from the depth estimation network in the detection phase. The proposed method outperforms all the existing stereo-based 3D object detection methods on the challenging
KITTI benchmark in all categories for all classes.M.A.S
Active Perception for Estimating 6D Poses of Textureless Shiny Objects
Get PDFTextureless shiny objects, such as metal parts, are essential components of many products. Detecting and estimating the 6D poses of these objects is an important and challenging task in many robotic applications. Shiny surfaces can introduce spurious edges in RGB images and missing depth measurements even when using industrial-grade cameras, resulting in a significant drop in the object pose estimation performance. A solution to this problem is to place the camera at different viewpoints and fuse multi-view measurements to improve the performance. However, due to the strict cycle-time requirement in many applications, such as robotic bin-picking, collecting numerous viewpoints is often impractical and inefficient. In this dissertation, we present novel methods to actively predict the next best camera viewpoint to improve the pose estimation performance of textureless shiny objects. First, we present an active eye-in-hand calibration approach for the robot manipulator to acquire the accurate multi-view camera poses. Second, we use the calibrated robot manipulator to collect a large-scale, real-world dataset for shiny objects and provide benchmarks for various robotic perception tasks (e.g., object detection and pose estimation). Third, we present a multi-view depth fusion approach to improve the depth data quality for shiny objects, which are then used as the input for object pose estimation. Forth, we present two active perception approaches to predict the next-best-view for the depth acquisition and the object pose estimation, respectively. Finally, we extend our active perception formulation to the RGB-based approach and expand our research problem to the 6D pose estimation for general textureless objects.Ph.D
Learning-Driven Visual Anomaly Segmentation for Space Applications
Get PDFNASA's Lunar Gateway will demand unprecedented robotic autonomy, requiring systems like Canadarm3 to detect environmental hazards without human oversight. This visual anomaly segmentation task is complicated by extreme lighting, dynamic perspectives, and the lack of suitable datasets. To address this, we first introduce the Anomaly Localization in Lunar Orbit (ALLO) dataset, a large-scale photorealistic benchmark that reveals the shortcomings of current unsupervised methods. We then propose FlowCLAS, a hybrid framework that enhances normalizing flows for scene-level anomaly segmentation. By integrating a contrastive loss with Outlier Exposure, FlowCLAS enforces clear separation between normal and anomalous features within the probabilistic model's latent space. Experiments show FlowCLAS achieves new state-of-the-art performance on ALLO and multiple driving anomaly benchmarks, including Fishyscapes, Road Anomaly, and SegmentMeIfYouCan. These results highlight FlowCLAS as a versatile framework, advancing both space autonomy and terrestrial anomaly segmentation.M.A.S
Out of Distribution Detection via Normalizing Flows for Open World Machine Learning
Get PDFOut-of-distribution (OOD) detection is a critical task for safe deployment of learning systems in the open world setting. Traditional machine learning systems are trained with a “closed world” assumption: they are optimized for a fixed set of training data, validated against a set of reserved data, and then deployed to the “open world” - where they may encounter unfamiliar data. In this work, we investigate the use of feature density estimation via normalizing flows for OOD detection and present a fully unsupervised approach which requires no exposure to OOD data, avoiding researcher bias in OOD sample selection. This is a post-hoc method which can be applied to any pretrained model, and involves training a lightweight auxiliary normalizing flow model to perform the out-of-distribution detection via density thresholding. Experiments on OOD detection in image classification show strong results for far-OOD data detection with only a single epoch of flow training, including 98.2% AUROC for ImageNet-1k vs. Textures, which exceeds the state of the art by 7.8%. We additionally explore the connection between the feature space distribution of the pretrained model and the performance of our method. Finally, we provide insights into training pitfalls that have plagued normalizing flows for use in OOD detection.M.A.S
3D Object Detection and Depth Completion for Scene Perception
No full text3D object detection is a fundamental component in the autonomous driving perception pipeline. While images provide rich semantic information, depth information is lost during perspective projection. LiDAR scans provide accurate depth measurements, but the data is sparse at longer distances. In this thesis, we first propose a method to fuse LiDAR and image features in an end-to-end trainable network to achieve robust, real-time 3D detection performance. Next, to alleviate the sparsity of LiDAR data, we propose a fast, CPU based, depth completion algorithm using only classical image processing techniques, which outperforms several learning based methods. The densified depth map is then used to synthesize virtual views of pedestrians, which are incorporated into a proposed orientation estimation pipeline to improve pedestrian heading estimation. Finally, a monocular 3D detection neural network is proposed, which leverages proposal regression and the task of instance reconstruction using densified object point clouds.M.A.S
- …
