787 research outputs found
Synthesis quality prediction model based on distortion intolerance
No full textFree-viewpoint video system will provide viewers with freedom to navigate through the scene at different viewpoints. In the system, arbitrary viewpoints of videos are synthesized by the depth image-based rendering with multi-view plus depth videos. Despite the widespread of technologies for free-viewpoint video system, the field of quality assessment for the free-viewpoint video, especially the quality prediction of a synthesized image, has not yet been thoroughly investigated. This paper analyzes how distortions in color and depth images influence on the quality of a synthesized image. Then, an objective quality prediction model for a synthesized image is proposed based on the concept of intolerance of synthesis distortion. Experimental results show that the proposed model provides outstanding performance in predicting the quality of a synthesized image compared to other models
Pedestrian proposal generation using depth-aware scale estimation
No full textIn this work, we propose an efficient method that generates pedestrian proposals suitable for the autonomous vehicle. Our main intuition is that depth information provides an important cue to assign the scale of pedestrian proposals. Based on the observation that in a 3-D world coordinate the scales of pedestrians are almost similar, we formulate the scales of pedestrian patches by projecting 3-D models to an image plane with its corresponding depth. We also introduce a scale-aware binary description using both color and depth images. By using this descriptor, the regression models are trained to rank the pedestrian proposal candidates and adjust the proposal bounding boxes for an accurate localization. Our algorithm achieves significant performance gains compared to conventional proposal generation methods on the challenging KITTI dataset
Robust corner detector based on corner candidate region
No full textCorner detection is a fundamental step for many computer vision applications to detect the salient image features. Recently, FAST corner detector has been proposed to detect the high repeatable corners with efficient computational time. However, FAST is very sensitive to noise and detects too many unnecessary corners on the noise or texture region. In this paper, we propose a robust corner detector improved from FAST in terms of the localization accuracy and the computational time. First, we construct a gradient map using the Haar-wavelet response by integral image for efficiency. Second, we define a corner candidate region which has large gradient magnitude enough to be corner. Finally, we detect the corner on the corner candidate region by FAST. Experimental results show the proposed method improves localization accuracy measured by the repeatability than standard FAST and the-state-of-art methods. Moreover, the proposed method shows the best computation efficiency. Especially, the proposed method detects the corners more accurately in the image containing many texture regions and corrupted by the Gaussian noise or the impulse noise. © 2013 IEEE
High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion
No full textWe address the problem of 3D reconstruction from uncalibrated LiDAR point cloud and stereo images. Since the usage of each sensor alone for 3D reconstruction has weaknesses in terms of density and accuracy, we propose a deep sensor fusion framework for high-precision depth estimation. The proposed architecture consists of calibration network and depth fusion network, where both networks are designed considering the trade-off between accuracy and efficiency for mobile devices. The calibration network first corrects an initial extrinsic parameter to align the input sensor coordinate systems. The accuracy of calibration is markedly improved by formulating the calibration in the depth domain. In the depth fusion network, complementary characteristics of sparse LiDAR and dense stereo depth are then encoded in a boosting manner. Since training data for the LiDAR and stereo depth fusion are rather limited, we introduce a simple but effective approach to generate pseudo ground truth labels from the raw KITTI dataset. The experimental evaluation verifies that the proposed method outperforms current state-of-the-art methods on the KITTI benchmark. We also collect data using our proprietary multi-sensor acquisition platform and verify that the proposed method generalizes across different sensor settings and scenes.
High-Precision Depth Estimation with the 3D LiDAR and Stereo Fusion
No full textWe present a deep convolutional neural network (CNN) architecture for high-precision depth estimation by jointly utilizing sparse 3D LiDAR and dense stereo depth information. In this network, the complementary characteristics of sparse 3D LiDAR and dense stereo depth are simultaneously encoded in a boosting manner. Tailored to the LiDAR and stereo fusion problem, the proposed network differs from previous CNNs in the incorporation of a compact convolution module, which can be deployed with the constraints of mobile devices. As training data for the LiDAR and stereo fusion is rather limited, we introduce a simple yet effective approach for reproducing the raw KITTI dataset. The raw LiDAR scans are augmented by adapting an off-the-shelf stereo algorithm and a confidence measure. We evaluate the proposed network on the KITTI benchmark and data collected by our multi-sensor acquisition system. Experiments demonstrate that the proposed network generalizes across datasets and is significantly more accurate than various baseline approaches
Unified multi-spectral pedestrian detection based on probabilistic fusion networks
No full textDespite significant progress in machine learning, pedestrian detection in the real-world is still regarded as one of the challenging problems, limited by occluded appearances, cluttered backgrounds, and bad visibility at night. This has caused detection approaches using multi-spectral sensors such as color and thermal which could be complementary to each other. In this paper, we propose a novel sensor fusion framework for detecting pedestrians even in challenging real-world environments. We design a convolutional neural network (CNN) architecture that consists of three-branch detection models taking different modalities as inputs. Unlike existing methods, we consider all detection probabilities from each modality in a unified CNN framework and selectively use them through a channel weighting fusion (CWF) layer to maximize the detection performance. An accumulated probability fusion (APF) layer is also introduced to combine probabilities from different modalities at the proposal-level. We formulate these sub-networks into a unified network, so that it is possible to train the whole network in an end-to-end manner. Our extensive evaluation demonstrates that the proposed method outperforms the state-of-the-art methods on the challenging KAIST, CVC-14, and DIML multi-spectral pedestrian datasets. (C) 2018 Elsevier Ltd. All rights reserved.
Fast affine-invariant image matching based on global Bhattacharyya measure with adaptive tree
No full textEstablishing visual correspondence is one of the most fundamental tasks in many applications of computer vision fields. In this paper we propose a robust image matching to address the affine variation problems between two images taken under different viewpoints. Unlike the conventional approach finding the correspondence with local feature matching on fully affine transformed-images, which provides many outliers with a time consuming scheme, our approach is to find only one global correspondence and then utilizes the local feature matching to estimate the most reliable inliers between two images. In order to estimate a global image correspondence very fast as varying affine transformation in affine space of reference and query images, we employ a Bhattacharyya similarity measure between two images. Furthermore, an adaptive tree with affine transformation model is employed to dramatically reduce the computational complexity. Our approach represents the satisfactory results for severe affine transformed-images while providing a very low computational time. Experimental results show that the proposed affine-invariant image matching is twice faster than the state-of-the-art methods at least, and provides better correspondence performance under viewpoint change conditions
Convolutional feature pyramid fusion via attention network
No full textWe present a novel fusion scheme between multiple intermediate convolutional features within convolutional neurual network (CNN) for dense correspondence estimation. In contrast to existing CNN-based descriptors that utilize a single convolutional activation, our approach jointly uses multiple intermediate features of CNN through the attention weight that balances the contribution of each features. We formulate the overall network as two sub-networks, correspondence network and attention network. The correspondence network is designed to provide multiple intermediate matching costs while the attention network is to learn the optimal weight between them. These two networks are learned in a joint manner to boost the correspondence estimation performance. Experiments demonstrate that our proposed method outperforms the state-of-the-art methods on various correspondence estimation tasks including depth estimation, optical flow, and semantic correspondence
LAT: Local area transform for cross modal correspondence matching
No full textEstablishing correspondences is a fundamental task in many image processing and computer vision applications. In particular, finding the correspondences between a non-linearly deformed image pair induced by different modality conditions is a challenging problem. This paper describes a simple but powerful image transform called local area transform (LAT) for modality-robust correspondence estimation. Specifically, TAT transforms an image from the intensity domain to the local area domain, which is invariant under nonlinear intensity deformations, especially radiometric, photometric, and spectral deformations. Experimental results show that LATransformed images provide a consistency for nonlinearly deformed images, even under random intensity deformations. LAT reduces the mean absolute difference by approximately 0.20 and the different pixel ratio by approximately 58% on average, as compared to conventional methods. Furthermore, the reformulation of descriptors with LAT shows superiority to conventional methods, which is a promising result for the tasks of cross-spectral and modality correspondence matching. LAT gains an approximately 23% improvement in the correct detection ratio and a 10% improvement in the recognition rate for the tasks of RGB-NIR cross-spectral template matching and cross-spectral feature matching, respectively. LAT reduces the bad pixel percentage by approximately 15% and the root mean squared errors by 13.5 in the task of cross-radiation stereo matching. LAT also improves the cross-modal dense flow estimation task in terms of warping error, providing 50% error reduction.
Memory-guided Image De-raining Using Time-Lapse Data
No full textThis paper addresses the problem of single image de-raining, that is, the
task of recovering clean and rain-free background scenes from a single image
obscured by a rainy artifact. Although recent advances adopt real-world
time-lapse data to overcome the need for paired rain-clean images, they are
limited to fully exploit the time-lapse data. The main cause is that, in terms
of network architectures, they could not capture long-term rain streak
information in the time-lapse data during training owing to the lack of memory
components. To address this problem, we propose a novel network architecture
based on a memory network that explicitly helps to capture long-term rain
streak information in the time-lapse data. Our network comprises the
encoder-decoder networks and a memory network. The features extracted from the
encoder are read and updated in the memory network that contains several memory
items to store rain streak-aware feature representations. With the read/update
operation, the memory network retrieves relevant memory items in terms of the
queries, enabling the memory items to represent the various rain streaks
included in the time-lapse data. To boost the discriminative power of memory
features, we also present a novel background selective whitening (BSW) loss for
capturing only rain streak information in the memory network by erasing the
background information. Experimental results on standard benchmarks demonstrate
the effectiveness and superiority of our approach
- …
